Provisioning Load Balancer on AWS Instances using Ansible

Hey Guy, Hope you all are well in this pandemic time, learning and growing each day.

Now, Let me continue with the part where I left right now If anyone does not about part one go to my Blog where I explain about how we can use ansible for Provisioning Web server over the AWS cloud using Ansible and discussed cloud computing in another blog link to these blogs mentioned below.

What is cloud computing?

Launch Web-Server On AWS using Automation

Now, Let me continue from where I left so last time I showed you all how to provision a web server over an AWS cloud using Ansible.

But this time I got stuck by a company demand or you can a used case where they find launching one webserver wasn’t enough as one web server can handle a limited amount of request and the company was losing their client so they want to launch more webserver but there is a problem in that part, if we launched too many servers then also are the problem is unsolved as how the client will know which server to reach basically the problem of load balancing arises, how to provide a client with one IP address so they reach the same place where they required to reach.

So, the solution to that part was launching a Load Balancer and providing all the webservers IP to that Load balancer so that this Load balancer handles the request and guides the traffic correctly and effectively.

This helped the company a lot but now a launching a load balancer and then the webservers and manually proving the IP to the load balancer isn’t a waste of time.

Yes, It is so I bring you all the automated solution to such a problem which will not only help you to launch but also help the companies to manage servers if any server got down they do not require to manually enter the IP instead re-initialize the webserver and provide the IP of web server dynamically using dynamic Inventory to load balancer.

The load balancer I am using here is Haproxy.

Haproxy or High Availability Proxy is used by the Right Scale for load balancing in the cloud. Haproxy is installed with RightScale load balancer ServerTemplates. Load-balancer servers are also known as front-end servers. Generally, their purpose is to direct users to available application servers.

It works by using each server behind the load balancer in turns, according to their weights. It’s also probably the smoothest and most fair algorithm as the servers’ processing time stays equally distributed. As a dynamic algorithm, Round Robin allows server weights to be adjusted on the go.

Load balancing is defined as follow:

A performance optimization tactic that provides fault-tolerance by splitting incoming tasks across multiple computing resources, thereby preventing anyone from the resource from becoming overloaded.

Since I have already all the concepts used in today’s task in the previous blog therefore I am directly moving on to the practical part today.

Please configure your System exactly same as we did in previous blog of provisioning the webserver over the AWS cloud and the most Important for this Task is Dynamic Inventory. Please follow all the resolution of all the errors provided in previous blog if commited.

The given roadmap below will give oversight of how Ansible will automate the whole process and after provision how load balancer will work.

Copyright Ayush Gupta

Now, Let’s define or break the task into some parts:

Deploy a Load Balancer and multiple Web Servers on AWS instances through ANSIBLE!

TASK DESCRIPTION

→Provision EC2 instances through ansible.

→ Retrieve the IP Address of instances using the dynamic inventory concept.

→Configure the web servers through the ansible role.

→Configure the load balancer through the ansible role.

→The target nodes of the load balancer should auto-update as per the status of web servers.

Now, Lets’ get started

1] Provisioning EC2 instances through ansible.

I have used ROLES to complete this task and to also make it easier for the person going to use it.

NOTE: Guy’s I will perform this task on the Redhat Linux but the same will work with other OS too little bit of configuration might differ.

1] Start with the Installation of boto software

Boto is the Amazon Web Services (AWS) SDK for Python. It enables Python developers to create, configure, and manage AWS services, such as EC2 and S3. Boto provides an easy to use, object-oriented API, as well as low-level access to AWS services.

pip3 install boto

2] Launch AWS instance using Ansible Playbook

For launching AWS we require to pass credentials of AWS account and as the credentials are something highly confidential so we have one module in AWS named ansible-vault remarkably ansible uses AES256 encryption which is till date most secure algorithm used for encryption. Ansible encryption is used for YAML files.

→ For Encryption

ansible-vault encrypt <filename.yaml>

Here we have encrypted files with passwords. Let us check whether it shows us the password without asking the vault password. It will not show file content without asking for the vault password.

→ For Decryption

ansible-vault decrypt <filename.yaml>

→ For viewing, editing Data

ansible-vault view <filename.yaml>
ansible-vault edit <filename.yaml>

Now let's maintain are code using the role module and this will also help us to automate the things in single Ad-hoc command,

ansible-galaxy init<rolename>

An example is shown below

Inside this role, you can see ansible itself created some predefined folders so we will be specifying variables, tasks in specified files

vars/main.yml     → Variable declaration
template/main.yml → Dynamic Files
tasks/main.yml    → here main working of the role to be written
handlers/main.yml → tasks that you want to run or to be called when                         some condition meets 
README.md         → Is used in the role for instruction or as description file of the role for the user to make changes or how to use that role.
we have other folders to but right now knowing about them not required for completing this task

Role For Launching EC2 Instance over the AWS cloud under comput as a service.

I have given role name as aws_instances,

1} vars folder has two files one which contains general variables used in the role another file that is required for the confidential data required as variables.

Variables need to be passed in JSON format.

2} tasks/main.yml

Now, I am gonna create a new role for Setting Up LoadBalancer over the instance which we launched on the AWS cloud.

1} tasks/main.yml

2} handler/main.yml

3} templates/haproxy.cfg

Now, I am gonna create a new role for Setting Up Web Servers over the instances which we launched on the AWS cloud.

1} tasks/main.yml

Ansible playbook for running above roles

Now, the main part of the whole tasks comes the execution

Before execution of Roles using Playbook let us resolve some configuration for ansible so as to prevent further errors

1} Passing roles path to ansible, So for that we require to make some configuration in the ansible conf file.

Address of Conf file of Ansible

Underlines are very important for ansible to be used while using for AWS

->In the above file, we specified the inventory folder as /hosts [which contain all the info required for ansible to connect to a Managed Node]        ---->remote_user for telling ansible to go as a user ec2-user 
-> roles_path= used to specify the path for ansible to search for all the roles used inside the playbook in that particular folder
->private_key_file= is specified for ansible so that AWS should grant access to ansible for making changes inside the AWS cloud instance.

Now we need to download a file for dynamically fetch the IP of the AWS instance link as follows

Command given below I run in /hosts folder which I passed to ansible as it's inventory folder now use the command to fetch the required file.
wget https://raw.githubusercontent.com/onkar2701/ansible-task-2/master/ec2.py                                             
wget https://raw.githubusercontent.com/onkar2701/ansible-task-2/master/ec2.ini

Now one you need to change inside the ec2.py file,

Add in the first line of the file

#!/usr/bin/python3

Now, make the file ec2.py and ec2.ini executable and remember both the files must be placed at the same location inside the /hosts folder which we passed as inventory folder inside the ansible.

chmod +x ec2.py
chmod +x ec2.ini

for checking that if the ec2.py is executable or not

Now, we need to export some variables for dynamic inventory

Commands need to be run inside the /root folder
export EC2_INI_PATH=path_of_ec2.ini_file
export AWS_ACCESS_KEY_ID="aws_access_key"
export AWS_SECRET_ACCESS_KEY="aws_secret_key"
export AWS_REGION='app-south-1'

Since exporting this variable is not a permanent setting so better we should set these above commands in a file and make that file secured and ask the bash shell to run it whenever it restarts the system [This basically an idea which one can do because the maximum time we forget to do the same By the way I am not sharing that here]

Also, The private key which we passed inside the ansible conf file that also needs to be executable

chmod 400 <keyaddress/keyname.pem>

So we need to work carefully while passing the credentials, now after launching the instance ansible now will use the ec2.py and ec2.ini file for dynamically fetching the IP of the instance. After fetching the IP ansible stores the IP of the instance in a folder which I put at the location /hosts [We need to create a file inside this folder and then make this file executable using command ]

chmod +x /hosts/host

now ansible will create an entry in the above file and will use this entry for configuring web-server over the instance.

Since, we know how all the process gonna work now let's go to the task and run what we created,

Rub this command in the same place where the playbook is kept.

ansible-playbook --ask-vault-pass <playbook>

Now, Running the Playbook

Showing that instances are successfully launched over the AWS cloud

Haproxy file saved over the LoadBalancer

→ LoadBalancer is working correctly

Thank You Guy's

The Task is successfully completed !!!