Will & Skill Developers

Will & Skill Developers


Thoughts, snippets and ideas from the team at Will & Skill AB, Stockholm.

Erik Svedin
Author

Share


W&S devops part 1.5 - Getting started with Ansible, Vagrant and django

Erik SvedinErik Svedin

Before we start, all of the code for this tutorial is available on my github

So you've got a basic understanding of both Vagrant and Ansible after reading part 1 in this series. Now make sure you've installed them both on your local machine. The official sites provide clear enough instructions so I'll just provide some links here for you to go ahead and install them:

  1. Installing Vagrant
  2. Installing Ansible (assuming you have pip installed)

My current setup:

$ vagrant -v
Vagrant 1.7.4  
...
$ ansible --version
ansible 1.9.4  

Setting up our Vagrant machine

Go ahead and create a new directory and jump in to that. From there we will run vagrant init to create our vagrant file:

$ mkdir wands-devops-demo && cd wands-devops-demo
$ vagrant init

This Vagrantfile comes with a whole bunch of commented out stuff which are all optional to configure. What we want to do however is:

  1. Make sure we install Ubuntu 14.04
  2. Forward a port from our guest to the host machine
  3. Kick off provisioning with Ansible
  4. Sync a folder on our host to the guest machine. This will be the folder on our host where the project code, the django app, lives (not yet created, we'll get there).

So step by step in our Vagrant file:

  1. Change config.vm.box = "base" to config.vm.box = "ubuntu/trusty64"
  2. Uncomment the line with config.vm.network and change it to something like: config.vm.network "forwarded_port", guest: 8000, host: 8000
  3. There is a chunk somewhere in the Vagrantfile about provisioning, you can remove all that and replace it with:
config.vm.provision "ansible" do |ansible|  
  ansible.playbook = "provision/vagrant.yml"
end  

4 . Comment out the line with synced folder and replace it with: config.vm.synced_folder "./project", "/home/django/project", mount:false

So now that we got our Vagrantfile set up we're still missing two key parts:
1. The ansible playbooks for provisioning
2. Our actual django project.

Lets go ahead and create or pull an existing django project under a folder called project as described in the Vagrantfile (won't go in to detail here since i assume previous experience with django).

Given a completely fresh django app (wands_demoproject) your folder should now look like this:

├── Vagrantfile
└── project
    ├── manage.py
    └── wands_demoproject
        ├── __init__.py
        ├── settings.py
        ├── urls.py
        └── wsgi.py

So lets go to the final and most exciting part:

Provisioning the Vagrant machine with Ansible

Finally! First of create the following files on the same level as our Vagrantfile:

├── Vagrantfile
└── provision
    ├── provisioner.yml
    └── roles
        ├── deploy
        │   ├── tasks
        │   │   └── main.yml
        │   └── vars
        │       └── main.yml
        └── setup
            ├── tasks
            │   └── main.yml
            └── vars
                └── main.yml

The provisioner.yml will act as our master playbook that vagrant will start. So in our provisioner.yml file add the following:

- hosts: all
  roles:
      - setup

This tells ansible to start the role setup on all hosts. Vagrant and Ansible plays very well together so by defining ansible in our Vagrantfile, vagrant will actually go ahead and create the Ansible hostfile (invetoryfile) for us. So we don't have to worry about that for now. Just assume that it works.

So lets get started with the roles:
Roles are a way of organizing your ansible playbooks by using a certain folder structure to group variables, templates and tasks. More about roles here.

For our setup role go ahead and edit the setup/tasks/main.yml and add the following:

- name: Install packages
  sudo: yes
  apt: pkg={{ item }} state=installed update_cache=yes
  with_items:
      # Database, libpq-dev and python-psycopg2 is needed by ansible
      # to interact with postgres.
      - postgresql
      - libpq-dev
      - python-psycopg2

      # Python
      - python-dev
      - python-setuptools
      - python-virtualenv

      # Redis
      - redis-server

      # Since you're a modern web developer. There is a high
      # chance you're using node in one way or another, so lets install
      # that along with ruby as well.
      - nodejs
      - npm
      - ruby
      - ruby-full

So what is happening here? Lets try to break it down.

  1. First of the name could be anything, it will be printed out to the console when the task is run. Which gives a very nice and verbose way to see whats going on.
  2. sudo: yes, really just instrucs ansible to run this task as sudo.
  3. apt: pkg=.... this is where its getting interesting:
    Ansible comes with a BUNCH of these modules (the module being used here is apt). The modules (also called library plugins) is kind of a shortcut to execute different tasks.

    So traditionally if you manually ssh in to a ubuntu machine you might have done something like this to install postgres:

$ sudo apt-get update
$ sudo apt-get install postgresql

The above task does all that for you using the apt module and an ansible loop. Learn more about loops in ansible here. And really, go check out the list of available modules

Add the following tasks to your setup/tasks/main.yml as well:

- name: Add django user
  sudo: yes
  user: name=django password=django

- name: Create virtualenv
  sudo: yes
  command: virtualenv /home/django/env creates="/home/django/env"

- name: Create Database
  sudo: yes
  sudo_user: postgres
  postgresql_db: name={{ db_name }}

- name: Create User
  sudo: yes
  sudo_user: postgres
  postgresql_user: name={{ db_user }} password={{ db_password }} state=present role_attr_flags=NOSUPERUSER,CREATEDB

- name: Provide user with DB permissions
  sudo: yes
  sudo_user: postgres
  postgresql_user: user={{ db_user }} db={{ db_name }} priv=ALL

You might've noticed there is a bunch of variables regarding db related stuff. You might also recognize the syntax since you're a django user. Ansible uses Jinja2 templates, which will be very familiar if you've worked with django templates previously.

But we still need to add the actual data for our variables. So in our setup/vars/main.yml add the following:

db_name: fortknox  
db_user: fortknox  
db_password: fruitsalad  

So now for the moment of truth, we're actually not quite done yet. But lets go ahead and run

$ vagrant up

to build and automatically provision our machine with our playbooks! Note that the first time this is run it will probably take a while.

If everything is working as intended you should get output similar to:

GATHERING FACTS ***************************************************************  
ok: [default]

TASK: [setup | Install packages] **********************************************  
changed: [default] => (item=postgresql,libpq-dev,python-psycopg2,python-dev,python-setuptools,python-virtualenv,redis-server,nodejs,npm,ruby,ruby-full)

TASK: [setup | Add django user] ***********************************************  
changed: [default]

TASK: [setup | Create virtualenv] *********************************************  
changed: [default]

TASK: [setup | Create Database] ***********************************************  
changed: [default]

TASK: [setup | Create User] ***************************************************  
changed: [default]

TASK: [setup | Provide user with DB permissions] ******************************  
changed: [default]

PLAY RECAP ********************************************************************  
default                    : ok=7    changed=6    unreachable=0    failed=0  

If something went wrong in the provisioning ansible tells you that in a very clear way. You might have just misspelled or forgot to add something, in that case correct your error and run $ vagrant provision to run the Ansible playbook once again.

Starting our django project and get to work

So what now? Our machine is set up and a bunch of dependencies are installed. You can go ahead and $ vagrant ssh to login to your newly created vagrant machine. You should see a virtualenv and your django project under /home/django/ since that's where we synced our project folder.

To actually start the project we're still missing a few pieces though:
1. Make sure our virtualenv is properly set up with our apps dependencies from a requirements.txt file.
2. Make sure we're using the postgres database which we created, also make sure its migrated properly.

In our deploy roles task.yml file add the following:

- name: Setup Virtualenv
  sudo: yes
  pip: virtualenv={{ virtualenv_path }} requirements={{ project_path }}/requirements.txt

- name: Django makemigrations
  django_manage: command=makemigrations app_path={{ project_path }} virtualenv={{ virtualenv_path }}

- name: Django migrate
  django_manage: command=migrate app_path={{ project_path }} virtualenv={{ virtualenv_path }}

Ofcourse we also need to add our variables data in the vars.yml file in our deploy role:

virtualenv_path: /home/django/env  
project_path: /home/django/project  

The first task installs all our django apps dependencies given a requirements.txt file, which we actually dont have yet. So lets go ahead and create it in pur project folder on our host, and add the following:

Django==1.8.6  
psycopg2==2.6  

If you're inside the vagrant machine still, you can actually see that the /home/django/project/ folder now contains a requirements.txt file. Since all changed on the host are synced inside the guest machine. And vice versa.

Now for our last piece of change:

In our settings.py update so that we're using our postgres database with the 'fortknox' user and 'fruitsalad' password as defined in our ansible variable file:

DATABASES = {  
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'fortknox',
        'USER': 'fortknox',
        'PASSWORD': 'fruitsalad',
        'HOST': 'localhost',
        'PORT': '',
    }
}

One last play

So now we got everything set up to start the django project, how do we start the "deploy" role we created with ansible? We could add it to our provisioner.yml file so that it automatically gets run by vagrant. But to get a little more comfortable only using ansible lets go ahead and create another playbook which we simply call deploy_django.yml on the same level as our provisioner.yml. And fill it with:

- hosts: all
  roles:
      - deploy

To manually call ansible playbooks we also actually need a ansible.cfg file. Create it at the project root next to our Vagrantfile and make sure it looks like this:

[defaults]
remote_user = vagrant  
private_key_file = ./.vagrant/default/virtualbox/private_key

# aka inventory file
hostfile = ./.vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory  

This tells ansible about where it actually gets is ssh info from. And also about the different hosts it uses (as mentioned earlier, vagrant creates this for us automatically).

When all that is set up go ahead and from the project root, run:

$ ansible-playbook provision/deploy_django.yml

This should be all! Now if you're not already, log in to the Vagrant guest machine and start up your django project.

$ vagrant ssh
@vagrant...$ cd /home/django
@vagrant...$ source env/bin/activate
@vagrant...$ python project/manage.py runserver 0.0.0.0:8000

Now from your host computer fire up a web browser such and go to: http://127.0.0.1:8000/. You should be met by your django application. Now go ahead and code something amazing! In a following tutorial we will look at how we can take what we've done and actually deploying it to a remote server in the cloud. OHHH .. meanwhile I would love some feedback on this.

Also all the code for this tutorial can be found on my github page:

Get the code

Happy coding!

Erik Svedin
Author

Erik Svedin

Comments