[OLD] DevOps your HomeLab

Hey guys! 👋

In the past days I’ve forced myself to work on stuff that I’ve been procastinating for months (like every project 🙄👉👈).
The basic need for me was to automate all that kind of provisioning/setup tasks for my Raspberry Pi that I use as a tiny server, and I’ve decided to use two new software or IaC tools/languages:

• Ansible used for standard provisioning of the server (like creating folders, self-signed certificates and installing docker)
• Terraform used to create docker images and container on the server’s docker engine

There isn’t a specific reason on why I’ve decided to use these two softwares, I guess they’re really popular and I wanted to learn new things. Also, this is my first post on the blog so I apologize if there are any typos or so.

To maintain everything as clean as possible I’ve created a private Gitlab repo to store all my convenient scripts.

Chapters

• Chapter 0x00: Setting everything up
• Chapter 0x01: Translating docker-compose into Terraform
• Chapter 0x02: Validating and testing Terraform script
• Chapter 0x03: Ansible playbooks and community Roles reuse
• Chapter 0x04: Run Ansible Playbook
• Chapter 0x05: Final Thoughts

0x00. Setting everything up

In order to start writing my terraform and ansible script I’ve installed those 2 tools following the official guides on their websites. Obviously I went for the most convenient and fastest way which my current operating system (Ubuntu) offers me.

Installing Terraform

sudo apt-get update && sudo apt-get install -y gnupg software-properties-common curl
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
sudo apt-get update && sudo apt-get install terraform

Installing Ansible

sudo apt update
sudo apt install software-properties-common
sudo add-apt-repository --yes --update ppa:ansible/ansible
sudo apt install ansible

0x01. Translating docker-compose into Terraform

First thing first, I found a community-developed provider for Terraform which is Terraform Provider for Docker.

Terraform providers
For those who are asking, providers in Terraform are like plugins who know how to interact with cloud providers or specific APIs, like in this case, the docker provider knows how to talk to the Docker daemon APIs in order to pull images, create container and so on…

Then I created a new folder with a basic Terraform file just to specify the provider that I’m going to use

terraform {
  required_providers {
    docker = {
      source  = "kreuzwerker/docker"
      version = "2.16.0"
    }
  }
}

provider "docker" {
}

Then with the commandterraform initI’ve initialized the directory and automagically 🪄✨ Terraform downloads the provider.

Translating docker-compose

Since I had somedocker-compose.ymlfiles writter to specify the type of images and containers to create on my server I used them to read all the properties and write a.tffile that would pull the images and set up the containers.

Thedocker-compose.ymlI had were like:

services:
    run:
        container_name: nginx
        volumes:
            - '/etc/nginx/nginx.conf:/etc/nginx/nginx.conf:ro'
            - '/etc/nginx/ssl/cert.crt:/etc/nginx/ssl/cert.crt'
            - '/etc/nginx/ssl/cert.key:/etc/nginx/ssl/cert.key'
        ports:
            - '8443:8443'

What I’m going to add next to the Terraform script is the SSH configuration to the docker provider to create the containers on my Raspberry remotely.

terraform {
  required_providers {
    docker = {
      source  = "kreuzwerker/docker"
      version = "2.16.0"
    }
  }
}

provider "docker" {
+    host = "ssh://<user>@<IP>:22"                   
+    ssh_opts = ["-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=/dev/null"]                               
}

And to get the same result in Terraform I went through the docker provider documentation and written this nginx.tf:

terraform {            
    required_providers {             
        docker = {    
            source = "kreuzwerker/docker"      
            version = "2.16.0"                              
        }                                              
    }                                              
}

provider "docker" {
    host = "ssh://<user>@<IP>:22"                   
    ssh_opts = ["-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=/dev/null"]                               
}

resource "docker_image" "nginx" {
    name = "nginx:latest"
}

resource "docker_container" "nginx" {
    image = docker_image.nginx.latest
    name = "nginx"
    restart = "unless-stopped"

    volumes {
        container_path = "/etc/nginx/nginx.conf"
        host_path = "/etc/nginx/nginx.conf"
        read_only = true
    }

    volumes {
        container_path = "/etc/nginx/ssl/cert.crt"
        host_path = "/etc/nginx/ssl/cert.crt"
    }

    volumes {
        container_path = "/etc/nginx/ssl/cert.key"
        host_path = "/etc/nginx/ssl/cert.key"
    }

    ports {
        external = 8443
        internal = 8443
        protocol = "tcp"
    }
}

If you were wondering, the volumes are for setting nginx proxy settings and to use also self-signed certificates.

0x02. Validating and testing Terraform model

1. Validate the Terraform model

   terraform validate
   

2. Test the Terraform script In order to test Terraform scripts you can either:
   • directly on the server w/docker installed

     terraform apply
     

   • locally using a local installation of docker
     Comment these two lines in order to test it locally

     ...
     provider "docker" {
     -    host = "ssh://<user>@<IP>:22"                   
     -    ssh_opts = ["-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=/dev/null"]                               
     }
     ...
     

Docker containers should have been created 🐳

0x03. Ansible playbooks and community Roles reuse

Now that the docker containers provisioning part has been completed it’s time to move on to the previous one Ansible 🌩️ This part is the one that drove me a little bit crazy, partly cause it was the first time writing ansible and partly because it was late I guess.

The basic things I wanted to achieve here were:

1. Install docker on the Raspberry
2. Upload the nginx proxy configuration
3. Generate a Self-Signed Certificate for HTTPS connections

To do so, I’ve made some research and found extremely magical things called “Ansible Roles” which basically are independent piece of Ansibe tasks, files, variables etc.. and can be written and made public by the open-source community.

To address the first goal I’ve used the Docker for ARM written by @geerlingguy (which has also a great Youtube channel in my opinion)

Installing the role is pretty easy and straight-forward, when installing Ansible in Chapter. 0x00 a utility called ansible-galaxy got installed too.

Ansible Galaxy It’s a utility to manage community shared roles like the one I’m going to use.
To install the community docker role:

ansible-galaxy install geerlingguy.docker

A basic playbook to just install docker on ARM would look like this:

- hosts: rpi

  vars:
    pip_package: python3-pip
    pip_install_packages:
      - name: docker

  roles:
    - geerlingguy.docker_arm

and this basically installs docker on the hosts you’re running this playbook against.

But I needed more than this since my configuration requires also an nginx.conf and a self-signed certificate. So I’ve written tasks myself 😄

By referring to the omniscent G and the Ansible Docs I’ve managed to create tasks like these:

• Copy nginx.conf to the remote host and setting the owner (assumes the configuration file is placed in the same directory of the playbook)

...
tasks:
 - name: Copy nginx configuration
   copy:
     src: "{{ playbook_dir }}/nginx.conf"
     dest: "{{ nginx.nginxPath + 'nginx.conf'}}"
     owner: "{{ owner }}"
...

Those you see in curly brackets like {{ playbook_dir }} and {{ owner }} are Ansible variables.

• {{ playbook_dir }} is a reserved variable that is used by Ansible itself referring to the directory’s playbook.
• {{ owner }} is a user-defined variable that contains a value (in this case a string with the owner)

Ansible Variables
Generally speaking you can define variables in a playbook like this

...
  vars:
+    owner: test
+    nginx:
+       nginxPath: /etc/nginx
+       sslPath: /etc/nginx/ssl
+   cert:
+       certificateName: test_certificate
+       csrName: test_csr
+       privateKey: test_privatekey
    pip_package: python3-pip
    pip_install_packages:
      - name: docker

  roles:
    - geerlingguy.docker_arm
...

Here for nginx I’ve defined a nested type of variables that you can refer to as {{ nginx.nginxPath }} which contains the /etc/nginx string.

Then using the community.crypto collection I created a private key, CSR (Certificate Signing Request) and finally the certificate itself.

tasks:
  - name: Create private key (RSA, 4096 bits)
    community.crypto.openssl_privatekey:
      path: "{{ nginx.sslPath }}{{ cert.privateKey }}.key"
  
  - name: Generate an OpenSSL Certificate Signing Request
    community.crypto.openssl_csr:
      path: "{{ nginx.sslPath }}{{ cert.csrName }}.csr"
      privatekey_path: "{{ nginx.sslPath }}{{ cert.privateKey }}.key"
      common_name: "{{ cert.certificateName }}"
  
  - name: Generate Certificate
    community.crypto.x509_certificate:
      path: "{{ nginx.sslPath }}{{ cert.certificateName }}.crt"
      privatekey_path: "{{ nginx.sslPath }}{{ cert.privateKey }}.key"
      csr_path: "{{ nginx.sslPath }}{{ cert.csrName }}.csr"
      provider: selfsigned

So far this is what you should have:

- hosts: rpi

  vars:
    owner: "test"
    
    nginx:
        nginxPath: /etc/nginx/
        sslPath: /etc/nginx/ssl/
    pip_package: python3-pip
    pip_install_packages:
      - name: docker

  roles:
    - geerlingguy.docker_arm

  tasks:
   - name: Copy nginx configuration
     copy:
       src: "{{ playbook_dir }}/nginx.conf"
       dest: "{{ nginx.nginxPath + 'nginx.conf'}}"
       owner: "{{ owner }}"
  
    - name: Create private key (RSA, 4096 bits)
      community.crypto.openssl_privatekey:
        path: "{{ nginx.sslPath }}{{ cert.privateKey }}.key"
  
    - name: Generate an OpenSSL Certificate Signing Request
      community.crypto.openssl_csr:
        path: "{{ nginx.sslPath }}{{ cert.csrName }}.csr"
        privatekey_path: "{{ nginx.sslPath }}{{ cert.privateKey }}.key"
        common_name: "{{ cert.certificateName }}"
  
    - name: Generate Certificate
      community.crypto.x509_certificate:
        path: "{{ nginx.sslPath }}{{ cert.certificateName }}.crt"
        privatekey_path: "{{ nginx.sslPath }}{{ cert.privateKey }}.key"
        csr_path: "{{ nginx.sslPath }}{{ cert.csrName }}.csr"
        provider: selfsigned

0x04. Run Ansible Playbook

Finally to run an ansible playbook is easy as:

ansible-playbook main.yml

Although if this is you’re first time trying running an ansible playbook, probably you’ll get an error 😃 That’s because you’ll need to specify in your Ansible inventory /etc/ansible/hosts some hosts under the tag [rpi]

[rpi]
10.10.10.1
10.10.10.2

Otherwise you can specify the host directly in the playbook

- hosts: 10.10.10.1

You can re-try theansible-playbook main.ymlcommand to run the playbook.

0x05. Final Thoughs

Now what you’re left with is some really basic theory of Terraform and Ansible. These two software are really powerful and I guess they take time to master, although what I really wanted to share with this first post was the whole process that made me get myself into this project and not a bunch of technical details.

I hope you’ve found this post useful or just interesting, and that you’ll get inspired by this to learn new things as I did here even just out of curiosity. 😄
Stay secure!