In the previous post, I wrote an extensive guide on deploying Cloudflare as the upstream DNS for Pi-Hole over HTTPS. This is a follow-up where Cloudflare is replaced with Unbound as the upstream DNS server.

Unbound is a recursive DNS that sits between Pi-Hole and authoritative DNS servers. Cloudflare's 1.1.1.1 and Google's 8.8.8.8 are examples of recursive DNS services. By making Unbound the upstream DNS server for Pi-Hole, you're cutting out other third parties from tracking your web presence. A more detailed read-up of this setup can be found on the official Pi-Hole Unbound guide.

Back on June 11, 2021, Cloudflare DNS experienced outages in the Los Angeles and Chicago area. The result was over an hour of downtime with its DNS service. I was able to avoid that outage by switching over to Unbound and letting it handle domain resolution directly with authoritative DNS servers. This method can be a little slow but DNS caching in Pi-Hole becomes beneficial on subsequent lookups.

Deploying Unbound with Pi-Hole

In the previous post, Pi-Hole and Cloudflare DNS were deployed using Docker Swarm and managed through Portainer with Traefik as the reverse proxy. This will follow the previous guide closely. Let's start by cloning the project:

git clone https://github.com/foureight84/traefik-pihole-doh.git && cd traefik-pihole-doh

This guide assumes that your Docker Swarm, Portainer, and Traefik have been properly configured. If not, follow this guide.

Go to your Portainer web portal and click on App Templates -> Custom Templates and click on the "+ Add Custom Template" button.

This stack uses Unbound Docker image created by Kyle Harding (https://github.com/klutchell)
Image: https://hub.docker.com/r/klutchell/unbound
Github: https://github.com/klutchell/unbound-docker

You'll need to fill in a relevant title for the template. I called mine recursive_dns. Add a description - Pi-hole and Unbound. Make sure the template Type is set to Swarm. Then click on the Upload option and choose the docker-compose.yaml in the dns-unbound folder in the cloned project.

After it has been uploaded, find the newly created custom template in the list of templates and click edit.
We will need to check that PIHOLE_DNS_=172.18.0.1#5053 environment variable matches your docker_gwbridge IPV4 IPAM Gateway address. Once verified, deploy the stack. That's it!

Deploy the stack once the custom template has been uploaded. The klutchell/unbound Docker image now listens on port 53 by default. Setting the PIHOLE_DNS environment variable to the unbound service name is all that's needed.

While uncached DNS queries may be slower than using Google's public DNS (8.8.8.8) we can see that Pi-Hole's caching outpaces all other public DNS services by far. Plus the millisecond differences in uncached queries are not noticeable in a real use case scenario. It's actually faster than using Cloudflare!

  192.168.  1.  4 |  Min  |  Avg  |  Max  |Std.Dev|Reliab%|
  ----------------+-------+-------+-------+-------+-------+
  + Cached Name   | 0.000 | 0.001 | 0.001 | 0.000 | 100.0 |
  + Uncached Name | 0.015 | 0.060 | 0.187 | 0.052 | 100.0 |
  + DotCom Lookup | 0.015 | 0.049 | 0.081 | 0.022 | 100.0 |
  ---<O-OO---->---+-------+-------+-------+-------+-------+
                     pihole.home
                Local Network Nameserver


    1.  1.  1.  1 |  Min  |  Avg  |  Max  |Std.Dev|Reliab%|
  ----------------+-------+-------+-------+-------+-------+
  - Cached Name   | 0.012 | 0.013 | 0.018 | 0.001 | 100.0 |
  - Uncached Name | 0.014 | 0.069 | 0.355 | 0.077 | 100.0 |
  - DotCom Lookup | 0.014 | 0.022 | 0.048 | 0.009 | 100.0 |
  ---<-------->---+-------+-------+-------+-------+-------+
                     one.one.one.one
                    CLOUDFLARENET, US
                    
    8.  8.  8.  8 |  Min  |  Avg  |  Max  |Std.Dev|Reliab%|
  ----------------+-------+-------+-------+-------+-------+
  - Cached Name   | 0.012 | 0.015 | 0.023 | 0.002 | 100.0 |
  - Uncached Name | 0.014 | 0.038 | 0.158 | 0.040 | 100.0 |
  - DotCom Lookup | 0.014 | 0.016 | 0.025 | 0.002 | 100.0 |
  ---<-------->---+-------+-------+-------+-------+-------+
                       dns.google
                       GOOGLE, US

I've been using Pi-hole as a whole network ad blocker for a while now and it's been great. The only mistake I made the first time was using it as a standalone docker instance. Whenever I perform a manual update to a newer release, there would be a few minutes of internet downtime. Personally, it's not a big deal and I could just force a DHCP-renew. But I share my internet connection with my parents living next door and an internet outage immediately leads to a phone call.

There are various ways to update a running Docker container and minimize downtime, and I've chosen to update my stack to use Docker Swarm to accomplish this goal. Yes, the setup is overkill for a home network, but making things causal is always better down the road when you need to make changes.

I am currently using a J5005 Dell Wyse 5070 thin client as my server, but this article should apply to a Raspberry Pi. You should take a look at the Docker images used and swap them for ARM-compatible.

Disclaimer: This is meant for home use only. DO NOT deploy this in a Production environment. Pi-hole uses SQLite and concurrent write is not supported. While DNS resolution and blocking won't have issues, there will be loss of log data if two or more swarm nodes are writing to DB at the same time. Also, make sure your router is properly configured to not expose your DNS to the public internet. It will crash and burn.

Stack Overview

• Management Stack (mgmt)
  • Traefik - reverse proxy and load balancer
  • Portainer - Web UI Docker management tool
  • Whoami - Tiny Go webserver that prints os information and HTTP request to output
• DNS Stack (dns)
  • Cloudflared DoH Proxy
  • Pi-hole DNS adblocker and DHCP server

A few Details

Traefik is a GoLang reverse proxy and load-balancer with built-in support for Let's Encrypt. In my earlier post about using Terraform to deploy this Ghost blog, I used jwilder/nginx-proxy along with jrcs/letsencrypt-nginx-proxy-companion to serve as a reverse-proxy and automatic SSL certificate handling. However, that setup is on a Docker standalone deployment using Docker Compose. While it is possible to use an Nginx reverse-proxy with Docker Swarm, Traefik is a much better fit with Docker Swarm with fewer configurations involved.

Portainer is a container management tool with a web UI. It's used in this project to manage the DNS swarm, which makes scaling and service updates more casual. I personally prefer this to keep the project simple and doesn't require looking up Docker CLI commands.

Pi-hole, the main reason for this project, is a Linux based DNS server with built-in DHCP support. The main use for Pi-hole is to block advertisement domains, but it also has additional benefits in blocking known malware and phishing domains as well. As it is a DNS server, it can be used to apply to an entire network.

Getting Started

We'll first need to clone the project

git clone https://github.com/foureight84/traefik-pihole-doh.git && cd traefik-pihole-doh

You'll need to enable Docker swarm on your host if not already.

docker swarm init

We will need to create a Docker overlay network called traefik. Additional Docker services you choose to add later that require the use of Traefik will need to be attached to this network.

docker network create --driver=overlay --attachable traefik

Now we spawn the mgmt stack and use Portainer to deploy and maintain swarms from here on out.

docker stack deploy -c mgmt/docker-compose.yaml mgmt

If your server is running Ubuntu >= 18.04, chances are, systemd-resolved is probably occupying port 53. We'll need to turn that off.

Update your hosts file with the hostnames for the running web services. Windows: C:\Windows\System32\drivers\etc\hosts, Linux/Mac: /etc/hosts/. For my setup, my home network has the domain of .home. You will need to change this for your setup. 192.x.x.x denotes the server this docker stack is currently running on.

192.x.x.x	portainer.home
192.x.x.x	traefik.home
192.x.x.x	whoami.home
192.x.x.x	pihole.home

Traekfik dashboard is available via http://traefik.home/local/dashboard/ don't forget the / at the end. You should see entry points for port 80 as well as 53 tcp and udp.

Go to Portainer via http://portainer.home and create your admin account. Once logged in, we will need to create a Docker Secret that will hold the web password for Pi-Hole web ui. The secret needs to be named pihole_webpw.

Make note of the docker_gwbridge network's gateway IPV4. This is used to forward DNS requests to the upstream Cloudflared DoH docker container. We can't use the IP the container gets assigned on the traefik network since that is not static.

Open the docker-compose.yaml in the dns folder with a text editor and update the DNS1 field with your docker_gwbridge's IP address.

environment:
      - TZ=America/Los_Angeles
      - DNS1=172.18.0.1#5054 #replace with docker_gwbridge's gateway ip
      - DNS2=no
      - REV_SERVER=true
      - REV_SERVER_CIDR=192.168.1.0/24 #Update these fields to match your environment
      - REV_SERVER_TARGET=192.168.1.1
      - REV_SERVER_DOMAIN=home
      - WEBPASSWORD_FILE=/run/secrets/pihole_webpw

The REV_* environment variables should also be updated to match your setup as well. REV_SERVER and related fields allow Pi-Hole to perform a reverse DNS lookup against the router. I'm currently using my router's DHCP server (REV_SERVER_TARGET). Static ip and hostnames are kept on the router. Requests for my local NAS for example, will go through Pi-Hole and that will get forwarded to my router.

After making changes to the dns/docker-compose.yaml file, we will need to upload this to Portainer as a docker-swarm template and deploy the swarm.

It should take a few minutes for Pi-Hole deployment to complete. Update the DNS on your computer with your server's IP and test before applying it as the DNS to use on your router. To check if DoH is working properly, head to https://1.1.1.1/help

Update Services

Since we are running in Docker Swarm mode, we can take advantage of rolling updates to minimize downtime. Of course, you need more than one instance of a service running in order to perform a rolling update.

We first need to change the --update-delay flag on the following services (or any that you want to apply a rolling update):

• Traefik
• Pi-hole
• Cloudflared

We will use 120s (2 minutes) as the delay from one service instance to the next. Pi-Hole and Cloudflare services can take up to 30 seconds to reach ready-state and process new requests. 120 seconds should give us ample time for a new Docker image to download and update the first instance while the other instances remain unchanged and continue to process new requests.

In the Service details view, click on Update configuration on the right-hand menu to scroll to the section shown in the image above. Update Delay should be entered as 2m or 120s.

To update a service, scroll to the top and click on Update the service and toggle Pull latest image version on the confirmation screen. Now Docker will update each running instance one-by-one 120 seconds apart.

If you've noticed from the Service list view screenshot, I am only running 1 instance for all my services since it's a home network and I don't need the redundancy or load balancing at the moment (although I might scale Pi-hole and Cloudflared to 2 instances each as these containers have halted in the past). These services will be scaled to 2 or 3 before performing a service update in order to utilize the rolling update. Once all instances are updated, I will downscale them back to the initial scale.

Happy adblocking!

FAQ

Q: Where can get more blocklists?

Filterlists is a good place to start (https://filterlists.com/)

Q: What the hell are you on about?

For anyone starting from scratch, the initial steps not mentioned in this guide are:

1. Installing Ubuntu server (https://ubuntu.com/download/server), for Raspberry Pi (https://ubuntu.com/download/raspberry-pi)
2. Enable SSH after booting into Ubuntu: sudo apt install openssh-server. This will allow remote terminal login to your server via ssh. e.g. ssh your-username@your-server-ip-address from another computer with an SSH client. Windows 10 has SSH built-in or you can use PuTTY.
3. Installing Docker-CE: https://docs.docker.com/engine/install/ubuntu/

You do not need to install Ubuntu for the Rasberry Pi, there's also Raspian OS. Just make sure to do steps 2 and 3.

Q: How do I deal with the port 53 conflict error with the Traefik container?

There are two approaches to this. I will guide based on Ubuntu. You can use this as a guide for your system.
First, we need to find out what's using port 53 and determine if we can stop, or disable it. If you can't then read further for the alternative.

sudo netstat -pna | grep 53

If you're on Ubuntu 18.04 (I believe) and newer, it will most likely be caused by systemd-resolved.

If it is then:

1. sudo systemctl stop systemd-resolved to stop the service
2. sudo nano /etc/systemd/resolved.conf
3. Uncomment #DNS= and replace it with our DNS of choice. It shouldn't matter as this is for your host. I chose cloudflare. It should then look like this DNS=1.1.1.1
4. Uncomment #DNSStubListener=no or add the line if it's not there.
5. Save and exit the text editor.
6. sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf creates a symbolic link and replaces the current link in the existing path.
7. Check that DNS still works by doing a ping github.com
8. If you see an error about sudo: unable to resolve host ubuntu: Name or service not known then edit your hosts file and add 127.0.0.1 your-machine-name below 127.0.0.1 localhost.

If you cannot stop the service that's currently using port 53 then you can update the mgmt stack docker-compose.yaml to use docker's loopback.

traefik:
  image: traefik:latest
  ports:
    - target: 53
      published: 53
      protocol: tcp
      mode: host
    - target: 53
      published: 53
      protocol: udp
      mode: host
    - target: 80
      published: 80
      protocol: tcp
      mode: host

Unfortunately, IP address format is not supported in long-form port declaration. It will need to change to:

traefik:
  image: traefik:latest
  ports:
    - 0.0.0.0:53:53/udp
    - 0.0.0.0:53:53/tcp
    - target: 80
      published: 80
      protocol: tcp
      mode: host

A caveat of using this method is that Traefik won't be able to forward the real IP of the requestor in the request header for X-Forwarded-For and X-Real-Ip. That IP will be a docker NAT IP (test this out using http://whoami.test.local). It's not important for our purpose, but if you were to deploy another web service that utilizes this then it could be an issue.

Q: Pi-Hole client id in the query log only shows Traefik's hostname

When a request comes in through a reverse proxy via HTTP, the request's originating IP is passed along in the request headers as X-Forwarded-For and X-Real-Ip. However, with DNS, there's no such flag. There's been a proposal for DNS X-Proxied-For but that's not a standard. https://tools.ietf.org/id/draft-bellis-dnsop-xpf-03.html
Pi-hole only makes notes of which immediate client sent the request, and in this case, it's Traefik. If you really want to make this work then you can implement a macvlan network and attach your Pi-Hole instance(s). This would allow a Docker container to receive an IP on your router's subnet and allow direct communication.
If pi-hole was not in swarm mode then this would not be an issue (this was my previous setup). In swarm mode and behind a load balancer, you would need to allocate a block of IPs to dedicate to the swarm (IP assignment is not supported in swarm mode) while the load balancer would only apply to port 80 traffic to pi-hole's web UI.

Q: How do I test my adblocking capability?

https://d3ward.github.io/toolz/src/adblock.html is a good place to start. It should show you whether you are blocking major players on the DNS level.

You could also turn off your browser ad block extension and go to a known site with embedded ads. https://tweaktown.com is a good place to test.

Without a browser Adblock extension, it doesn't look pretty but it saves on your bandwidth usage since ads do not load at all. Use this in combination with a browser-based adblocker to remove those HTML elements. Without Pi-Hole ads will be hidden with browser adblockers but they would still load.

Table of contents

• Terraform
  • Linode
  • Cloudflare
  • Variables & Definitions
• Docker
  • Ghost Stack
  • Rclone
• Deployment

At the beginning of the pandemic, I decided to build a custom keyboard (post on this later). The project led to me wanting to add a Blackberry trackball and this would require a special way to mount it to the keyboard PCB. After weeks of trying to jury-rigged a mount from common parts to attach the trackball to the keyboard, I realized that it would be much easier to design and 3D print the necessary part.

I also needed a better way to document my projects and starting a blog is better for visibility and accessibility than a markdown README file on Github. Ghost is a good candidate, offering a lightweight straightforward writing platform (with markdown support). There's also a robust list of third-party integrations which would be good down the road if I needed to scale.

I also wanted to be able to easily backup my data in case I needed to move to a different host. At first, I was eyeing an AWS S3 bucket or perhaps Linode's offering called "Object Storage." At the moment, I don't need that much backup space. A free option would be best. This is where Rclone is handy as it would allow me to tarball essential ghost files and have a daily backup. I could also deploy Rclone on another server, such as a local NAS, as an added backup destination.

I chose Terraform in order to automate the setup tasks so that I can easily switch hosts in the future. The one plus side with Terraform is that it uses declarative language to define tasks and this is, to me, a lot easier to read and understand in the future. If you haven't looked at your code in over six months, it may as well been written by someone else.

Project Overview

• Terraform deployment
• Cloudflare proxied DNS
• Docker
  • Nginx reverse proxy
  • Let's Encrypt for automatic SSL
  • Ghost 4
  • Rclone backup blog data to Google Drive

This project can be found on my github. Clone and follow along:

git clone https://github.com/foureight84/ghost-linode-terraform.git && cd ghost-linode-terraform

Terraform

A declarative configuration-based infrastructure as code API wrapper. There are a few pros and cons to consider. Keep in mind that this is my first time using Terraform these are surface-level points observed.

Pros:

• Quick and straightforward (standardized) configuration syntax declaring what tasks to perform.
• Easy to read and double-check my work. Terraform's 'plan' feature to check for syntax errors and preview the final result.
• Configurations can be changed and applied quickly without having to go through service consoles.
• All pieces in my stack can be managed through Terraform (see cons).

Cons:

• Providers often offer web APIs to perform the same tasks, and at best, there will be feature parity. But there is a chance that feature-parity on Terraform is a second priority to a provider and support lags behind.
• Not all services may have Terraform support, which could lead to the additional complexity of having to manage multiple tools. This is often the case in a real-world scenario.

• Applying my Terraform configuration update doesn't always behave as expected. The functionality limitation depends greatly on the provider. For example, after a Linode is created, applying changes to certain properties such as root password will result in Terraform destroying the instance to create a new node using updated configurations.

Before diving into specific files in the project, here is a brief overview and explanation of the project structure:

[..root]
│  cloudflare.tf                   //resource module for CloudFlare rules
│  data.tf                         //templates ex. docker-compose.yaml, bash scripts, etc...
│  linode.tf                       //resource module for Linode instance setup
│  outputs.tf                      //handle specific data to display after terraforming
│  providers.tf                    //tokens, auth keys, etc required by service providers
│  terraform.tfvars.example        //example answers for input prompts 
│  variables.tf                    //defined inputs required for terraforming
│  versions.tf                     //declaration of providers and versions to use
│
└─[scripts]
  ├─[linode]
  │       docker-compose.yaml      //main stack. nginx proxy, letsencrypt, ghost
  │       stackscript.sh           //boot time script specific to Linode to setup env
  │
  └─[rclone]
    │     backup.sh                //cron script for backing up ghost blog directory
    │     docker-compose.yaml      //rclone docker application
    │
    └──[config]
            rclone.conf.example    //rclone configuration for cloud storage

Linode

linodes.tf, stackscript.sh, data.tf

This is a straightforward configuration to create a Linode with the use of Linode's Stackscript. Which is essentially a run-once bash script that gets executed on the first boot.

The stackscript.sh lives in the ghost-linode-terraform/scripts/linode/ directory and is parsed by Terraform as a data template. Terraform's templating syntax needs to be taken into consideration when parsing text files. Most notable are variables and their escape characters. Any $string or ${string} notation will be regarded as a template variable, while $$string and $${string} are escaped bash variables. More about strings and templates.

script = "${data.template_file.stackscript.rendered}"

Alongside the Stackscript, the "linode_instance" resource block also includes stackscript_data property. This is a way of providing data to the one-time boot script. The key-value assignments within this block correspond with the 'User-defined fields' at the top of stackscript.sh.

#!/bin/sh
# <UDF name="DOCKER_COMPOSE" label="Docker compose file" default="" />
# <UDF name="ENABLE_RCLONE" label="(Bool) Flag to turn on RClone Support" default="false" />
# <UDF name="RCLONE_DOCKER_COMPOSE" label="RClone docker compose file" default="" />
# <UDF name="RCLONE_CONFIG" label="RClone configuration file" default="" />
# <UDF name="BACKUP_SCRIPT" label="Backup script" default="" />

stackscript_data = {
    "DOCKER_COMPOSE" = "${data.template_file.docker_compose.rendered}"
    "ENABLE_RCLONE" = var.enable_rclone
    "RCLONE_DOCKER_COMPOSE" = "${data.template_file.rclone_docker_compose.rendered}"
    "RCLONE_CONFIG" = "${data.template_file.rclone_config.rendered}"
    "BACKUP_SCRIPT" = "${data.template_file.backup_script.rendered}"
  }

data "template_file" "docker_compose" {
  template = "${file("${path.module}/scripts/linode/docker-compose.yaml")}"

  vars = {
    "ghost_blog_url" = "${var.ghost_blog_url}"
    "letsencrypt_email" = "${var.letsencrypt_email}"
  }
}

In the snippet above, docker-compose.yaml for the Ghost stack is parsed as a data template and ghost_blog_url and letsencrypt_email variables get evaluated and then passed as a string to the Stackscript at runtime.

At the time of deployment, the Stackscript will be created before the Linode instance. Terraform allows direct referencing named values. This can be seen in the "linode_instance" block where linode_stackscript.ghost_deploy.id is assigned as the stackscript ID to include when creating the Linode instance. Finally, other parsed data templates such the docker-compose.yaml are assigned to stackscript user-defined fields and sent as stackscript_data.

Cloudflare

cloudflare.tf

The Cloudflare configuration is relatively straightforward. Since the foureight84.com domain is already managed by Cloudflare, a lookup is performed and the named value is referenced with each resource call as zone_id property.

data "cloudflare_zones" "ghost_domain_zones" {
  filter {
    name   = var.cloudflare_domain
    status = "active"
  }
}

An 'A' record is created for the blog (blog.foureight84.com) and end-to-end HTTPS encryption is enforced (ssl="strict") along with requiring all HTTPS origin pull requests only come from Cloudflare. Nginx-proxy-companion will generate a validation file under the path blog.foureight84.com/.well-known/<some random string> for SSL certification requests. Let's Encrypt validation of this generated file accepts both HTTP and HTTPS (ports 80 and 443), a page rule is created to ensure that the request does not get blocked.

Variables and Definitions

variables.tf

As seen in linode.tf, cloudflare.tf, and especially data.tf, ${var.<some string>} are used throughout. These are references to declared variables in variables.tf such as API tokens for our services, domain names, and etc. These variables show up as input prompts when performing Terraform plan, apply, or destroy actions. Variables contain properties such as data type, descriptions, default values, and custom validation rules. More about input variables.

terraform.tfvars.example

This Terraform deployment requires 17 data inputs in order to perform its tasks and that's 17 possible chances to introduce error. Luckily, Terraform supports dictionary referencing in the form of .tfvars files. Tfvars are key-value files where the key is the variable name. A .tfvars file is referenced during run-time in order to provide required inputs. For example:

terraform plan -var-file="defined.tfvars"

Docker

This project has two separate docker-compose environments. The first is our blog stack, and the second is Rclone to perform data backup. I decided to separate the Rclone docker service from the primary Ghost stack for two reasons:

1. I want to be able to mount the cloud storage prior to running the Ghost stack so that data restore can be performed should a backup exists (see stackscript.sh).
2. Cloud drive mount should always stay active. Changes to my Ghost stack should not impact Rclone's availability.

Ghost Stack

docker-compose.yaml

This runs 3 containers using the following images:

• jwilder/nginx-proxy
• jrcs/letsencrypt-nginx-proxy-companion
• ghost:4-alpine

The default directory is /root/ghost. This can be changed via Terraform during deployment from the tfvars file. Since this is a set default value, Terraform does not prompt for the input value.

jwilder/nginx-proxy

This is the "front" facing container on the origin server (Linode) sitting behind Cloudflare. Visitors' incoming requests for https://blog.foureight84.com will go through Cloudflare, which is forwarded to our reverse-proxy, where the request is relayed to the running Docker service matching the VIRTUAL_HOST value. Finally, nginx-proxy will collect the served content from the Ghost service and send it back to the visitor.

The reverse proxy will be listening to exposed ports 80 and 443.

jrcs/letsencrypt-nginx-proxy-companion

Nginx-proxy-companion handles automatic SSL registration for the running Docker services. The important environment variables to keep in mind are:

environment:
  - LETSENCRYPT_HOST=${ghost_blog_url}
  - LETSENCRYPT_EMAIL=${letsencrypt_email}

The above environment variables are added to Docker services that require SSL certificates (Ghost container in this example). In the future, if I need to add additional subdomains, such as www.foureight84.com, then that service will require LETSENCRYPT_HOST=www.foureight84.com. I can avoid having to repeatedly define LETSENCRYPT_EMAIL by setting the DEFAULT_EMAIL environment variable in the nginx-proxy-companion instead.

By default, a production certificate will be requested. Let's Encrypt has a limit of 10 cert requests every 7 days for normal users. All requested certs are stored in the path /etc/acme.sh which is mounted to the acme Docker volume.

The nginx-proxy and nginx-proxy-companion share certs, vhost.d, and nginx.html volumes. If I need to move to a different host, then the files store in certs and acme docker volumes will need to be backed up. The former is where the generated private key used for SSL certificate requests is stored, whereas the latter is contains the generated certificates and checked against upon service startup. Without the original private key then the SSL certificate cannot be renewed and without SSL certificate would force a new request.

Avoid performing a docker volume prune or docker system prune without proper backup. They are technically not essential unless the weekly certificate request limit has been reached. To get the path to these volumes, run the terminal command docker volume inspect <volume name>.

ghost:4-alpine

This is an all-in-one image. I believe prior versions used MySQL. With Ghost v4, SQLite is now the recommended DB. This works out better for my requirement as it is easier to backup.

ports:
  - 127.0.0.1:8080:2368

I am just remapping the default port 2358 to 8080. This will be internal and not required as the Ghost service will be not directly exposed to public traffic. As mentioned earlier this will be handled by the reverse proxy where incoming requests will be directed to the matching VIRTUAL_HOST.

Rclone

docker-compose.yaml

The default directory is /root/rclone but can be changed using Terraform tfvars at runtime.

Rclone is a command-line application that enables cloud storage to be mounted on the host filesystem. I believe it supports over 30 well-known services such as Google Drive, Dropbox, AWS S3, Amazon Drive, etc. I decided to stick with Google Drive since I have 100GB of underutilized storage. I plan on incorporating Rclone in a TrueNAS SCALE self-built NAS as a redundant backup in the future.

Here are the steps to setting up Google Drive with Rclone:

• Start with this: https://rclone.org/drive/#making-your-own-client-id
  • Don't forget to submit app for verification. It will be an automatic approval. The generated auth token will not renew if the app is in development mode.
• Then follow this guide to attach the Google Drive account to Rclone: https://rclone.org/drive/

volumes:
  - ${rclone_dir}/config:/config/rclone
  - ${rclone_dir}/mount:/data:shared
  - /etc/passwd:/etc/passwd:ro
  - /etc/group:/etc/group:ro

/etc/passwd and /etc/group mounts are required for FUSE to work properly inside the container. Additionally, a premade configuration from another Rclone instance is needed. This can be done by completing a Rclone setup wizard for Google Drive. Make sure to copy the configuration to the project's folder: ghost-linode-terraform/scripts/rclone/config/

In my setup, the Google Drive configuration is called 'gdrive.' This needs to reflect in the docker-compose's command block:

command: "mount gdrive: /data"

The default mount path is /root/rclone/mount.

backup.sh

This script is responsible for creating tarballs from the ghost blog directory on the host machine. By default, the script maintains a rolling 7-day backup with latest.tgz being the last run archive.

A crontab is added through Linode's Stackscript and is set to run daily at 11PM (system time). Keep in mind that UTC is the default system time zone.

Deployment

If you have not done so, install Terraform CLI on your localhost. Follow this installation guide.

Clone project

git clone https://github.com/foureight84/ghost-linode-terraform.git && cd ghost-linode-terraform

Initialize Terraform workspace

terraform init

Create your tfvars definition file

cp terraform.tfvars.example defined.tfvars

Open defined.tfvars and fill in all required values

//project_dir = ""                                // default /root/ghost
//rclone_dir = ""                                 // default /root/rclone

linode_api_token = ""
linode_label = ""
linode_image = "linode/ubuntu20.04"               // see terraform linode provider documentation for these values
linode_region = "us-west"                         // see terraform linode provider documentation for these values
linode_type = "g6-nanode-1"                       // see terraform linode provider documentation for these values
linode_authorized_users = [""]                    // user profile created on linode with associated ssh pub key. https://cloud.linode.com/profile/keys
linode_group = "blog"
linode_tags = [ "ghost", "docker" ]

linode_root_password = ""

cloudflare_domain = ""                           // requires that your domain is already managed by cloudflare. value ex: foureight84.com
cloudflare_email = ""
cloudflare_api_key = ""                          // not to be mistaken with cf api token

letsencrypt_email = ""

ghost_blog_url = ""                              // ex. blog.foureight84.com

enable_rclone =                                  // boolean (default false). change to true if using rclone. see README.md in rclone directory on how to setup config beforehand

Double-check that everything is correct and get a deployment preview

terraform plan -var-file="defined.tfvars"

Apply Terraform changes to production

terraform apply -var-file="defined.tfvars"