Self-hosting handbook: A docker-compose tutorial
Welcome to the second page of a handbook on self-hosting. Begin here.
Topics covered on this page
- How a reverse proxy works and why you need one
- Creating and understanding docker-compose files
- Fleshing out your docker-compose file
- Hitting the “up -d” button on your self-hosted stack
You’re being overcharged for your cloud computing.
We fixed that. For a limited time, during our 7-year anniversary sale, save $2,592 with a 16GB RAM server. Deploy with honest value from your cloud hosting provider. ⚡Get limited-time pricing: →
How a reverse proxy works and why you need one
A reverse proxy is a web server that accepts incoming traffic, analyzes it, and directs it to the appropriate backend service/server. Reverse proxies are often used as load balancers or caching servers, but for the sake of this handbook, we’re mostly interested in how a reverse proxy can accept traffic from multiple domains, all directed toward the same VPS, and deliver the appropriate content in turn.
Let’s look at this example diagram:
ssdnodes.com ----| |-- LAMP stack + PHP files | reverse proxy | x.ssdnodes.com --|--- @ ---|-- WordPress blog | 123.456.78.90 | y.ssdnodes.com --| |-- Node.js web app
In this case, we have three domains—
y.ssdnodes.com—that are all pointed toward a VPS at the IP address
123.456.78.90. We also have three backend services—a LAMP stack, a WordPress blog, and a Node.js web app—that we want to serve.
When someone types one of those three domains into their browser’s navigation bar, they send a request to the VPS via DNS (remember DNS from the previous page?). The reverse proxy receives this request, analyzes it, and recognizes that the request is looking for
ssdnodes.com. The reverse proxy then sends a request to the LAMP stack, which returns an HTML page. Finally, the reverse proxy “forwards” that HTML page (proxies it, actually, hence the name) to the user’s browser.
When they type
x.ssdnodes.com into their browser, the reverse proxy returns the WordPress blog, and when the type
x.ssdnodes.com into their browser, they get the Node.js app in return.
In this way, a reverse proxy allows you to host multiple websites or web apps from a single VPS.
I consider a reverse proxy necessary to self-hosting because it’s inconvenient enough to remember the IP address of your VPS, much less the various ports on which each of your services will be running. Also, the reverse proxy will request and serve the Let’s Encrypt SSL certificate that will allow you to have HTTPS-enabled websites and apps.
Why is SSL important? Expand
SSL, which stands for Secure Sockets Layer, is the standard technology for encrypting traffic between a browser and web server. SSL is the encryption standard your bank’s website, for example, will use to ensure that no one can “sniff” your traffic and discover your bank account number, password, or other identifying information.
In the past, SSL was used primarily on websites that stored personal information about its users, like bank accounts, but also email, social media, e-commerce, and others. Many small sites, like personal blogs, didn’t use SSL, as it was difficult to set up and often cost hundreds of dollars per year.
Enter Let’s Encrypt, which delivers free, automated SSL certificates to anyone with a valid domain name (like
example.com). A Let’s Encrypt certificate allows you to encrypt the traffic entering and leaving the self-hosting stack on your VPS.
No matter what you’ll ultimately be hosting on your VPS via this handbook, SSL is critical. If you’re going to host a website or blog, for example, Google will severely dock your SEO scores without SSL enabled. If you have any personal information stored on your VPS, SSL will help prevent anyone from reading that information. Even if you’re only self-hosting a private Git server like Gitea, SSL ensures full encryption between your server and your browser.
Creating and understanding docker-compose files
Now that you have some context for what we’re trying to build with this self-hosting stack, it’s time to take a look at the primary tool we’re going to use moving forward:
You installed this program on the previous page.
docker-compose is a “wrapper” around the
docker command-line program itself, and uses configuration files called
docker-compose.yml to launch multiple Docker containers instead of typing out long commands with
Log into your VPS if you’re not already, create a new
proxy directory in your home directory, and
cd into it.
$ mkdir -p ~/proxy $ cd ~/proxy
Create a new file using the editor of your choice. I use
nano for these kinds of tasks, but you may prefer to use
emacs instead. Just replace
nano with your editor of choice from here on out.
$ nano docker-compose.yml
Let’s start with building out the “skeleton” of a
docker-compose.yml file, which will then allow us to talk about how each component works, and the syntax behind it. Inside the new file, type in the following.
version: '2' services: volumes: networks:
docker-compose expands and becomes more feature-rich, it needs to evolve and adopt new ways of explaining how users can deploy a Docker-based infrastructure. The maintainers infrequently introduce a new version of the syntax to accommodate these adaptations. We’re sticking with the
'2' syntax, even though there’s already a third version
version: '3' available already.
Version 3 introduces a new way of describing volumes that I haven’t gotten working precisely the way I’d like yet. I’ll be sure to update this guide when I have a version 3-compatible version of this
docker-compose.yml file ready.
services section defines which Docker containers you’ll launch when you finally do run
docker-compose. It’s the most important section we’ll cover in on the rest of this page in the self-hosting handbook!
We will define each service in a separate indented “block.” When we run
docker-compose, it will iterate over these services blocks and download/create/launch each Docker container in sequence.
Volumes are essentially folders that get shared between your user’s filesystem and the Docker container’s filesystem. Think of it as a symbolic link. Or, if you’re not too familiar with those, think about it as a Dropbox folder on your current machine. You put something in that folder, and it appears on the Dropbox web interface, or on another device of yours—the folders are synchronized.
Because volumes are synchronized, you can define various configuration files and make changes to them without diving into the Docker container’s filesystem, which can be quite tricky.
You can also create a single volume that’s shared among many Docker containers, such as using one MySQL database for five different WordPress installations. But that’s, well, not recommended. Not by a long shot.
networks section in a
docker-compose file allows you to define internal networks through which different Docker containers can communicate. We’re going to keep it simple throughout this handbook, with only a single network, but more complex infrastructures might require multiple internal/external networks.
Fleshing out your docker-compose file
Time to build out the
services block, add your first service, which we’re going to call
proxy. Remember that indentation is important with
services label should be aligned fully to the left, while each defined service is indented once, and its contents are indented twice or three times.
services: proxy: image: jwilder/nginx-proxy container_name: proxy restart: unless-stopped labels: com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy: "true" volumes: - /var/run/docker.sock:/tmp/docker.sock:ro - certs:/etc/nginx/certs:rw - vhost.d:/etc/nginx/vhost.d - html:/usr/share/nginx/html ports: - "80:80" - "443:443" networks: - "default" - "proxy-tier"
Let’s take a minute to look at what this
services block accomplishes.
The first line of the block defines a new Docker container. You can name them however you’d like, but let’s keep it simple.
This line defines the Docker image that we’re going to launch—in this case, the wonderful nginx-proxy image.
We’re going to name the container
proxy in this case, and refer to it as that moving forward, but you can name it however you’d like.
labels: com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy: "true"
labels line creates a label so that the Let’s Encrypt container, which we’ll launch in a moment, knows which proxy to connect to.
volumes: - /var/run/docker.sock:/tmp/docker.sock:ro - certs:/etc/nginx/certs:rw - vhost.d:/etc/nginx/vhost.d - html:/usr/share/nginx/html
Here we create some volumes for our
proxy Docker container. First, we connect the Docker
.sock, followed by three additional volumes that create those “synchronized” folders between the
proxy folder we created earlier and various folders inside of the container.
ports: - "80:80" - "443:443"
We’re now defining which ports the Docker container will listen on. By listening on ports
proxy container will accept all web traffic arriving on your VPS, which is exactly what we want—the reverse proxy needs to receive all the traffic so that it can route it correctly.
networks: - "default" - "proxy-tier"
Finally, we’re defining which networks the
proxy container will communicate through. The
default network will handle traffic entering and exiting the VPS, whereas the
proxy-tier network allows all the containers in our self-hosting stack to communicate with one another.
Let’s Encrypt container
Time to add the Let’s Encrypt container to the file.
proxy-letsencrypt: image: jrcs/letsencrypt-nginx-proxy-companion container_name: letsencrypt restart: unless-stopped environment: - NGINX_PROXY_CONTAINER=proxy volumes: - /var/run/docker.sock:/var/run/docker.sock:ro volumes_from: - "proxy" depends_on: - "proxy" networks: - "default" - "proxy-tier"
There are a few new configurations here, namely
depends_on. All these help the Let’s Encrypt container find and connect to the
proxy container we just specified. You shouldn’t need to edit these, so we won’t spend time discussing the nuances of how they work.
I like to add a Portainer container to any self-hosting stack as a means of monitoring my containers and doing some basic administrative work that’s much more difficult using the many options and flags of the
docker command line tool.
As a bonus, this portion of the
docker-compose.yml file will serve as an example of how we’ll eventually set up other self-hosted services.
Time to add one last block to our
portainer: image: portainer/portainer container_name: portainer restart: always environment: - VIRTUAL_HOST=docker.DOMAIN.TLD - LETSENCRYPT_HOST=docker.DOMAIN.TLD - LETSENCRYPT_EMAIL=EMAIL volumes: - ./portainer/:/data - /var/run/docker.sock:/var/run/docker.sock ports: - "9000:9000"
Take a look at the
environment block here, as it’s critical to how this entire stack functions, from the reverse proxy to how Docker containers communicate with one another. You will put a similar block in every new container you add to the
services section of your
There are three essential environment variables to set here: the
LETSENCRYPT_HOST, and the
LETSENCRYPT_EMAIL variable is simplest to explain: Replace the
LETSENCRYPT_HOST both reference the domain at which you’ll access the Portainer service using your browser. Both these variables must be set on every Docker container you want to access via the reverse proxy. On the previous page, you set up a DNS entry that pointed toward
docker.DOMAIN.TLD. Here’s where that DNS entry comes in. Flesh out both those entries with your domain name.
Volumes and networks
Finally, add the following to your
volumes: certs: vhost.d: html: networks: proxy-tier:
We won’t spend too much time on these, as they don’t need to be edited and will never change, but they set up a number of volumes and a single
proxy-tier network that allows containers to communicate amongst themselves.
The full file, for reference
Knowing that copy-pasting or re-typing configuration files is a rather error-prone process, here is the full file for your reference:
version: '2' services: proxy: image: jwilder/nginx-proxy container_name: proxy restart: unless-stopped labels: com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy: "true" volumes: - /var/run/docker.sock:/tmp/docker.sock:ro - certs:/etc/nginx/certs:rw - vhost.d:/etc/nginx/vhost.d - html:/usr/share/nginx/html ports: - "80:80" - "443:443" networks: - "default" - "proxy-tier" proxy-letsencrypt: image: jrcs/letsencrypt-nginx-proxy-companion container_name: letsencrypt restart: unless-stopped environment: - NGINX_PROXY_CONTAINER=proxy volumes: - /var/run/docker.sock:/var/run/docker.sock:ro volumes_from: - "proxy" depends_on: - "proxy" networks: - "default" - "proxy-tier" portainer: image: portainer/portainer container_name: portainer restart: always environment: - VIRTUAL_HOST=docker.DOMAIN.TLD - LETSENCRYPT_HOST=docker.DOMAIN.TLD - LETSENCRYPT_EMAIL=EMAIL volumes: - ./portainer/:/data - /var/run/docker.sock:/var/run/docker.sock ports: - "9000:9000" volumes: certs: vhost.d: html: networks: proxy-tier:
If you copy-paste this entire file, you must still replace the
TLD variables before moving forward.
Hitting the “up -d” button on your self-hosting stack
Now that the
docker-compose.yml file is finished, it’s time to run
docker-compose for the first time. Type the following into your VPS’ terminal:
$ docker-compose up -d
Enter, and you’ll see quite a bit of output as Docker downloads the images. If all goes correctly, you should see the following as the process ends:
Creating network "proxy_default" with the default driver Creating network "proxy_proxy-tier" with the default driver Creating portainer ... done Creating proxy ... done Creating letsencrypt ... done
Let’s do a quick check of our self-hosted stack to make sure that everything is running correctly. Run
docker ps and you should see the following:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES bf830886f0a2 jrcs/letsencrypt-nginx-proxy-companion "/bin/bash /app/entr…" 3 minutes ago Up 3 minutes letsencrypt 5d0845395978 jwilder/nginx-proxy "/app/docker-entrypo…" 3 minutes ago Up 3 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp proxy 10bd13114c5d portainer/portainer "/portainer" 3 minutes ago Up 3 minutes 0.0.0.0:9000->9000/tcp portainer
The output shows that
nginx-proxy, the Let’s Encrypt helper, and Portainer are all running.
Open up a browser and navigate to
docker.DOMAIN.TLD. You should see the Portainer setup process—create an administrative account, log in, and you should be able to see all your containers running in the nice web UI.
If you made it this far, congratulations! You’re already 75% of your way to having a fully-functional self-hosting stack on your VPS.
From here on out, I’ll explain how Let’s Encrypt is working, why that’s important, and how you can add more containers. After that, I’ll start creating a small “database” of example
services blocks for you to pick and choose from.
Next page: Adding more apps to the self-hosting stack
The third page of this self-hosting handbook is now available: Using docker-compose to add web apps.