Docker Ingress with Caddy
Some things I like:
- Caddy
- Docker (well, containers at any rate)
- Having a little box of blinking lights in my basement
- Running my own services for personal use
Part of what I need to do to run services at home is what’s called “ingress”:
- Route hostnames to the appropriate local service
- Handle TLS certificates
- Do so without exposing inner services or creating problems
I’ve decided to solve this by running Caddy inside my Docker and having a shared network connecting all the public-facing containers to it. (This is similar to how Docker Swarm does it.)
Architecture
I’m building up my environment through a series of compose projects. I suggest putting them in a global directory: /etc/compose (used by my systemd-compose bodge) or /opt/docker-configs or whatever, one directory per project.
Compose
First: The compose file.
services:
caddy:
container_name: caddy
build: .
restart: always
ports:
- 80:80/tcp
- 443:443/tcp
- 443:443/udp
volumes:
- caddy-data:/data/caddy
networks:
- ingress
volumes:
caddy-data:
networks:
ingress:
name: ingress
This is largely a straight-forward Caddy compose: 80/443 for HTTP(S) (UDP for HTTP/3), caddy-data volume for certs and other cache.
The interesting thing here is the definition of the ingress network: this is referenced by other compose projects.
Dockerfile
This is also a pretty standard xcaddy setup.
FROM caddy:builder AS builder
WORKDIR /project
RUN xcaddy build \
--with github.com/caddy-dns/acmeproxy@v1.1.1
FROM caddy:latest
COPY --from=builder /project/caddy /usr/bin/caddy
COPY Caddyfile /etc/caddy/Caddyfile
VOLUME /data/caddy
I do recommend just doing this from the get-go: You need to compile caddy to install any plugins, and any sufficiently advanced setup will have non-standard plugins.
As for the acmeproxy plugin I’m using, see my post on acmeproxy.
The one thing you may want to change is that I’m including the Caddyfile in the image build. You may not want to do this, in which case, add a bind mount to the volumes in the above compose file. (Tip: Don’t bind mount the file directly, a bunch of editors will break that. Create a subdirectory and bind mount that.)
Caddyfile
The actual Caddyfile is largely a matter of taste. It needs to container site entries for every container you’re exposing, as well any options you want to add. (eg, logging, acme email, etc)
Usage
For each service you expose, you’ll need a compose project an a snippet of Caddyfile
The compose needs some specific invocations, which we can demonstrate with testserver:
services:
testserver:
image: docker.io/httptoolkit/testserver
container_name: testserver
networks:
- ingress
restart: always
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
ingress:
external: true
A few critical things:
- The external
ingressnetwork has to be declared—this will refer to the ingress network defined above - Connect the web app to the
ingressnetwork - Set
container_name:on the web app—this must be unique to the server, and is used by the Caddyfile below -
restart:andextra_hosts:should be set as practice
The Caddyfile snippet is similarly straightforward:
http://test.qwertyuiop.home {
reverse_proxy testserver:3000
}
- Use the above
container_nameas the host forreverse_proxy- Also, set the port correctly
- If this is private, you need to declare it as http or set up ACME DNS-01
- If your application needs additional configuration, do it here
- If you want to access your application by relative domain or by IP, be sure to add them