Spegel for p2p docker registries in k3s

I was recently setting up a new k3s cluster for running Atuin. Since I wrote my last note on the subject, a few things have changed!

I definitely need to do a v2 of my post, but in the meantime I learned a bit about Spegel, an optional integrated registry included with k3s.

If server nodes are started with --embedded-registry, then they will setup + run a Spegel registry. This means they host a local OCI registry on port 6443, while connecting to a p2p network over port 5001.

In order for upstream registry mirroring to work, you must edit /etc/rancher/k3s/registries.yaml

For example

mirrors:
  docker.io:
  registry.k8s.io:

enables mirroring of those two registries. This file is read at startup. You can enable wildcard mirroring with "*":, quoted required.

The p2p routing works in a very similar way to BitTorrent - Spegel also uses a Kademlia DHT for resolving digests.

gif sourced from the spegel readme