Design¶
Decisions so far, the features they lead to, and what is still open.
Decisions¶
| # | Decision | Why |
|---|---|---|
| D1 | Image-based, built from the minimal Fedora bootc base. Add only what a gateway needs. | Atomic updates with rollback; a failed update must never take the network down. A small image is less to patch and audit. |
| D2 | Small x86 hardware first (Protectli-class boxes). ARM later. | Most likely target hardware; one architecture keeps early testing simple. |
| D3 | SELinux enforcing, always. Stronghold ships its own policy module; no component may require permissive mode. | Non-negotiable for a gateway. |
| D4 | Cockpit is installed, and Stronghold is a set of Cockpit plugins. | Login, TLS, terminal, logs, updates and accounts come from Cockpit. Stronghold adds the gateway pages. |
| D5 | Firewall is firewalld (on nftables), driven by Stronghold. | firewalld 2.x has zone-to-zone policies, flowtables and rule counters, and libvirt and Podman already integrate with it. If a feature is missing, Stronghold adds a small nftables table of its own for just that. |
| D6 | eBPF is optional, for fast paths and early drop (XDP), not the main rule engine. | The firewall rules stay the source of truth; eBPF is added only where it measurably helps. |
| D7 | HA with keepalived (VRRP) and conntrackd is in scope from the start. | It affects the config model and the firewall, so it can't be added later. |
| D8 | HA supports two WAN modes: VRRP and MAC-move. | Some sites have a static IP block and some get a single DHCP address; both must work. |
| D9 | A lab environment with the needed topologies (static block, single DHCP WAN, HA pairs) is the test bed. | Failover can't be checked on paper; real timings come from the lab. |
| D10 | HA heartbeats run over the SYNC link and a local LAN zone. | Losing one link must not cause split brain, which in MAC-move mode would break the WAN. |
| D11 | Link health monitoring is a core service: interface facts plus active probes (ping, DNS, HTTP). | HA failover, multi-WAN and the UI all need to know whether a link actually works, not only whether it has carrier. |
| D12 | strongholdd and the CLI are written in Go. | Good netlink and HTTP libraries, a single binary, and Fedora's Go packaging allows vendored dependencies. |
| D13 | The config model is YAML, parsed strictly (unknown keys are errors) and validated against a schema, with a top-level version field. | Easy to read, diff and hand-edit; strict parsing avoids YAML's surprise type conversions. |
| D14 | All configuration can be managed as YAML in a git repository. A repository is connected from the UI; the box loads its config from it, and every UI change is committed and pushed back to it. Only signed commits are applied. | Review, history, rollback and many identical sites come from tools people already use. |
| D15 | DHCP is Kea (with Kea DHCP-DDNS); DNS is Unbound as resolver and Knot DNS for local zones. See dhcp-dns.md. |
Leases are registered in DNS (forward and reverse) through standard dynamic updates, and local zones support every record type. |
| D16 | A function runs in a container of its own unless there is a performance or security reason not to. Host functions (interfaces, firewall, NAT, routing, shaping, WireGuard, HA) stay native in the host; service functions (DNS, DHCP, router advertisements, IDS, headscale) get an image each, configured by stronghold-agent. See functions.md. |
The programs that parse what the network sends are confined to their own container; the traffic itself stays in the kernel, so routing speed doesn't change. The same images run on a box, a podman host and Kubernetes. |
| D17 | Service functions keep their standard ports and share a host by address. Any port may be configured, none has to be. | Clients can't be told to use another DNS or DHCP port. |
| D18 | Replaced by D23. In image mode the function containers came with the host image (their images in /usr, their quadlets in /usr/share/containers/systemd), so the host and every function updated and rolled back as one version. |
Every Fedora update needed a new Stronghold image. |
| D19 | FRR (BGP) is a service function, in a container of its own (stronghold-routing), host network, with NET_ADMIN, NET_RAW, NET_BIND_SERVICE and SYS_ADMIN. The kernel's routing stays the host's: zebra writes to it over netlink. |
Its daemons parse what peers send; a container confines them (SELinux, a mount namespace of their own) where frr.service would run them as root with everything. zebra only programs routes, so nothing on the forwarding path changes. |
| D20 | A service function is off, elsewhere or here, and here runs it as a container or as the host's RPM. Elsewhere points the box at a server it doesn't run (clients get its address; DDNS goes there); here, Stronghold runs it, as the route allows: image mode and OpenShift run containers, package mode RPMs by default, a podman host containers. functions.prefer (auto, container or host) is the box's choice where there is one, a function's runtime overrides it. strongholdd alone renders a function's config, whichever route installed it; from the UI it installs RPMs through PackageKit, the Ansible collection with dnf, both then enable the function through the API. See functions.md, "Enabling a function". |
Many sites already have a DNS or DHCP server and want the gateway to use it, not replace it; those that don't get it with one click. Some environments don't allow containers and want packages configured the normal way; others run everything in podman or OpenShift. One renderer keeps commit-with-confirm, history and rollback the same on all of them. |
| D21 | A box from packages installs a function when it is turned on, not before: the stronghold package doesn't pull in the functions' programs; turning one on installs its packages (PackageKit) or, to run it in a container, its quadlet (shipped in the package, installed by strongholdd) and its image (pulled from the functions' registry). See functions.md, "Installing". |
A box has only what it runs; the image (and its ISO) still carries every function, for offline installs. |
| D22 | Every component has a version x.y.z of its own, x.y shared by all: the release line. z is a component's own fixes, y a change that needs testing across them. Images are referred to by their exact version, never latest. See "Versions". |
Which RPMs and which images go together can be read off any box, and a box never runs an image it wasn't tested with. |
| D23 | The host image is Fedora's packages and dnf install stronghold, nothing else: no function's program or image in it. A function's container is installed when it is turned on, in image mode as from packages (D21); the disconnected ISO's image adds the functions' images as a cache, and installs DHCP and the resolver. The host image has a version of its own (the Fedora release and its build), and is rebuilt when Fedora updates. See functions.md, "Image mode", and install.md. |
Fedora's updates (the kernel, firewalld, OpenSSL) reach a box with no new Stronghold, and Stronghold's with no new base; installing is "dnf install stronghold" everywhere, in a Containerfile for image mode. |
| D24 | The web proxy is Squid; the reverse proxy is HAProxy, with no choice of nginx. Both are service functions (proxy.md): Squid caches for the inside networks, explicitly or, for plain HTTP, transparently; HAProxy routes sites by name to servers inside, ending TLS with the box's certificates or passing it through. |
Both are in Fedora and are pfSense's choices for the same jobs. HAProxy checks its servers, reloads without dropping connections and routes TLS by name; open-source nginx has no active checks. Two reverse proxies would be two renderers, policies and test sets for the same features. |
Versions¶
The components are versioned apart, as x.y.z (D22): the stronghold
packages (stronghold.spec), the UI's (stronghold-ui.spec), the Ansible
collection (galaxy.yml), and the images (stronghold-images/VERSION:
the functions' images and the controller). x.y, the release line, is the
same for all of them; scripts/test.sh refuses a build where it isn't.
The host image isn't one of them (D23): its version is the Fedora release
and its build, and it holds whichever Stronghold packages were published
then (a label says which).
- z: a component's own fixes, bumped with each change to it that is published (a new Unbound in Fedora is a new z of the images, nothing else).
- y (z back to 0, all components together): a change that needs testing across them: the agents' protocol, the config's schema, the API's contract.
- x: when the release line is 1.0.
- Snapshots between them are the next version with the commit: RPMs
0.1.1~git20261020.abc(sorting before 0.1.1).
Images and packages together. Every image is tagged with its exact
version (localhost/stronghold-dhcp:0.1.1, in the registry
stronghold-dhcp:0.1.1) and named by it; a registry may keep latest
on the newest, for people, but nothing of Stronghold's refers to it. Each
is labelled with the
Stronghold packages it was built from
(io.github.project-stronghold.stronghold). The functions' quadlets, in
the stronghold package (dist/quadlets), name the images' exact
version: a package says which images go with it, and an image which
packages it holds. A change to the images is a new images z and the
quadlets naming it, so a new stronghold z too; an upgrade of the
package then replaces the quadlets strongholdd installed and pulls the
new images (functions.md, "Installing"). A version published is never
replaced (scripts/publish.sh).
Checked at run time. Each agent reports its version; strongholdd
shows it with the function (stronghold functions, GET
/v1/functions's agent) and doesn't configure one of another release
line: the change fails, saying so, and the container is restarted on the
image its quadlet names (after an upgrade, while the new image is
pulled); the config is applied again once its agent is of strongholdd's
line.
Architecture¶
Cockpit plugin (UI) stronghold CLI Ansible (stronghold.gateway)
\ | /
HTTP API: root-only Unix socket, or HTTPS with tokens
|
strongholdd (confined SELinux domain)
|
config model: versioned desired state
|
renderers ──> firewalld, NetworkManager, keepalived,
conntrackd, Kea, Unbound, WireGuard
- One config model. All settings live in one declarative, versioned
document under
/var/lib/stronghold. Everything else is generated from it. This gives backup and restore, diffs, history and HA sync from one design. - The UI, CLI and Ansible never write config files themselves. They
call
strongholdd's HTTP API (docs/api.md, contractinternal/api/openapi.yaml): on the box through a Unix socket only root can open (Cockpit as the administrator, the CLI under sudo), elsewhere over HTTPS with tokens. Finer-grained rights (roles per token, polkit for local users) are still to come. - Commit with confirm. Every change is applied and reverts by itself unless kept within 60 seconds, so a bad rule can't lock you out. Kept changes are committed to the config history; a revert re-applies the last committed config.
- Changes strongholdd makes on its own are guarded too. A new
strongholdd can generate different files from the same config, and so
can a config edited outside it and reloaded. Each part (firewalld,
NetworkManager, DHCP/DNS) lists the files it manages; when an apply at
start or on reload changes any of them, the change waits 5 minutes to be
kept (longer than 60 seconds: after a reboot someone has to log in), and
otherwise the earlier files are put back and their services reloaded. The
earlier files are also saved on disk, so a restart during the window
doesn't keep the change either. The overview says what was rolled back
and offers to apply it again. This is in addition to bootc's image
rollback, which greenboot drives when strongholdd doesn't start
(
updates.md, "After an update"). - Image vs. config rollback. bootc rolls back the OS image; the config model has its own history. The two are rolled back separately.
Config as code¶
Everything Stronghold manages (ports, bonds, networks, zones, firewall policies, HA, link health) can be configured from YAML files in a git repository. The UI and the repository are two ways of editing the same config model.
Status (19 October 2026): the config directory, its local git history, the
history pushed to a remote (each box its own branch, backup.md) and a
config read back from it (a restore) are built; changes come in through the
API (UI, CLI, Ansible), checked and confirmed. Not yet: the box pulling and
applying what others push to the remote, signed commits only, webhooks.
The rest of this section is the design.
- The config model is a directory of YAML files, not one big file, so diffs and reviews stay readable:
stronghold.yaml # version, site, functions, api, alerts, ...
interfaces.yaml # ports, bonds, networks, routing
zones.yaml
dns.yaml # the resolver, local zones (optional)
ha.yaml # the pair, each node's own addresses (optional)
loadbalancer.yaml # (optional)
vpn.yaml # (optional)
firewall/
policies.yaml # zone-to-zone matrix and rules
objects.yaml # aliases, port groups, sets
nat.yaml # (optional)
Which node of a pair a box is lives outside the directory (ha.md), so
both boxes share one config.
- Local history is git too. On the box,
/var/lib/stronghold/configis a git repository. Every change applied from the UI or CLI is a commit, so history, diff and rollback are plain git. - Connecting a repository from the UI. In the UI you enter "this is my git repository" (URL, branch, credentials). Stronghold then:
- if the repository is empty, pushes the box's current config into it;
- if it already has a config, loads it, shows the diff against the running system and applies it with commit-with-confirm.
- Two-way sync with that repository. Once connected, every change made in the UI or CLI is committed and pushed to it, and changes pushed to it by others are pulled and applied (on a timer, or when a webhook says so). The repository always holds the complete config.
- Commit author is the person logged in to Cockpit; the committer is the box. The git log doubles as the audit log.
- Before committing, strongholdd fetches. If the remote has moved, it rebases the change on top. If both touched the same thing, the UI shows the conflict and lets the user pick, instead of overwriting.
- If the remote can't be reached, changes are committed locally and pushed when it is back. The UI shows how far behind or ahead the box is.
- Without a connected repository the box still keeps its own local git history, so the same history and rollback work offline.
- Signed commits only. A gateway that applies whatever it pulls is a high-value target, so strongholdd applies only commits signed by an allowed key (git's SSH or GPG signature verification). The box signs its own commits with its own key, which is one of the allowed keys.
- No secrets in the repository. WireGuard private keys, VRRP and PPPoE
passwords and the like are referenced by name in YAML and stored on the box
with
systemd-creds(TPM-bound where available). Encrypted secrets in the repository (age) are a later option. - The repository is the only way in. Nothing pushes config to the box directly; pipelines and other tools change the repository, and the box picks it up from there. The box never needs to accept inbound connections for config (a webhook only tells it to pull now).
- Same tool everywhere. The
strongholdCLI runs on a workstation or in CI too:stronghold validatechecks a repository against the schema, andstronghold set SECTION DIR --dry-runshows what would change on a running box. - HA: both nodes share one config (
ha.yamlholds what differs per node) and sync it between them; each pushes its own branch. - Zero-touch setup: a new box takes a config given with the install
(
/etc/stronghold/config: kickstart, image builder, cloud-init;first-boot.md). From a repository URL and key: not yet. - Git operations use the
gitcommand (in Fedora) rather than a Go library.
Config schema versions¶
stronghold.yaml's version is the schema the config is written in; a
build knows its own (config.SchemaVersion, 1 so far).
- Forward, at once. A config of an older version is migrated when it
is loaded (
internal/config/migrate.go): one step per version, N to N+1, each working on the files' YAML before they are decoded (an older config may have keys this build no longer has), free to change, add or remove files. strongholdd saves the result on start or reload as a commit of its own ("Config migrated from schema version 1 to 2: ..."), so the history shows the step and later changes start from it. A version with no step from it is refused, saying so.stronghold validatesays when a directory would be migrated. - Back, by the history. Nothing migrates a config down. An older image
(after
bootc rollback) finds a newer version and runs the commit it last applied, before the migration (updates.md, "Going back"); changes are refused there until the newer image runs again, so none are lost. - When a step is needed: a key renamed or moved, a meaning changed, a file split or merged. Adding an optional key needs none: an older config simply hasn't got it, and an older build refuses a config that has (strict parsing, D13), which is the going-back case above.
Features¶
Interfaces: ports, bonds and VLANs¶
- A port view that shows the physical NICs like a switch faceplate, with link state, speed and MAC address.
- Bonding in the view: select ports and choose "bond" (LACP, active-backup, ...).
- VLAN assignment as a matrix: VLANs × ports or bonds, each cell untagged, tagged or off. Distributing a VLAN across trunks is one click per cell.
- Optional VLAN-aware bridges for switch-like behavior across ports.
- Every interface, VLAN or bond is then assigned to a zone.
- Backend: NetworkManager (keyfiles and nmcli; Cockpit already talks to it).
Firewall¶
- Zone-based. Zones are first-class (WAN, LAN, GUEST, IOT, DMZ, SYNC, ...). Interfaces belong to zones; rules are written between zones, as firewalld policy objects.
- Zone matrix: a grid of source zone × destination zone with a default policy per cell (accept, drop, reject). Click a cell to see and edit its rules.
- Objects: host and network aliases, port groups, and dynamic sets (e.g. from blocklists) map to firewalld ipsets.
- NAT: masquerade, port forwards, 1:1 NAT.
- Flowtables for the software fast path (firewalld's
NftablesFlowtable). - Rule hit counters (
NftablesCounters) and a live log view in the UI. - Stronghold renders firewalld's configuration in full from the config model; the UI is the only place rules are edited.
Link health¶
A shared monitor in strongholdd, used by HA, multi-WAN and the UI. It works on any interface, whether or not HA or MAC-move mode is in use.
- Interface facts: carrier, speed and duplex, error and drop counters, and DHCP lease and IPv6 prefix state (from NetworkManager and the kernel).
- Active probes: ICMP ping (to the gateway or chosen targets), DNS queries, and HTTP(S) requests to a URL with an expected status. Several targets per link, so one dead target doesn't mark the link down.
- Bound to the link: probes are sent out of the interface being tested (socket bound to the device plus a routing mark), never through whatever the default route happens to be.
- Thresholds and hysteresis: loss, latency and jitter limits, and a link must stay bad (or good) for a set time before its state changes, so it doesn't flap.
- Consumers:
- HA: keepalived tracks link health and lowers its priority or fails over.
- Multi-WAN: gateway groups pick routes by health.
- UI: live status, history graphs and alerts.
- In MAC-move mode the standby's WAN is down, so only interface facts are available there; active probes run on the active node.
High availability¶
- keepalived (VRRP) gives each internal zone a virtual IP that moves between nodes.
- Two WAN modes, chosen per WAN interface, because both situations will come up:
- VRRP mode: for WANs with a static block (three or more addresses). Each node has its own address and the WAN gateway address is a VRRP virtual IP, as on the internal zones.
- MAC-move mode: for WANs with a single DHCP address. Both nodes use the same MAC address (and the same DHCPv6 DUID) on WAN, but only the active node brings WAN up. On failover, keepalived tells strongholdd, which activates the WAN connection on the new active node with that MAC, runs DHCP, and sends gratuitous ARP. The ISP sees the same client, so it gives back the same IPv4 lease and IPv6 prefix, and the NAT state that conntrackd synced still matches. The standby node keeps its WAN link down so the two never both claim the address. WAN is down during the DHCP exchange; how long that takes depends on the ISP.
- conntrackd syncs connection state over a dedicated SYNC zone, so connections survive a failover.
- Split-brain protection: the two nodes talk over the dedicated SYNC link and over a local LAN zone. VRRP heartbeats run on both, in one keepalived sync group, so losing only the SYNC link does not make the standby take over. A node takes the active role (and, in MAC-move mode, brings up WAN) only when it hears no peer on any path.
- Config sync from primary to secondary is built into strongholdd. Both nodes share one config model, with per-node overrides (real IPs, priority).
- Kea's HA hook (
libdhcp_ha.so, in Fedora'skea-hooks) keeps DHCP leases in sync. - The firewall renderer adds VRRP (protocol 112) and conntrackd traffic automatically when HA is enabled.
Later¶
Done since: IPv6 with prefix delegation, traffic shaping (CAKE), IDS
(Suricata), UPnP and NAT-PMP (miniupnpd, upnp.md). Still later: XDP
blocklists.
Planned (2026-10-17), and where it stands¶
- Secrets: done (
secrets.md): the box's own files, kept in an OpenBao elsewhere too. OpenBao on the box itself: not yet. HashiCorp Vault itself is BSL 1.1 since 2023: not open source, not in Fedora, not shippable (as a server elsewhere, its PKI works). - Workload identity: SPIFFE/SPIRE (Apache-2.0) isn't in Fedora; it would have to be packaged first. Later.
- Certificates: done (
certificates.md): ACME (lego, DNS-01), OpenBao or Vault PKI, FreeIPA (certmonger), for Cockpit and the API. For the load balancer's services: not yet. - Dynamic DNS: done (
dyndns.md): ddclient, a service function. - Proxies: done (
proxy.md, D24): the web proxy (Squid, caching, transparent for plain HTTP) and the reverse proxy (HAProxy: sites by name, TLS ended with the box's certificates or passed through), service functions. Logins (Kerberos) and HTTPS by name, never decrypted (peek and splice): done. Decrypting HTTPS (SSL bump) and PAC files: later. - Routing: static routes, policy routes and BGP are done
(
routing.md; FRR in a container of its own, D19). OSPF and BFD later, with the same FRR. - Active-active WAN in an HA pair: done (
ha.md, "Active-active"): each node is home to some networks and WANs and forwards for them; traffic between the two groups crosses the sync link both ways. A later step: both boxes on the same flows (ECMP, or BGP upstream), where return traffic may reach either box: conntrackd writing synced state into the kernel (DisableExternalCache), loose TCP tracking, and each box NATing to its own port range.
Fedora packaging check¶
| Component | In Fedora (F44) |
|---|---|
| bootc | yes |
| cockpit | yes |
| firewalld, nftables | yes |
| NetworkManager | yes |
| keepalived | yes |
| conntrack-tools (conntrackd) | yes |
| kea, kea-hooks (incl. HA hook), kea-dhcp-ddns | yes |
| unbound | yes |
| knot | yes |
| radvd | yes |
| suricata (with suricata-update; GPL-2.0-only) | yes |
| wireguard-tools, qrencode | yes |
| libreswan (GPL-2.0-or-later) | yes |
| libbpf, bpftool | yes |
| miniupnpd (BSD) | yes |
| openbao | yes (MPL-2.0) |
| vault | no: BSL 1.1, not allowed |
| spire (planned) | no: Apache-2.0, would need packaging |
| lego, certmonger | yes |
| realmd, sssd, freeipa-client, adcli | yes |
| ddclient | yes |
| frr | yes |
| squid (GPL-2.0-or-later) | yes |
| haproxy (GPL-2.0-or-later) | yes |
Open questions¶
None at the moment; the areas' own notes have theirs.