Proxies¶

Two proxies, each a service function in a container of its own (functions.md):

The web proxy (web-proxy): Squid, a caching forward proxy for the inside networks. Clients set up to use it (or, for plain HTTP, sent through it transparently) fetch through the box, which keeps what they fetch: a package or an image is downloaded once for the whole network.
The reverse proxy (reverse-proxy): HAProxy, sites by host name on the box's HTTPS port, sent on to servers inside, their TLS ended at the box with one of its certificates (certificates.md) or passed through to the servers.

Both are in Fedora (Squid 7, HAProxy 3.0; GPL-2.0-or-later), and both are what pfSense offers for the same jobs. The reverse proxy is HAProxy alone, not a choice of HAProxy or nginx (D24): HAProxy checks its servers (open-source nginx doesn't: only NGINX Plus does), reloads without dropping a connection, routes TLS by name without ending it, and reports each server's state; a choice would be two renderers, two policies and two sets of tests for the same features. What nginx does better (serving files, caching responses) a gateway doesn't need, and Squid caches.

Config¶

# proxy.yaml
web-proxy:
  networks: [lan, guest]       # whose clients may use it
  port: 3128                   # default
  transparent: true            # the networks' plain HTTP goes through it
  transparent-https: true      # and their HTTPS, read by name, never decrypted
  cache:
    disk: 10240                # MB on disk (default 2048; 0: memory only)
    memory: 512                # MB in memory (default 256)
    max-object: 2048           # MB, the largest response kept (default 1024)
    path: /srv/proxy-cache     # where it is on the box: a disk of its own mounted there
  blocked: [ads.example.net, tracker.example]   # with their subdomains
  access-log: false            # default true: each request in its journal
  auth:                        # the clients log in: Kerberos, without a prompt
    type: kerberos
    name: proxy.example.com    # the proxy's own name, in DNS at its address
    domain: ad.example.com     # default: stronghold.yaml's identity's
    networks: [lan]            # whose clients log in; default all of networks

reverse-proxy:
  networks: [wan]              # where clients reach it; default: the WANs
  http-port: 80                # default 80; -1: none
  https-port: 443              # default 443
  sites:
    - id: cloud
      hostnames: [cloud.example.com, "*.apps.example.com"]
      certificate: web         # stronghold.yaml's; its names cover the hostnames
      http: redirect           # default; serve: plain HTTP served too
      tls: false               # TLS to the servers (not checked: they are inside)
      balance: roundrobin      # leastconn, source (a client keeps its server)
      allow-from: office       # a network, address object or address
      check: { type: http, path: /status.php, status: 200, interval: 5s, timeout: 3s, retries: 2 }
      servers:
        - { address: 10.71.1.10, port: 8080 }
        - { address: nas.lan, port: 8080, weight: 0 }   # drained
        - { address: 10.71.1.12, port: 8080, backup: true }
    - id: vpn                  # no certificate: TLS passed through, by its name
      hostnames: [vpn.example.org]
      servers:
        - { address: 10.71.30.20, port: 443 }

The API's section is proxy (PUT /v1/config/proxy, null removes it), the CLI's stronghold set proxy DIR; Interfaces › Proxies in the UI; stronghold.gateway.config with section: proxy (the gateway role's stronghold_proxy, after the certificates).

The web proxy¶

Who uses it. It listens on every address of the box, port 3128; the firewall lets each of networks reach it from its subnet (IPv4, and the ULA /64), and Squid answers those subnets only. Clients are set up with the box's address there (the virtual address in a pair) and the port: by hand, by policy, or by their system's proxy settings.
Transparent (transparent: true): the networks' plain HTTP (port 80, to anywhere but the box), but theirs whose clients log in (below), is redirected to Squid's intercept port (3129) in Stronghold's NAT table, so it goes through the cache without the clients knowing. IPv6 isn't redirected (the NAT table is IPv4's): IPv6 clients use the proxy when set up for it.
Transparent HTTPS (transparent-https: true): the networks' TCP 443 (but theirs whose clients log in), redirected to Squid's TLS intercept port (3130), never decrypted (peek and splice):
Squid reads the site's name from the client's TLS hello (SNI), closes the connections to blocked domains (ssl_bump terminate: the browser says it couldn't connect) and passes the rest on untouched, to the address the client asked for. What isn't TLS on 443 passes on too.
The access log has each one, with the name (%ssl::>sni, the last field): who went where, by name, over HTTPS.
Not cached: it isn't decrypted.
QUIC (HTTP/3, UDP 443) would go around it: the networks' UDP 443 to the WANs is refused (a system rule), and browsers fall back to TCP.
Squid wants a certificate on that port although it never shows it: a self-signed one made on the box (/etc/squid/intercept.pem).
A client that checks a site's name against its address (a pinned app) is unaffected: nothing is changed in its connection.
Decrypting it (SSL bump, with a CA of the box's that every client trusts) is a step of its own: later, and opt-in.
IPv4 only, as for HTTP.
What it caches. Plain HTTP: Fedora's mirrors and most Linux repositories, much of the software updates of other systems. Packages and images (.rpm, .deb, .iso, ...) are kept as long as the cache has room, since a published one doesn't change; repository metadata (repomd.xml, metalinks, Release) is always asked for again. HTTPS through the proxy (CONNECT) is passed on, not cached. A response over max-object isn't kept; memory keeps responses up to a tenth of its size (32 MB at most), the disk the rest.
Blocked domains are refused with their subdomains (Squid's error page; for HTTPS the connection is refused). It is the proxy's own list, for the clients that use it: the DNS blocklists (dhcp-dns.md) are the ones every client has.
Safety: only ports 80, 443 and above 1024 (and the few Squid's defaults allow), CONNECT only to 443; never to the box itself or to link-local addresses; the clients' addresses aren't sent on (forwarded_for delete).
The cache is on disk, kept across restarts and images; a new size, or a cache where there was none, restarts Squid (the directory is made first, squid -z); other changes are checked (squid -k parse) and read again (SIGHUP) without a restart. See "Where the cache is".
State: stronghold proxy, GET /v1/proxy and the UI show the requests since it started, the clients lately, how much was answered from the cache (requests and bytes, last five minutes and hour) and what it holds, from Squid's cache manager (from the box itself only).
In a pair both boxes run it, each with its own cache; clients use the network's virtual address, so the active box answers.

Where the cache is¶

cache.path is the cache's directory on the box: where to put more room, a disk (or a logical volume) of its own mounted there. Without it, the cache is in the function's own directory, /var/lib/stronghold/functions/web-proxy/cache in its container (/var/spool/squid with Squid on the host), on the root file system.

The directory must exist (strongholdd doesn't make one, so a disk not mounted yet isn't filled in its place) and isn't the system's (/usr, /etc, /var/lib, /home, /srv itself, anything else of Stronghold's). Its files become the proxy's: a directory of its own.
The cache must fit: a disk bigger than the file system is refused. stronghold proxy, GET /v1/proxy, the Proxies page and the Overview show where the cache is, how big its disk is and how much is free (the Overview's tile turns yellow under a tenth free).
In its container strongholdd puts the container's volume there: a drop-in of its quadlet (/etc/containers/systemd/stronghold-web-proxy.container.d/50-stronghold-cache.conf, the quadlet's volumes with the cache's at the path; podman labels it for the container), and the container restarted on it. The old cache stays where it was, for the admin to remove. On a podman host the quadlet is the host's: its volume is set there.
On the host (runtime: host) Squid's cache_dir is the path, which must be Squid's for SELinux and its owner. strongholdd changes no policy at run time (selinux.md), so it says what to run when it isn't: semanage fcontext -a -t squid_cache_t '/srv/proxy-cache(/.*)?' && restorecon -RF /srv/proxy-cache && chown -R squid:squid /srv/proxy-cache.

Logins¶

auth has the clients of its networks (all of the proxy's by default) log in, with Kerberos (Negotiate): a domain's computers, Windows' among them, log their users in by themselves, without a prompt; Edge and Chrome do for a proxy, as do Firefox (network.negotiate-auth.trusted-uris) and curl (--proxy-negotiate). The users are the domain's: Active Directory, or FreeIPA / IdM.

The proxy's name (auth.name) is what the clients are set up with (a group policy, their proxy settings), in DNS at the box's address there (in a pair the virtual address): Kerberos tickets are for a name, never an address. A name of its own (proxy.example.com), not a box's: both boxes of a pair answer for it, with one key.
The key, a keytab with HTTP/<name>@<REALM>, is the box's secret (web-proxy/keytab, shared by a pair), never in the config. Either:
stronghold proxy join -u USER (or the Proxies page, or stronghold.gateway.proxy_join) with a domain user who may add computers or services, once: in AD a computer account of the name of its own (adcli, as realmd joins: PROXY$, its HTTP principal; nobody changes its password after, so its key stays until it is joined again), in FreeIPA a host entry for the name and its HTTP service (ipa-getkeytab; the box an IPA client: identity.md). --server names a domain controller, when DNS doesn't find one. A box from packages installs adcli (or freeipa-client) first, through PackageKit.
or one made elsewhere (ktpass on Windows, ipa-getkeytab): stronghold proxy keytab FILE, the proxy section's keytab, the Proxies page's upload, Ansible's proxy_keytab.
Checking a ticket needs only the key, not the domain: the box needn't be a member (identity), and logins go on while the domain controllers are away. Squid's negotiate_kerberos_auth checks them (any principal of the keytab), and the user name is in the access log.
Until there is a key, the clients that must log in can't use the proxy at all (rather than all of them without), and its state says so.
Not transparent: a client sent through the proxy without knowing can't be asked to log in, so the networks whose clients log in aren't redirected; transparent is for the others (a guest network).
Who of the domain may use it (groups) is later: every user of the domain may now.

Troubleshooting and the Overview¶

Overview: a tile each: the web proxy's share from the cache, what it holds and its disk's free room (yellow under a tenth free, or without a key for its logins); the reverse proxy's servers up and its sites.
Troubleshoot › Proxies: Fetch through the web proxy (a URL, from the box, no login: its headers, Cache-Status saying whether it came from the cache, hit); Web proxy requests (the access log, by client, user or site: TCP_HIT, TCP_MISS, TCP_DENIED/407); Ask a site (a site's name through the reverse proxy from the box: which site it goes to, ended or passed through, its servers, and the answer). Logs and Service status have squid and haproxy.
The same from the CLI: stronghold tool proxy-fetch url=..., stronghold tool proxy-log grep=..., stronghold tool site host=....

The reverse proxy¶

Where it listens. On every address of the box, IPv4 and IPv6, ports 80 and 443; the firewall lets networks (default the WANs) reach them. A WAN with an address from DHCP works as well as a static one. Inside clients reach the sites too, if a network of theirs is listed (or through NAT reflection); split DNS (dhcp-dns.md, views) can send them to the box's inside address.
TLS ended at the box (a site with certificate): the certificate is one of stronghold.yaml's certificates, from ACME, OpenBao or Vault, or FreeIPA (certificates.md), whose names must cover the site's. The site is chosen by the request's Host; HTTP/2 to clients; the servers get plain HTTP (or TLS, tls: true, their certificates not checked) with X-Forwarded-For and X-Forwarded-Proto. Until a certificate is issued the site has a self-signed one; once issued or renewed, HAProxy reads it again (certs.Manager's Issued).
TLS passed through (no certificate): the site is chosen by the name the client asks for in its TLS hello (SNI), and the connection goes to the servers as it is; they end TLS with their own certificates. Allow-from and checks still apply.
Plain HTTP on port 80 is redirected to HTTPS (http: redirect, the default, for both kinds), or served too (serve, for a site with a certificate). Port 80 also takes ACME's HTTP-01 challenges (below).
Names: exact ones win over wildcards (*.apps.example.com: any name under it); two sites can't have the same name. A name of no site gets a 404 (or, passed through, the connection refused).
Servers: addresses or host names (resolved when HAProxy starts, and started without them if they can't be yet); weight 0 drains one, a backup gets requests only while no other is up. Checks: TCP by default, an HTTP or HTTPS request (check, as the load balancer's), or none; a server that fails retries checks is left out until it passes as many.
Changes are checked (haproxy -c) before HAProxy reads them, and read without dropping a connection: HAProxy runs master-worker (-W), SIGUSR2 starts new workers and the old ones finish what they have.
State: each site's servers (up, down, drained; the last check; the connections now and since it started), from HAProxy's stats on 127.0.0.1:8404, in stronghold proxy, GET /v1/proxy and the UI.
In a pair both boxes run it; the active one gets the connections (the WAN's virtual address moves to it).
The load balancer (loadbalancer.md) is layer 4 and the kernel's; the reverse proxy is for HTTP(S) and TLS by name. A virtual server on the reverse proxy's ports is refused (IPVS would take its connections).

ACME's HTTP-01 behind the reverse proxy¶

lego answers an HTTP-01 challenge on the WAN's port 80, which the reverse proxy has. So while the reverse proxy is on with port 80, lego listens on 127.0.0.1:8402 instead and HAProxy passes /.well-known/acme-challenge/ on to it, whatever the name; the firewall has port 80 open for the reverse proxy already. DNS-01 needs neither.

How it runs¶

	Web proxy	Reverse proxy
Program	Squid (`squid -N`; on the host Fedora's unit)	HAProxy (`haproxy -W -db`)
Image, quadlet	`stronghold-web-proxy`	`stronghold-reverse-proxy`
Host unit (package mode, `runtime: host`)	`squid.service`	`haproxy.service`
Files strongholdd writes	`/etc/squid/squid.conf`	`/etc/haproxy/haproxy.cfg`, `/etc/haproxy/certs/<id>.pem` (0600)
Capabilities	`SETUID SETGID CHOWN KILL`	`NET_BIND_SERVICE SETUID SETGID KILL`
Reload	SIGHUP (a new cache: restart)	SIGUSR2
Code	`internal/proxy/squid.go`	`internal/proxy/haproxy.go`

On the host (package mode), the stronghold SELinux module lets strongholdd write Squid's config and run squid -k parse and haproxy -c, Squid listen on unreserved ports and reach any port, and HAProxy listen on and connect to any port (not the squid_connect_any and haproxy_connect_any booleans: selinux.md).

Lab¶

A clean Fedora VM (stronghold-proxy, LAN 192.168.74.2, WAN by DHCP; create-fedora-vm.sh -a, setup-with-containers.sh), both functions installed when turned on (their images from the lab's registry), the servers on the lab host (192.168.74.254: plain HTTP on 8080, TLS on 8443). 20 October 2026, 0.3:

The web proxy, from the LAN: a package fetched twice, a miss (0.11 s) then a hit from memory (0.002 s); a blocked domain refused (403, Squid's page); HTTPS through it as a tunnel (CONNECT). Transparent: the lab host's route to a mirror through the box, and a plain request without a proxy came from the cache (TCP_MEM_HIT), the blocked domain refused there too. A new blocked domain: Squid read its config again (SIGHUP), no restart. stronghold proxy showed the share answered from the cache.
The reverse proxy, on the WAN's address (from DHCP): port 80 redirected to HTTPS; cloud (TLS ended at the box, with its self-signed stand-in, the certificate's issuer being unreachable) answered by its plain server over HTTP/2; vpn passed through, the client seeing the server's own certificate; a name of no site, 404; ACME's path handed on (503: no lego running). 443 from the LAN: refused (the WANs only, the default). The plain server stopped: DOWN (L4CON) in a few seconds, UP (L7OK) once back.
Off: the reverse proxy's container masked, its section gone, ports 80 and 443 free.
On the host (runtime: host, Fedora's squid and haproxy installed by PackageKit, SELinux enforcing): the same, by Fedora's units (squid.service, haproxy.service); Squid's log in /var/log/squid/access.log, the transparent request through it.

Logins and the cache's place, 0.3.5: an Active Directory of the lab's (Samba's AD DC, Fedora's samba-dc, in a container on the VM's LAN at 192.168.74.10, realm LAB.EXAMPLE, proxy.lab.example in its DNS at the box) and a client beside it (kinit jo, curl --proxy-negotiate):

stronghold proxy join -u Administrator --server 192.168.74.10 made PROXY$ and its keytab (HTTP/proxy.lab.example, host/...), kept as the secret; stronghold proxy listed its keys.
No ticket: 407 Negotiate; with jo's: the file, no password typed, jo in the access log; from the cache the second time. The same with Squid on the host and in its container.
cache.path: /srv/proxy-cache (in its container): the quadlet's drop-in, the container restarted on it, Squid's directories made there and labelled for the container by podman; stronghold proxy showed the path and its disk's room.
The tools: Fetch through the web proxy (Cache-Status: ...;hit), Web proxy requests by user, Ask a site.

Found on the way: - The container's agent may write only the files its quadlet names: the keytab wasn't one (a test now checks every quadlet against its manager's files). - A box from packages hasn't adcli: the join installs it first. - Squid 7 says Cache-Status (RFC 9211), not X-Cache. - Squid's Kerberos helper on the host reads the network sysctls: denied, harmless; allowed now. - A cache at a path of its own, moved from the container to the host: refused until it is Squid's, as it should be; but the hint said restorecon -R, which leaves container_file_t (a customizable type) as it is (-RF now), and strongholdd couldn't see squid_cache_t to check it (the module lets it). - Squid's master puts /dev/null in place of its output, so its logs went nowhere; and Squid, as squid, can't open a pipe of root's again: one process (-N), and the agent's --output-user. - Squid's pid file: root without DAC_OVERRIDE can't write squid's directory; none in the container. - The status page asking Squid's cache manager every few seconds filled the access log: left out (!manager). - HAProxy warned of option forwardfor in TCP mode and of a redirect after a use_backend: set where they apply. - A redirect to an HTTPS port other than 443 lost the port. - On the host, strongholdd may not make /etc/haproxy/certs (SELinux: directories in etc_t); the module lets it. haproxy -c, as strongholdd, tries to keep its startup messages in /dev/shm: denied, and not needed (not audited now). Denials are in the journal, not ausearch (lab.md, "Known issues").

Transparent HTTPS, 0.4.2, the same VM and a client on its LAN set up for no proxy (transparent-https, example.org blocked):

https://example.com/: 200, with the site's own certificate (spliced, not decrypted); in the access log TCP_TUNNEL/200 CONNECT example.com:443 with the name read from the hello.
https://www.example.org/: the connection closed during TLS (curl's 35), logged as NONE_NONE/000 with its name; QUIC to the internet (UDP 443): refused at once.
The same with Squid on the host and in its container (its quadlet's new --path for the certificate, put in place by the upgrade). No SELinux denials.

Later¶

HTTPS through the web proxy decrypted, opt-in (SSL bump with the box's own CA, the clients told to trust it): caching it, blocking by URL. The names from the TLS hello (peek and splice) are done.
Who may use the web proxy by the domain's groups; logins for computers outside the domain (Basic over LDAP, behind TLS); a PAC file (WPAD) handed out by DHCP and DNS.
ICAP to a virus scanner (c-icap with ClamAV).
The reverse proxy: paths to other servers within a site, HSTS, rate limits, the runtime API for draining without a reload.