Locking Down Authelia: TOTP, Real SMTP, and the Hostname-vs-IP TLS Trap

There’s a particular kind of false confidence that comes from standing up Authelia in front of your apps and seeing the login page render. It looks done. You type a username and password, you get challenged, you get in. Box checked, right?

Except a username-and-password-only Authelia instance is just a fancier login form sitting in front of the exact same single point of failure every other login form has: whatever password you picked. The entire reason to run a centralized auth layer in the first place is to get to do hard things — like enforcing real two-factor authentication — once, centrally, instead of bolting it onto every app individually. If you skip that part, you’ve built infrastructure for a feature you’re not actually using.

So this post is about closing that gap: turning on real TOTP-based two-factor, and wiring up the notification system Authelia needs to actually deliver codes, links, and alerts — which sounds like a footnote but turned into its own small adventure once I hit a TLS certificate quirk that’s worth flagging for anyone else who runs into it.

Why two-factor isn’t optional once you centralize auth

Once Authelia is gating Nextcloud, Open WebUI, and whatever else lives behind it, that one login becomes the single key to the entire DMZ. That’s the whole point — but it cuts both ways. A compromised password used to mean a compromised app. Now it means a compromised everything, all at once, through the front door I built specifically to make access more convenient. Centralizing authentication without centralizing strong authentication is just centralizing risk — and it’s the same reason the DMZ network itself is VLAN-isolated from the rest of the homelab in the first place: a single compromised layer shouldn’t mean a compromised everything else too.

Authelia supports TOTP out of the box, and enabling it for an access_control policy is a one-line change in configuration.yml:

yaml

That’s it on the policy side — flip one_factor to two_factor for any domain you want gated, restart Authelia, and the access control layer now requires a second factor before granting a session. I deliberately left auth.yourdomain.com itself — Authelia’s own login portal — on a bypass policy, since that’s the page doing the authenticating in the first place; there’s nothing to gate behind itself.

The part that actually takes thought isn’t the policy line, though — it’s how the person setting up TOTP for the first time actually receives the QR code and registration link. Authelia’s identity-verification flow for first-time 2FA registration works by emailing you a link to confirm it’s really you before letting you scan a QR code and bind an authenticator app. No working notification channel, no registration. The login page rendering correctly was never actually proof the auth system was complete; it was proof exactly one feature — first-factor password checking — was complete.

Bootstrapping without email: the filesystem notifier

Here’s a chicken-and-egg problem I didn’t expect going in: you generally need a working SMTP setup to register 2FA, but I hadn’t actually wired up SMTP yet when I wanted to register my own TOTP for the first time. Authelia’s answer to this is a notifier mode that doesn’t try to send anything anywhere — it just writes the notification to a file on disk:

yaml

With this in place, instead of an email landing in an inbox, the identity-verification link gets written straight into a text file inside the Authelia container. Reading it is just:

bash

Copy the link out, paste it into a browser, and the registration flow proceeds exactly as if a real email had arrived. This is a genuinely useful bootstrapping trick worth knowing about independent of whatever your final SMTP setup ends up being — any time Authelia needs to send something and you don’t yet trust (or haven’t yet built) the real delivery path, dropping back to the filesystem notifier lets you keep moving without blocking on infrastructure that isn’t ready yet.

Wiring up real SMTP, and the trap waiting inside it

Bootstrapping aside, a filesystem notifier obviously isn’t a long-term answer — it only works because I have shell access to the box, which defeats the entire purpose of a notification system the moment any other person needs to register their own 2FA. The real fix is pointing Authelia at an actual mailbox.

The notifier config for SMTP looks innocent enough:

yaml

The submissions:// scheme tells Authelia this is an implicit-TLS SMTP submission connection on port 465, rather than the older plaintext-then-STARTTLS dance on port 587. That part’s fine and fairly standard.

What actually tripped me up was something that, in hindsight, is an obvious consequence of how TLS works, but is exactly the kind of thing you don’t think about until it bites you: my first instinct was to point the address field at the mail server’s raw IP rather than its hostname, since I already had the IP handy and it felt like one less moving part — no DNS resolution step, no dependency on DNS staying healthy. Authelia started up, attempted the connection, and immediately failed with x509: cannot validate certificate for <IP> because it doesn't contain any IP SANs.

That error message, once you actually read it instead of just reflexively googling it, explains the whole problem. The mail server’s TLS certificate was issued for the hostname mailserver.domain.com. TLS certificates assert an identity — “I am this specific name” — and the client connecting is supposed to check that the name it asked to connect to matches the name on the certificate it received back. When I connected by raw IP, there was no hostname in the conversation for the certificate to match against; the certificate says “I’m mailserver.domain.com,” and the connection was nominally to 203.0.113.45 or whatever the IP happened to be, and those two things have no relationship as far as the TLS handshake is concerned, unless the certificate specifically lists that IP as an additional valid identity (an IP SAN, which this one didn’t have, because almost no certificates bother to).

This is a small, almost embarrassing thing to get wrong, and also a genuinely common trap, because using a raw IP feels like it should be strictly safer or simpler — fewer dependencies, no DNS lookup that could fail or get hijacked. For plaintext or non-validated connections, sure. The moment TLS certificate validation enters the picture, the hostname isn’t an implementation detail you can route around — it’s load-bearing. The certificate’s whole job is binding a cryptographic identity to a name, and if you don’t connect using that name, there’s no name for the certificate to authenticate against, full stop. (The same hostname-vs-certificate confusion shows up again, in a slightly different shape, when verifying a wildcard cert covers a new subdomain — worth a read if this kind of TLS mismatch interests you beyond just SMTP.)

The fix was simply using the hostname Authelia was always meant to use:

yaml

Once that one piece changed, the connection succeeded cleanly, the startup check passed, and Authelia logged a clean Startup complete with no certificate complaints.

Confirming it actually works, end to end

Configuration that merely starts without errors and configuration that actually works are two different claims, and I’ve been burned often enough by the first one masquerading as the second that I always insist on testing the real flow before calling something done. With SMTP wired up correctly, I went back through the TOTP registration process one more time, this time as a genuine end-to-end test rather than the filesystem-notifier bootstrap — triggered the identity verification flow, and watched a real email land in the actual inbox a few seconds later, with a working confirmation link.

From there, registering TOTP itself is almost anticlimactic after all that setup: scan the QR code with an authenticator app, enter the six-digit code it generates to confirm the binding succeeded, and from that point forward, every login to a two_factor-policy domain prompts for that code after the password. The login page, from the outside, looks exactly like it did before any of this work — username, password, and now an extra six digits. The difference is everything behind that screen actually does what centralizing authentication was supposed to buy in the first place.

The actual lesson here

None of this was hard, exactly — every individual piece was a short config block and a restart. What made it worth writing up is how each step quietly assumed the previous one actually worked, and how easy it would have been to stop at “the login page renders” and call it done. Two-factor without a working notification channel isn’t two-factor, it’s a setting nobody can finish enabling. A notification channel that starts without errors isn’t necessarily a notification channel that delivers anything, if a hostname-vs-IP TLS mismatch is quietly swallowing every send attempt. The only way to know any of it actually works is to walk the whole path a real, first-time user would take, end to end, and watch what actually lands in an actual inbox.

This same 2FA-gated access_control pattern gets reused directly in the next build — Ditch Google, which gates a self-hosted SearXNG instance behind exactly this TOTP setup, plus a network-scoped bypass rule for Open WebUI’s API calls. And if locking yourself out of your own auth layer is a worry — it should be, once you’ve made Authelia the single key to everything — Never Lock Yourself Out covers the equivalent safety net on the OPNsense side.

Leave a Comment

Your email address will not be published. Required fields are marked *