Certificates, Host Keys and Knowing Who You're Talking To

Encryption answers one question — can anyone read this? — and leaves a second one open: am I talking to the server I think I am?

Without an answer to the second, the first is close to worthless. An attacker who intercepts your connection can present their own encryption, decrypt everything you send, and forward it on. You would see a padlock the whole time. That is a man-in-the-middle attack, and defeating it is what certificates and host keys exist for.

The verification problem

When you connect to ftps://example.com, something answers. How do you know it is the real server, and not whatever your DNS resolver, your Wi-Fi access point, or a compromised router decided to send you?

The two protocols solve it differently.

FTPS: certificates

An FTPS server presents a TLS certificate during the handshake — the same mechanism as HTTPS. The certificate contains the server’s public key and its hostname, signed by a Certificate Authority your Mac already trusts.

If the certificate is signed by a public CA (Let’s Encrypt, DigiCert…), validation is automatic and invisible: the chain checks out, the hostname matches, the connection proceeds. Nothing to decide.

Very often, though, an FTPS server on a NAS or a self-managed box has a self-signed certificate — no CA involved, so nothing to validate against. Your client cannot say whether it is genuine. This is where the warning dialog comes from, and where most people click through without reading.

SSH: host keys

SFTP takes a different route. There are no certificate authorities in SSH. Instead, every server has a host key pair, and it presents its public host key on connection.

The first time you connect you have no way to verify it — SSH says so plainly:

The authenticity of host 'example.com' can't be established.
ED25519 key fingerprint is SHA256:uPvZ3nCq...+8fK4M.
Are you sure you want to continue connecting (yes/no)?

You answer yes, and the fingerprint is written into ~/.ssh/known_hosts. Every future connection is checked against it.

Trust on first use

Both cases end up at the same model: TOFU, trust on first use. You accept an identity once, it gets pinned, and from then on any change is flagged loudly.

It is a genuine trade-off, worth stating plainly. The first connection is unverified — if you were being intercepted at that exact moment, you would pin the attacker’s identity and never know. But every connection afterwards is strongly protected, which is a large improvement over checking nothing, and it requires no certificate authority.

You can close the first-use gap by verifying the fingerprint out of band: most hosting providers publish their SSH fingerprints in the control panel or in the welcome email. Comparing takes ten seconds and eliminates the only weak moment in the model.

Upstream implements TOFU for both protocols: the SHA-256 fingerprint of an FTPS server’s leaf certificate, and of an SFTP server’s host key, is pinned per host and port on first use. A subsequent mismatch is not a warning you can wave away in passing — the connection is refused, because a changed key on a server you have used before is exactly what an attack looks like.

When a fingerprint changes

You will see something like “host key verification failed” or “the certificate has changed”. There are three real explanations:

The server was rebuilt or reinstalled. New machine, new host key. Common after a VPS migration or a NAS firmware reset, and by far the most likely cause.

The certificate was renewed. Let’s Encrypt certificates rotate every 90 days. Note that a client pinning the leaf certificate will see a change at every renewal — which is why a properly CA-signed certificate should be validated by chain rather than pinned.

Someone is intercepting your connection. Rarer, but it is the possibility the whole mechanism exists to catch.

The correct response is always the same: do not click through it. Find out why. Ask your host whether the server was rebuilt, or check the published fingerprint. Only after you have an explanation should you remove the old pin and accept the new one — and if the answer is “we didn’t change anything”, stop and escalate.

Checking a fingerprint yourself

For SSH, from a machine you already trust:

ssh-keyscan -t ed25519 example.com | ssh-keygen -lf -

For a TLS certificate:

openssl s_client -connect example.com:990 </dev/null 2>/dev/null \
  | openssl x509 -noout -fingerprint -sha256

Compare the output with what your client shows you. They should match exactly.

Step by step

1. Verify the fingerprint on the first connection against something your host published, rather than accepting it blindly.
2. Prefer a CA-signed certificate for FTPS where you control the server — Let’s Encrypt is free and removes the self-signed question entirely.
3. Let the client pin the identity so that any future change is detected.
4. Treat a changed fingerprint as an incident until proven otherwise — investigate before accepting.
5. Re-verify out of band after a legitimate server rebuild, then accept the new key once.

Questions people ask

What is a certificate fingerprint?

A cryptographic hash — normally SHA-256 — of the certificate or public key. It is short enough to compare by eye while still uniquely identifying the key.

Is a self-signed certificate insecure?

The encryption is identical. What is missing is third-party proof of identity, which shifts the burden to you: verify the fingerprint once, then rely on pinning.

Why does my client refuse to connect after the server was rebuilt?

Because the pinned fingerprint no longer matches, and that is indistinguishable from an attack. Confirm the rebuild with your host, then remove the old pin and accept the new identity.

Should I just click “trust” on the warning?

Not on a changed key. On a first connection with a self-signed certificate it is reasonable, ideally after comparing the fingerprint. On a change for a server you have used before, find out why first.

Does FTPS verify my identity too?

Not usually. The certificate authenticates the server; you still authenticate with a password. Client certificates exist but are rarely deployed. If you want strong two-way authentication, SFTP with SSH keys is the practical answer.