SSH - Handshakes and Fingerprinting

Logging In

If you have machines you access via SSH, this is for you. When you first SSH into a machine, you'll get a dialogue like the following:

$ ssh me@randomhost.example.ca
The authenticity of host 'randomhost.example.ca (192.168.137.195)' can't be established.
ECDSA key fingerprint is SHA256:oR04sE48r0tLBTekCaGt5Ia9cUl+GrqKucNAPTt8JSg.
Are you sure you want to continue connecting (yes/no)?

If you're using an older version of SSH, it may look like this:

$ ssh me@randomhost.example.ca
The authenticity of host 'randomhost.example.ca (192.168.137.195)' can't be established.
ECDSA key fingerprint is MD5:0b:64:e5:02:2f:3c:09:3d:57:d4:f4:23:64:03:c3:69.
Are you sure you want to continue connecting (yes/no)?

And, if you're using Fedora 25, you might be lucky enough to see this:

$ ssh me@randomhost.example.ca
The authenticity of host 'randomhost.example.ca (192.168.137.195)' can't be established.
ECDSA key fingerprint is SHA256:oR04sE48r0tLBTekCaGt5Ia9cUl+GrqKucNAPTt8JSg.
ECDSA key fingerprint is MD5:0b:64:e5:02:2f:3c:09:3d:57:d4:f4:23:64:03:c3:69.
Are you sure you want to continue connecting (yes/no)?

This is hugely useful ... we'll get to why in a moment. I don't know whether the credit lies with Fedora or SSH or someone else entirely, but I was glad to see it providing the two commonest fingerprint types.

Security is Inconvenient

What you're supposed to do after you see one of the above dialogues is to compare that key fingerprint to the one actually hosted on the server. There are a variety of ways in which that can be done - none of them convenient:

  • go to the console of the server and check the key fingerprint there
  • call someone else who has access to the server to log in and get the fingerprints for you
  • look at the list of valid key fingerprints provided by your server administrator over a secure channel

Did I mention this was inconvenient? Security often is. Why do we care about the server fingerprint? It establishes the identity of the server, and, once established, that identity will be stored in your ~/.ssh/known_hosts file so it can't be spoofed and you'll never be asked again. What are the risks? The possibility that a Man-in-the-middle attack (aka "MITM") is being perpetrated against you: the attacker pretends to be your server (but has a different key) and forwards your login requests and responses to the real server, so it looks like you have a session but they now have your credentials.

Verifying keys is decidedly inconvenient, and - unless you're a high profile target like a bank - the chance of a MITM attack is exceedingly small. Together, this means that the vast majority of people type "yes" to the fingerprint dialogue without ever trying to find out if the server key is valid. In my ongoing attempts to make security enough easier that people will actually use it, I'm trying to generate lists of server key fingerprints and distribute them in a relatively secure way.

OpenSSH v6.x generally provides an MD5 fingerprint (Debian 8, aka "jessie," currently "oldstable," uses this). MD5 has now been shown to be subject to a variety of vulnerabilities - including collisions, which would allow two different keys to have the same fingerprint. As a result OpenSSH v7.x generally provides a SHA256 fingerprint (Debian 9/"stretch"/"stable" and Fedora 25 use this).

Getting Key Fingerprints from the Server

Getting the key fingerprints from your server isn't obvious, but at first it seems fairly straight-forward:

$ for key in /etc/ssh/*.pub; do ssh-keygen -lf ${key}; done
256 SHA256:oR04sE48r0tLBTekCaGt5Ia9cUl+GrqKucNAPTt8JSg no comment (ECDSA)
256 SHA256:QGyQ6iNGZHpEdbYNRNzk8QdIArVVy1krBHUED+RSY04 no comment (ED25519)
2048 SHA256:yyVo9mOCx2qi2q1su15CjUSeUQ3vEYV1GiBXaWgw8fA no comment (RSA)

Note that we're looping through the server's public keys: this is recommended as the private keys produce the same fingerprint but are inaccessible to non-privileged users.

The above is what you'll get on a OpenSSH v7.x server. But ... what if you're trying to attach to the machine with a v6.x client? It will ask you:

The authenticity of host 'randomhost.example.ca (192.168.137.195)' can't be established.
ECDSA key fingerprint is MD5:0b:64:e5:02:2f:3c:09:3d:57:d4:f4:23:64:03:c3:69.
Are you sure you want to continue connecting (yes/no)?

This is fairly easily solved. Again, on the server, run:

$ for key in /etc/ssh/*.pub; do ssh-keygen -E md5 -lf ${key}; done
256 MD5:09:71:57:15:ed:0e:d8:04:f2:5f:5b:57:cb:d7:4b:2a no comment (ECDSA)
256 MD5:6e:9f:07:4b:6c:91:31:c7:82:6c:6c:a4:96:b1:e4:28 no comment (ED25519)
2048 MD5:fe:24:28:ab:84:4c:76:7b:46:a9:bb:c9:fd:49:64:68 no comment (RSA)

The reverse is much more difficult: if the server is running an older SSH, and your client is asking about a SHA256 fingerprint, the server will want to produce MD5 fingerprints ... and the v6.x server doesn't support the -E switch. So (thanks StackOverflow, as so often ...) go back to the server and run this:

$ for key in /etc/ssh/*.pub; do awk '{print $2}' ${key} | base64 -d | sha256sum -b | awk '{print $1}' | xxd -r -p | base64 | sed -e 's/=$//' ; done
zYPfbWH4+9G/5vK5T5JlHra+rpODi6JQJ2ieEtJoJhc
QGyQ5iNGHHpEdbYERNzk8QdIBrWWy1krBHUED+MSY04
yyVl9mOGx2gi2q1su51CjUSeUQ3vCYO0GiBXaWnx8fA

The output isn't as pretty - it doesn't list the key size, comment, or type - but should do the job. I cannot claim to fully understand what's going on there, but I can confirm from using this a number of times that it works well. (Note that 'xxd' is in the xxd package and apparently included in the base install on Debian, but in Fedora it's part of the vim-common package which isn't automatically installed.)

So the plan is to generate a full set of key fingerprints (both MD5 and SHA256) for my servers when they're installed, and then put them in a git repo. The "git repo" part isn't a perfect solution, but if someone has hacked your repo without your noticing, you've got worse problems on your hands.

Finding Your SSH Version Number

The first step of that plan is to figure out which of the above methodologies I'm going to need to generate the fingerprints. And to do that, I need to know the major version of SSH. The geniuses at SSH have decided that the output of -V (for "version") is an error, and thus should go to stderr. This makes it more difficult than usual to work with, and leads to the odd syntax you see below:

$ ssh -V 2> >( awk '{ print $1 }' ) | awk -F '_' '{ print $2 }' | awk -F '.' '{ print $1 }'
7

This will also work on version 6. Please note!! The ssh -V 2> >( awk '{ print $1 }' ) is a Bashism that won't work in sh or other shells. Had they chosen to use stdout, it would have been as simple as ssh -V | awk '{ print $1 }' in any shell, instead of the shell-specific contortions that have proven necessary ... It took a long time to find a good solution for sh (another reason I'm raging against the ssh programmer's very odd and incorrect choice of printing their version number to stderr instead of stdout), and it's ugly:

$ { ssh -V 2>&3 ; } 3>&1 | awk '{ print $1 }'  | awk -F '_' '{ print $2 }' | awk -F '.' '{ print $1 }'
7

(Source: https://stackoverflow.com/questions/9112979/pipe-stdout-and-stderr-to-two-different-processes-in-shell-script .) If I understand this correctly, you're running the original process and sending the stderr (file descriptor 2) to a new file descriptor 3. You then redirect file descriptor 3 to stdout (file descriptor 1) which you then send through a pipe. This is also confirmed to work with both v6.x and v7.x of OpenSSH.