SSH certificates, as we mentioned in our blog outlining the problems with SSH keys, are the best way to avoid the many issues that come with using SSH keys. In this blog, we will take a deep dive into how SSH certificates work; before we can do that, though, we must review public key authentication.
Public key authentication contains pairs of cryptographic keys that have both a public key and a private key. These public and private keys are mathematically related in such a manner that you are able to verify that somebody has the private key without even knowing the private key, so long as you have the public key. Don’t worry, we aren’t here to bore you with the mathematical intricacies of public key authentication, just to go over the basics. Maybe in another blog.
Now that we’ve reviewed the basics of public key authentication, we should run through exactly how SSH works. So, first things first, you have a server and you have a client. The server contains all of the public keys that are authorized to authenticate into it in the “authorized_keys” file. Then, when a user wants to authenticate to that server, the following occurs:
1) The user’s computer sends a list of every single public key owned by the user.
2) The server checks if any of the keys are in the “authorized_keys” file.
3) If the server is able to match any of the keys with a key in the “authorized_keys” file, the server creates a challenge.
4) The user’s computer utilizes the private key to complete the server’s challenge and to then send the response to the server.
5) The server checks the returned value. If the returned value can be validated, then the session starts.
Of course, these steps were incredibly oversimplified for the purpose of getting to the explanation of SSH certificates, but (hopefully) you get the general gist. Unfortunately, this simply does not scale. Sure, it’s great for securing your home server or something on a similar level, but the same cannot be said for any server or endpoint with more complexity. What does scalability mean in this sense? A full time job. When we say cloud scale, we mean adding and maintaining authorized parties’ keys that might need to access a server at some point, for all of the thousands to millions of servers a company may have. This is what led to the creation of SSH certificates – a dream of making management across larger enterprises simpler.
When using SSH certificates for authentication, the CA’s key is the one that is trusted by the server; as such, any single SSH key that is signed by the CA’s key will be inherently trusted. We know that this can be a bit of a confusing notion, so let’s try to break it down. First things first, let’s go through the set up experience.
1) A Certificate Authority (CA) is created. In essence, this is an SSH key pair that just signs other SSH keys.
2) The CA’s public key is added to the server’s “trusted_ca_keys.pub” file.
3) The “sshd_config” file is then modified to look at the “trusted_ca_keys.pub” file as the sole source of truth for SSH certificate authorities.
These three steps cause the server to trust any SSH key that’s signed by the CA key, so long as it is trusted. Now that the CA is trusted, all that each engineer needs to do from now on is merely request their own certificate. Before we dive into how to do that, let’s review some certificate properties, shall we?
Public Key: This is the engineer’s public key that you would want to sign.
Key ID: This is simply an ID that you can give to the key.
Valid Principals: These are the Linux principals that the user can SSH as – for instance, should you want to gain access as a root, you’d need to add “root” to this section.
Valid After: This is the Unix time from when it’s valid from. The certificate will not be accepted at any time before this.
Valid Before: This is just the expiration date. We like to set ours as a few hours, but this can honestly be set as far out or close as your organization wishes.
Signature Key: This is the CA’s public key. It is used by the server in order to verify that the CA is a trusted CA.
Signature: This is the hash of all the certificate fields signed by the certificate authority private key.
1) Just 1 key needs to be added per server.
2) The user’s SSH keys now have a set expiration date.
3) The certificate authority can provide JEA to the user every time that the user requests access.
4) It is markedly easier to associate each key with its respective owner.
5) If short term certificates are used, then organizations have reduced exposure to SSH key theft brought about by SSH key mismanagement.
6) User onboarding goes by much faster.
1) The user’s computer sends a list of every single one of the user’s certificates.
2) The server checks if any of the sent certificates is signed by a CA in the “trusted_ca_keys.pub” file.
3) The server checks to see if the certificate is valid.
4) If the server finds a match, it creates a challenge.
5) The user’s computer completes the challenge and sends the response to the server using the certificate’s private key.
6) The server checks the returned value – if the server validates the returned value, the session starts.
What do these steps accomplish? They transfer key management from countless servers to a singular CA. Incredible!
First select the machine that will be your Certificate Authority. We recommend setting up a machine that’s only purpose is being a CA – this will help you protect your signing keys. If possible, we also recommend having your CA keys protected by a Hardware Security Module (HSM).
Once you have your machine setup, create your CA key:
ssh-keygen -f ca
Enter a strong passphrase to protect your private key. This will create two files, a “ca” file and “ca.pub” file. DO NOT share the “ca” file with anyone. That has your private key used to sign your certificates. Copy the ca.pub to all your servers (EZSSH saves it in a file called: “/etc/ssh/trusted_ca_keys.pub” in the servers). Then run the following line on the server you are setting up to accept certificates from this CA:
echo "TrustedUserCAKeys /etc/ssh/trusted_ca_keys.pub" >> /etc/ssh/sshd_config
This will let any SSH certificate signed by this CA authenticate to this server. Run the following command to restart the SSH service in the server.
service ssh restart
Now, your server is ready to accept SSH certificates.
Now a user can create an SSH key using:
ssh-keygen
This will create an id_rsa and id_rsa.pub files in your .ssh folder. NEVER SHARE your “id_rsa” file with anyone, this contains your private key.
Copy the “id_rsa.pub” to the certificate authority. On the CA run the following:
ssh-keygen -s ca -I YOURNAME -n root -V +1d -z YOURSERIALNUMBER id_rsa.pub
This will create a certificate signed by your CA. This certificate will have: your name as the “Key ID”, root as the “Valid Principal”, and will be valid for 1 day.
Copy the id_rsa-cert.pub into the users .ssh file. Now that you have a certificate you will be able to authenticate to your endpoint by running:
"ssh root@YOURENDPOINT"
This code is great for testing; however, it simply does not scale. Every time you would want to create a certificate, you would have to call someone in order to provide them with your public key. As we said, this setup is perfectly fine for testing, but by using it organization-wide, all it creates is bottlenecks, wasted time and frustration.
To make the whole process easier, check out EZSSH, our zero trust endpoint SSH management tool, where we automate certificate creation based on your AD user. The core principles of EZSSH are no agent required, enhanced security levels, and improved user experience. To learn more about how EZSSH can help your organization, schedule a FREE consultation with one of our experts today!
