Did you know that, according to a 2017 study from the University of Maryland, a cybersecurity attack happens every 39 seconds? And that was in 2017 – that number has only gotten smaller and more alarming in the years since, highlighting the critical need for organizations to prioritize reinforcing their defense strategies. Traditional security methods like firewalls, VPNs, and password policies – although essential facets of a security plan – do not address inherent security risks present at the code or infrastructure levels within organizations. This gap might explain the shift towards more unconventional security domains such as compliance, devops, and engineering.
Speaking to devops, the focus there is increasingly on safeguarding infrastructure. Often, this protection primarily revolves around the SSH network protocol. SSH is crucial for secure remote access, authentication, and encrypted data transfers. In standard server-client SSH configurations, the server accepts a client if its public key matches those authorized, and clients trust a server’s public key during their initial interaction; however, this configuration is not necessarily all it’s cracked up to be.
There are definitely a few things to watch out for with a basic server-client SSH setup. Firstly, the authorized_keys files can grow too large and become difficult to manage. Secondly, many SSH clients follow a “trust on first use” (TOFU) policy, which places the burden of deciding what to trust on the user – this can lead to users ignoring warnings and inadvertently connecting to untrusted hosts, exposing their machines to potential man-in-the-middle (MITM) attacks.
A possible solution? Shifting from public keys to SSH certificates and CAs for establishing trust. SSH certificates allow administrators to implement specific rules for their issuance and set expiration guidelines. Check out our blog on how to set up SSH certificates to learn more about setting them up on your own!
Globally, some of the largest companies are addressing SSH vulnerabilities by adopting significant changes, such as not issuing keys or implementing a zero trust model for network user or device authentication and authorization. The concept of zero trust challenges the traditional trust assumptions about internal network traffic and access granted to employees and third parties.
Google’s BeyondCorp model is a prominent example of zero trust implementation. Unlike other companies that have added multiple layers of firewalls, restrictions, and VPNs for system access, Google has taken a different approach – instead of relying on a network perimeter, it has made its internal systems accessible over the public internet! This approach necessitates that all systems within the BeyondCorp framework be as robustly built as and fortified as those intended for the public internet.
Now that the background is out of the way, let’s take a look at 3 of the world’s most prominent companies and how they approach SSH:
Facebook’s security team prefers to use certificates over public key authentication for several reasons:
1) Managing local accounts becomes increasingly complex as the company expands.
2) Relying on central authentication creates a risky single point of failure.
3) Scaling the trust of individual key-pairs is practically unfeasible.
In crafting a solution that suits their needs, Facebook adopted a concept commonly used in managing HTTPS traffic – they set up their SSH servers to trust their CA and any certificate it authenticates. The subsequent step involves authorization; the certificate includes specific access rights and privileges for individual employees. This ensures that each person has precisely the level of access necessary for their role.
Netflix’s security team created its BLESS (Bastion’s Lambda Ephemeral SSH Service) CA, integrating it with the company’s engineering ethos of “freedom and responsibility.” This approach expects engineers to:
1) Actively and transparently share information.
2) Explain all the reasons behind their actions.
3) Manage all services developed by their team.
Consequently, nearly every engineer at Netflix requires SSH access to their services, with the aim of minimizing barriers to this access.
BLESS operates on AWS Lambda and employs Amazon’s Key Management Service for encryption. The SSH bastion architecture at Netflix streamlines the process by utilizing SSO for user authentication and issuing certificates that are valid for only a short duration. Additionally, the company enhances its security through automated monitoring of SSH activities and triggering alerts for any abnormal or suspicious behavior.
Uber’s security team identified several issues with the traditional SSH public/private keypair model. Key management was problematic, and there was no automated system to set expiration dates for keys. The longer a key remained valid, the higher the risk of it being lost or compromised.
Another concern was the overwhelming frequency of 2FA requests faced by employees. Constant 2FA prompts could lead to alert fatigue, potentially causing employees to indiscriminately approve any old 2FA notification, a big security no-no.
Although most of these challenges were addressed when Uber switched to using SSH certificates with OpenSSH 5.4, they needed a system that could issue both user and host certificates and continuously authenticate users, rather than just at a single moment. To meet these needs, Uber developed the Uber SSH Certificate Authority (USSHCA) and a PAM module to ensure ongoing user validity. SSH certificates, each with a set lifespan and configurable based on the employee’s role or group, are issued to Uber employees by the USSHCA.
If you’re reading this article, then SSH management is, at the very least, something that you’re interested in and could even be something that you’re trying to make easier within your organization. While you can create your own SSH Certificate Authority in Linux in minutes, if you want to do SSH better than some of the biggest companies in the world and at a fraction of the cost, consider using EZSSH! EZSSH simplifies the process by eliminating the need for managing, updating, and deleting SSH keys across all users and hosts – this is achieved through the use of SSH certificates operating behind the scenes. For the user, the process feels like the usual SSO experience, utilizing their Azure AD (or, Entra ID) identity. This approach effectively removes the risk of SSH keys on engineers’ desktops being compromised by bad actors.
If this is something of interest to you, or if you just want to learn more, feel free to schedule a FREE consultation with one of our ex-Microsoft identity experts to discuss how EZSSH can help your organization SSH like no one else can without requiring you to take out a second mortgage to afford it.
