SSH Security for GPU Hosts: Keys, Fail2ban, and Sleeping at Night

Your auth log is already full

Before changing anything, look at what is hitting the box:

sudo grep "Failed password" /var/log/auth.log | tail -20

On a freshly exposed host this fills up fast: root, admin, ubuntu, postgres, tried from IPs all over the world. Every attempt is a dice roll against your password. The fix is not a stronger password. The fix is removing passwords from the game entirely.

Switch to SSH keys

A key pair replaces the guessable secret with a private key that never leaves your laptop. Three steps:

1. 1Generate a key on the machine you connect from (not the host):

   ssh-keygen -t ed25519 -C "gpu-host-key"

   Ed25519 is the modern default: small keys, fast, no known weaknesses. Add a passphrase when prompted.
2. 2Copy the public key to the host:

   ssh-copy-id user@your-host-ip

   This appends your public key to ~/.ssh/authorized_keys on the host with the right permissions. If you are on Windows with PuTTY, the manual placement steps are in the SSH key setup guide in our open source Linux tools collection.
3. 3Test it from a new terminal before you touch anything else. You should land on the host without a password prompt.

Turn off password authentication

With keys proven, close the password door. Edit /etc/ssh/sshd_config and set:

PasswordAuthentication no
PubkeyAuthentication yes
PermitRootLogin prohibit-password

Then validate the config and restart the daemon:

sudo sshd -t && sudo systemctl restart ssh

From this point, the brute-force traffic is noise. An attacker without your private key cannot log in no matter how many guesses they burn.

Add fail2ban to cut the noise

Keys make guessing futile; fail2ban makes it expensive. It watches the auth log and firewalls any IP that fails too many times. Install and enable it:

sudo apt update && sudo apt install -y fail2ban
sudo systemctl enable fail2ban
sudo systemctl start fail2ban

The stock config works, but a small /etc/fail2ban/jail.local tightens it: 3 failed attempts within 10 minutes earns a 30-minute ban on the SSH jail. The complete file, ready to paste, is in the fail2ban guide. Check your handiwork with:

sudo fail2ban-client status sshd

Come back a day later and admire the banned list. It is oddly satisfying.

Know exactly what is listening

Renters need their forwarded port range and you need SSH. Everything else listening on a public interface is attack surface you did not mean to offer. Audit it:

ss -tuln

Anything bound to 0.0.0.0 that you cannot name deserves an investigation. The port check guide covers verifying ports from outside your network, which is the view that actually matters.

Two habits round this out. First, do not forward SSH itself unless you need remote access from outside your LAN; if you do, moving it off port 22 cuts the scanner noise, though it is camouflage, not protection, so keys and fail2ban still do the real work. Second, keep openssh-server patched. On a hosting rig with automatic updates turned off, that means including it deliberately in your monthly maintenance window.

The other key you need to protect

SSH is not the only credential attached to a hosting business. Your marketplace API key can change prices and read your account, so it deserves the same discipline: store it in one place, never paste it into random tools, and rotate it if you suspect exposure. The details are in creating and protecting your Vast.ai API key.

If you are still building out the box itself, the rest of the foundation (OS, drivers, Docker, networking) is covered in the Ubuntu server setup checklist. Do the security section the same day you forward the ports, not the week after.