SSH Hardening - Securing Your Linux Servers

Oct 6, 2025

Practical SSH hardening for production Linux servers — key-based auth, sshd_config, 2FA, host-based auth, fail2ban, and log monitoring.

The default SSH configuration on most distributions is functional but not production-safe. After managing Linux infrastructure for several years — and finding over 50,000 failed login attempts in a single day’s auth log early in my career — I apply the same hardening steps to every server I manage.

Warning

Never lock yourself out. Always test each change in a separate SSH session before closing your original connection.

Guide Structure

Key-Based Authentication — replace password auth with cryptographic keys
SSH Daemon Hardening — production-ready sshd_config
Two-Factor Authentication — TOTP on top of key auth
Host-Based Authentication — automated server-to-server trust
Security Monitoring — fail2ban, connection management, log analysis
Troubleshooting & Best Practices — common issues, compliance, maintenance

Key-Based Authentication Setup

merox

Oct 6, 2025

#ssh #security #authentication

Replace password authentication with SSH keys to eliminate brute-force attack vectors

Password authentication can be compromised through brute-force, keyloggers, or credential stuffing. Keys eliminate all of that.

Generate Key Pair

On your local machine — not the server:

# ED25519 recommended
ssh-keygen -t ed25519 -C "your_email@example.com" -f ~/.ssh/id_prod_server

# RSA fallback for older systems
ssh-keygen -t rsa -b 4096 -C "your_email@example.com" -f ~/.ssh/id_prod_server

ED25519 is faster, more secure, and uses shorter keys than RSA. I’ve switched all my infrastructure to it.

Deploy Public Key

ssh-copy-id -i ~/.ssh/id_prod_server.pub username@server_ip

# If ssh-copy-id isn't available
cat ~/.ssh/id_prod_server.pub | ssh username@server_ip "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

Test Before Disabling Passwords

ssh -i ~/.ssh/id_prod_server username@server_ip

Confirm you can log in without a password before touching anything else.

Fix Permissions

SSH won’t use keys with incorrect permissions:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_ed25519

Next Steps

Proceed to the SSH Daemon Hardening guide.

SSH Daemon Configuration Hardening

merox

Oct 6, 2025

#ssh #sshd #security #hardening

Production-ready sshd_config settings with modern cryptography and security best practices

The real hardening happens in /etc/ssh/sshd_config.

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.backup.$(date +%F)
sudo nano /etc/ssh/sshd_config

Production Configuration

# Network Settings
Port 2222
AddressFamily inet
ListenAddress 0.0.0.0

# Authentication
PermitRootLogin no
PubkeyAuthentication yes
PasswordAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no
UsePAM yes

# Key Types (ED25519 preferred)
PubkeyAcceptedKeyTypes ssh-ed25519,rsa-sha2-512,rsa-sha2-256

# Limit user access
AllowUsers deployer sysadmin
# AllowGroups ssh-users

# Session Settings
MaxAuthTries 3
MaxSessions 2
LoginGraceTime 30
ClientAliveInterval 300
ClientAliveCountMax 2

# Disable Dangerous Features
X11Forwarding no
PermitUserEnvironment no
AllowAgentForwarding no
AllowTcpForwarding no
PermitTunnel no

# Logging
SyslogFacility AUTH
LogLevel VERBOSE

# Modern Cryptography (2025)
KexAlgorithms curve25519-sha256,curve25519-sha256@libssh.org
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com

# Security
HostbasedAuthentication no
IgnoreRhosts yes

Validate and Restart

sudo sshd -t
sudo systemctl restart sshd
sudo systemctl status sshd

Caution

Keep your current session open. Open a new terminal and test the connection before closing the original.

Update Firewall

# UFW (Ubuntu/Debian)
sudo ufw allow 2222/tcp
sudo ufw delete allow 22/tcp
sudo ufw reload

# firewalld (RHEL/CentOS)
sudo firewall-cmd --permanent --add-port=2222/tcp
sudo firewall-cmd --permanent --remove-service=ssh
sudo firewall-cmd --reload

Next Steps

Proceed to the Two-Factor Authentication guide.

Two-Factor Authentication with Google Authenticator

merox

Oct 6, 2025

#ssh #2fa #totp #security

Add TOTP-based 2FA to SSH for defense-in-depth security

Even if someone steals your private key, 2FA means they still can’t get in without the second factor.

Install

# Ubuntu/Debian
sudo apt install libpam-google-authenticator

# RHEL/CentOS
sudo yum install google-authenticator

Configure

google-authenticator

Prompts to answer: time-based tokens → Yes, update ~/.google_authenticator → Yes, disallow multiple uses → Yes, increase time window → No (unless you have time sync issues), enable rate-limiting → Yes.

Scan the QR code with Google Authenticator, Authy, or any TOTP app.

PAM Configuration

sudo nano /etc/pam.d/sshd

Add at the top:

auth required pam_google_authenticator.so nullok

nullok lets users without 2FA configured still log in. Remove it once all users have it set up.

Enable in sshd_config

ChallengeResponseAuthentication yes
AuthenticationMethods publickey,keyboard-interactive

sudo systemctl restart sshd

Connections now require both your SSH key and the 2FA code.

Next Steps

Proceed to the Host-Based Authentication guide for automated server-to-server trust.

Host-Based Authentication for Trusted Servers

merox

Oct 6, 2025

#ssh #host-based-auth #automation #enterprise

Configure automated server-to-server authentication for backup systems, monitoring, and CI/CD pipelines

Host-based authentication lets one server authenticate to another based on the client machine’s host key rather than user keys. I use it for automated backup systems, Ansible/Puppet, monitoring that executes remote commands, database replication, and CI/CD pipelines.

Warning

Only use this in controlled environments where you fully trust the client machines. It’s a complement to user key auth for specific automation use cases, not a replacement.

Prerequisites: Root access on both machines, DNS or /etc/hosts entries for hostname resolution.

Step 1: Enable on Server

Edit /etc/ssh/sshd_config on the server:

HostbasedAuthentication yes
HostbasedUsesNameFromPacketOnly yes
HostbasedAcceptedKeyTypes ssh-ed25519,rsa-sha2-512,rsa-sha2-256
IgnoreRhosts no
IgnoreUserKnownHosts no

sudo systemctl restart sshd

Step 2: Configure Trusted Hosts on Server

Create /etc/ssh/shosts.equiv:

# Format: hostname [username]
backup-server.example.com deployer
monitoring.example.com monitor
ci-runner-01.example.com jenkins

sudo chmod 600 /etc/ssh/shosts.equiv
sudo chown root:root /etc/ssh/shosts.equiv

For per-user trust, use ~/.shosts with the same format.

Step 3: Configure Client

Edit /etc/ssh/ssh_config on the client:

HostbasedAuthentication yes
EnableSSHKeysign yes
PreferredAuthentications hostbased,publickey,password

Step 4: Configure ssh-keysign

ssh-keysign must be setuid root to access host keys:

sudo chmod 4711 /usr/lib/openssh/ssh-keysign
# or
sudo chmod 4711 /usr/libexec/openssh/ssh-keysign

Step 5: Distribute Host Public Keys

On the client, get the host public key:

sudo cat /etc/ssh/ssh_host_ed25519_key.pub

Add it to /etc/ssh/ssh_known_hosts on the server:

# Format: hostname,ip key-type public-key
backup-server.example.com,192.168.1.10 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILo...

sudo chmod 644 /etc/ssh/ssh_known_hosts
sudo chown root:root /etc/ssh/ssh_known_hosts

Step 6: Test

ssh -v deployer@production-server.example.com
# Look for: "Authentication succeeded (hostbased)"

Automation Script

Save as /usr/local/bin/distribute-host-keys.sh on the server to collect keys from multiple clients:

#!/bin/bash
KNOWN_HOSTS="/etc/ssh/ssh_known_hosts"
TEMP_KEYS="/tmp/host_keys_collection.txt"

CLIENTS=(
    "backup-server.example.com"
    "monitoring.example.com"
    "ci-runner-01.example.com"
)

> $TEMP_KEYS

for client in "${CLIENTS[@]}"; do
    IP=$(dig +short $client | tail -1)
    KEY=$(ssh-keyscan -t ed25519 $client 2>/dev/null)

    if [ -n "$KEY" ]; then
        echo "$client,$IP $(echo $KEY | awk '{print $2, $3}')" >> $TEMP_KEYS
    else
        echo "Failed to get key from $client"
    fi
done

[ -f $KNOWN_HOSTS ] && cp $KNOWN_HOSTS ${KNOWN_HOSTS}.backup.$(date +%F)

cat $TEMP_KEYS >> $KNOWN_HOSTS
sort -u $KNOWN_HOSTS -o $KNOWN_HOSTS
chmod 644 $KNOWN_HOSTS

sudo chmod +x /usr/local/bin/distribute-host-keys.sh
sudo /usr/local/bin/distribute-host-keys.sh

Real-World Example: Backup Server

On production servers (/etc/ssh/sshd_config):

HostbasedAuthentication yes
Match User backup
    HostbasedAuthentication yes
    PasswordAuthentication no

On production servers (/etc/ssh/shosts.equiv):

backup-server.example.com backup

On backup server (/etc/ssh/ssh_config):

Host prod-*
    HostbasedAuthentication yes
    PreferredAuthentications hostbased
    User backup

Now the backup server can pull automatically:

rsync -avz prod-web-01:/var/www/ /backup/web-01/

Security Notes

Combining with user key auth is the safest approach:

AuthenticationMethods publickey,hostbased

Review /etc/ssh/shosts.equiv monthly. Ensure LogLevel VERBOSE is set so host-based authentications are logged. Restrict SSH access by firewall to trusted client IPs only.

Revoking Access

# Remove from shosts.equiv
sudo nano /etc/ssh/shosts.equiv

# Remove from known_hosts
sudo ssh-keygen -R hostname.example.com -f /etc/ssh/ssh_known_hosts

sudo systemctl restart sshd

Troubleshooting

# Server logs
sudo journalctl -u sshd -n 50 | grep hostbased

# Verify hostname resolution
hostname -f  # must match what's in shosts.equiv

# Check ssh-keysign permissions
ls -l /usr/lib/openssh/ssh-keysign
# Should be: -rws--x--x (4711)

# Verify host key on server
sudo grep "$(hostname)" /etc/ssh/ssh_known_hosts

# Full debug from client
ssh -vvv -o PreferredAuthentications=hostbased user@server

Next Steps

Proceed to the Security Monitoring guide.

Security Monitoring and Connection Management

merox

Oct 6, 2025

#ssh #fail2ban #monitoring #logging

fail2ban protection, SSH connection config, and log monitoring for hardened SSH

Fail2Ban

Fail2ban monitors logs and blocks IPs that show malicious behavior.

# Ubuntu/Debian
sudo apt install fail2ban

# RHEL/CentOS
sudo yum install epel-release && sudo yum install fail2ban

Create /etc/fail2ban/jail.local:

1
[DEFAULT]
2
bantime = 3600
3
findtime = 600
4
maxretry = 3
5
destemail = your_email@example.com
6
sendername = Fail2Ban
7
action = %(action_mwl)s
8

9
[sshd]
10
enabled = true
11
port = 2222
12
filter = sshd
13
logpath = /var/log/auth.log
14
# logpath = /var/log/secure  # RHEL/CentOS
15
maxretry = 3
16
bantime = 3600

sudo systemctl enable fail2ban
sudo systemctl start fail2ban

# Status and management
sudo fail2ban-client status sshd
sudo fail2ban-client get sshd banned
sudo fail2ban-client set sshd unbanip 192.168.1.100

SSH Client Config

On your local machine, ~/.ssh/config:

Host production-server
    HostName server_ip
    Port 2222
    User deployer
    IdentityFile ~/.ssh/id_prod_server
    ServerAliveInterval 60
    ServerAliveCountMax 3

Host staging-server
    HostName staging_ip
    Port 2222
    User deployer
    IdentityFile ~/.ssh/id_staging_server
    ProxyJump bastion-host

ssh production-server

Log Monitoring

# Real-time
sudo tail -f /var/log/auth.log       # Ubuntu/Debian
sudo tail -f /var/log/secure         # RHEL/CentOS

# Failed attempts
sudo grep "Failed password" /var/log/auth.log | tail -20

# Successful logins
sudo grep "Accepted publickey" /var/log/auth.log | tail -20

Monitoring Script

Save as /usr/local/bin/ssh-monitor.sh:

#!/bin/bash
LOG_FILE="/var/log/auth.log"
REPORT_FILE="/var/log/ssh-security-report.txt"

echo "SSH Security Report - $(date)" > $REPORT_FILE
echo "================================" >> $REPORT_FILE
echo "" >> $REPORT_FILE

echo "Failed Login Attempts:" >> $REPORT_FILE
grep "Failed password" $LOG_FILE | awk '{print $1, $2, $3, $11}' | sort | uniq -c | sort -nr | head -20 >> $REPORT_FILE
echo "" >> $REPORT_FILE

echo "Successful Logins:" >> $REPORT_FILE
grep "Accepted publickey" $LOG_FILE | awk '{print $1, $2, $3, $9, $11}' | tail -20 >> $REPORT_FILE
echo "" >> $REPORT_FILE

echo "Active SSH Sessions:" >> $REPORT_FILE
who >> $REPORT_FILE
echo "" >> $REPORT_FILE

echo "Current Fail2Ban Bans:" >> $REPORT_FILE
fail2ban-client status sshd 2>/dev/null >> $REPORT_FILE

cat $REPORT_FILE

sudo chmod +x /usr/local/bin/ssh-monitor.sh

# Daily email report
echo "0 9 * * * /usr/local/bin/ssh-monitor.sh | mail -s 'SSH Security Report' your_email@example.com" | sudo crontab -

Next Steps

Proceed to the Troubleshooting and Best Practices guide.

Troubleshooting and Best Practices

merox

Oct 6, 2025

#ssh #troubleshooting #compliance #maintenance

Common SSH issues, compliance requirements, and ongoing maintenance for production environments

Troubleshooting

Can’t Connect After Changes

sudo systemctl status sshd
sudo ufw status           # or: firewall-cmd --list-all
sudo sshd -t
sudo journalctl -u sshd -n 50

Permission Denied (publickey)

ls -la ~/.ssh
# .ssh: 700 | authorized_keys: 600 | private keys: 600

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Too Many Authentication Failures

Multiple keys in your SSH agent will trigger this:

ssh-add -D
ssh-add ~/.ssh/id_prod_server

# Or force a specific key
ssh -o IdentitiesOnly=yes -i ~/.ssh/id_prod_server user@server

2FA Code Not Working

timedatectl status
sudo systemctl restart chrony   # or ntpd

Monthly Security Audits

# Review authorized_keys
cat ~/.ssh/authorized_keys

# Check for weak host keys
for key in /etc/ssh/ssh_host_*_key.pub; do ssh-keygen -lf $key; done

# Anomalies in auth logs
sudo grep -i "POSSIBLE BREAK-IN" /var/log/auth.log

# Users with empty passwords
sudo awk -F: '($2 == "") {print $1}' /etc/shadow

Key Rotation

I rotate SSH keys annually: generate new pair → deploy to all servers → test → remove old public key → update documentation.

Enterprise Documentation

Track: SSH configuration changes, authorized users and their keys, justification for any non-standard settings, incident response procedures, key rotation schedule.