Introduction: Why the Linux Command Line Still Matters
The moment something breaks at 2 a.m. β a hung process, a full disk, a failed service β your browser-based control panel becomes useless. Linux powers the vast majority of the world's web servers, cloud infrastructure, containers, and embedded systems. The command-line interface is not a relic from the past; it is the primary interface for serious server work.
Scripts written in Bash run on a schedule and keep systems clean. SSH sessions give you full control over remote machines from anywhere. A single grep pipeline can sift through gigabytes of logs in seconds.
This tutorial is structured the way you actually use these tools: by task and scenario, from the basics you rely on every day to the advanced techniques that separate a junior sysadmin from a senior one. Every example was tested on real servers running Ubuntu, Debian, AlmaLinux, and Rocky Linux. Nothing here is theoretical padding.
Tutorial Roadmap
1. Navigating and Managing the Filesystem
2. User Accounts, Groups, and File Permissions
3. Process Management and System Monitoring
4. Service Management with systemd
5. Networking β Diagnostics and Configuration
6. Disk Management and Storage
7. Package Management
8. Bash Scripting for Server Automation
9. Advanced Commands for Power Users
10. A Practical Troubleshooting Framework
Quick Reference Cheat Sheet
2. User Accounts, Groups, and File Permissions
Access control is one of the most consequential responsibilities of a server administrator. Getting permissions wrong either locks people out of systems they need β or opens holes that attackers can walk through. There is no middle ground.
2.1 Managing User Accounts
| Command | What It Does |
|---|---|
useradd -m -s /bin/bash deploy | Create user 'deploy' with home directory and bash shell |
passwd deploy | Set or change the password for a user |
usermod -aG sudo deploy | Add user to the sudo group (Ubuntu/Debian) |
usermod -aG wheel deploy | Add user to the wheel group (RHEL/AlmaLinux/Rocky Linux) |
userdel -r olduser | Delete user and remove their home directory |
id deploy | Show user ID, group ID, and all group memberships |
whoami | Display the current logged-in username |
last | Show login history for all users |
w | Show who is currently logged in and what they are doing |
# Create a service account with no login shell (best practice for app users)
useradd -r -s /sbin/nologin -d /opt/myapp myapp
# Lock an account without deleting it
usermod -L username
# Unlock an account
usermod -U username
# Show all users on the system
cut -d: -f1 /etc/passwd
# Check what sudo privileges a user has
sudo -l -U username
2.2 File Permissions and Ownership
Linux file permissions are a three-part system: owner, group, and everyone else. Each part gets read (r=4), write (w=2), and execute (x=1) permissions independently.
| Command | What It Does |
|---|---|
chmod 755 /opt/app/script.sh | Owner: rwx, Group: r-x, Others: r-x β standard for executables |
chmod 644 /var/www/html/index.html | Owner: rw-, Group: r--, Others: r-- β standard for web files |
chmod 600 ~/.ssh/id_rsa | Readable only by owner β required for SSH private keys |
chmod -R 750 /opt/myapp/ | Apply permissions recursively to all files in directory |
chown www-data:www-data /var/www/html/ | Change owner and group of a file or directory |
chown -R deploy:deploy /opt/app/ | Change ownership recursively |
Understanding chmod Numbers
Each digit is a sum: read(4) + write(2) + execute(1) = 7 maximum
chmod 755 = owner:7(rwx) group:5(r-x) others:5(r-x) chmod 644 = owner:6(rw-) group:4(r--) others:4(r--) chmod 600 = owner:6(rw-) group:0(---) others:0(---)
Run 'ls -l' to read the permission string on any file.
2.3 Privilege Escalation and sudo
# Run a command with elevated privileges
sudo apt update
# Run a command as a specific user
sudo -u www-data php artisan migrate
# Open a root shell (use sparingly β stay in sudo for auditing)
sudo -i
# Edit the sudoers file safely β validates syntax before saving
visudo
# Give the 'deploy' user permission to restart nginx without a password
# Add this to /etc/sudoers.d/deploy :
# deploy ALL=(ALL) NOPASSWD: /bin/systemctl restart nginx
3. Process Management and System Monitoring
On a production server, you need to know what is running, how much resource it consumes, and how to intervene when something misbehaves. These commands give you full visibility into what your server is actually doing.
3.1 Viewing Processes
| Command | What It Does |
|---|---|
ps aux | Show all running processes with CPU and memory usage |
ps aux | grep nginx | Filter process list for a specific name |
pgrep nginx | Return the PID(s) for a named process |
top | Real-time interactive process monitor |
htop | Enhanced interactive viewer (install: sudo apt install htop) |
pstree | Show processes as a parent-child tree |
# Find the top 10 memory-eating processes
ps aux --sort=-%mem | head -10
# Find the top 10 CPU consumers
ps aux --sort=-%cpu | head -10
# Find exactly which process is using port 80
ss -tlnp | grep ':80'
# Alternative using lsof
lsof -i :80
3.2 Controlling and Killing Processes
| Command | What It Does |
|---|---|
kill 1234 | Send SIGTERM (graceful shutdown request) to PID 1234 |
kill -9 1234 | Send SIGKILL (force kill) β use only when SIGTERM fails |
killall nginx | Kill all processes named 'nginx' |
pkill -f 'python worker' | Kill processes matching a command pattern |
nice -n 10 ./backup.sh | Start a process with lower CPU priority |
renice 15 -p 1234 | Change priority of an already-running process |
3.3 Keeping Processes Running After Logout
When you disconnect from SSH, your shell session ends and takes any processes you started with it. Use these tools to keep long-running tasks alive.
# Run a process that survives SSH disconnection
nohup ./long-script.sh > /tmp/script.log 2>&1 &
# The better approach: tmux (a terminal multiplexer)
# Create a named session
tmux new -s deploy
# ... run your commands ...
# Detach without killing: press Ctrl+b, then d
# Reattach later from any SSH session:
tmux attach -t deploy
# List all tmux sessions
tmux ls
3.4 System Resource Monitoring
# System load, uptime, and number of logged-in users
uptime
# The three numbers are load averages: last 1, 5, and 15 minutes
# A load of 1.0 on a single-core server means 100% CPU utilized
# Memory usage summary
free -h
# Disk usage on all mounted filesystems
df -h
# How much space is a directory consuming?
du -sh /var/log
du -sh /var/log/* | sort -rh | head -20
# CPU, memory, disk I/O statistics sampled over time
vmstat 2 5 # Sample every 2 seconds, 5 samples
# Per-disk I/O statistics in real time
iostat -xz 1
4. Service Management with systemd
Every modern Linux distribution β Ubuntu 16+, Debian 8+, CentOS 7+, AlmaLinux, Rocky Linux β uses systemd. It manages your services, handles boot ordering, and captures logs. If you are still using init.d scripts on a server deployed in the last five years, it is worth learning systemd properly.
4.1 Controlling Services
| Command | What It Does |
|---|---|
systemctl start nginx | Start the nginx service immediately |
systemctl stop nginx | Stop the nginx service |
systemctl restart nginx | Stop and start the service β interrupts connections briefly |
systemctl reload nginx | Reload config without a full restart (where supported) |
systemctl status nginx | Check if it is running, its PID, and recent log lines |
systemctl enable nginx | Configure nginx to start automatically on boot |
systemctl disable nginx | Prevent nginx from starting automatically on boot |
systemctl list-units --type=service | List all loaded service units and their state |
systemctl --failed | Show all services that have failed |
# Full workflow: install, enable, start, verify
sudo apt install nginx
sudo systemctl enable nginx
sudo systemctl start nginx
sudo systemctl status nginx
# If a service fails to start, read why:
journalctl -u nginx.service -n 50 --no-pager
# If you create or edit a unit file, reload systemd first:
sudo systemctl daemon-reload
4.2 Reading Logs with journalctl
journalctl replaces reading raw /var/log/messages files and gives you powerful, structured log filtering built in. It is the log viewer for everything running under systemd.
# Follow nginx logs live (like tail -f but with structured filtering)
journalctl -u nginx.service -f
# Show logs from a specific service since a specific time
journalctl -u nginx.service --since '2026-03-20 08:00:00'
# Show only error-level messages from the last boot
journalctl -p err -b
# Show all kernel messages from the current boot
journalctl -k -b
# Show logs from the previous boot β very useful after a crash
journalctl -b -1
# How much space are journals consuming?
journalctl --disk-usage
# Trim old journals to reclaim space
sudo journalctl --vacuum-size=500M
5. Networking β Diagnostics and Configuration
Network troubleshooting is one of the most common tasks a server admin faces. Whether a service is unreachable, a firewall rule is blocking traffic, or DNS is resolving to the wrong address, you need the right commands to isolate the problem layer by layer.
5.1 Checking Network Interfaces and Routes
# Show all network interfaces with IP addresses
ip addr show
ip a # Shorthand
# Show the routing table
ip route show
# Show packet and error statistics for an interface
ip -s link show eth0
Note on Deprecated Commands
The 'ifconfig' and 'netstat' commands are deprecated on modern Linux systems.
Use 'ip addr' instead of 'ifconfig', and 'ss' instead of 'netstat'.
Both replacements are faster, more complete, and actively maintained.
5.2 Connectivity and DNS Testing
# Basic connectivity test
ping -c 4 google.com
# Trace the route packets take to a destination
traceroute google.com
# MTR: live combined traceroute + ping view (install: apt install mtr)
mtr google.com
# DNS resolution β what does this hostname resolve to?
dig google.com +short
# Reverse DNS lookup
dig -x 8.8.8.8
# Test if a specific port is reachable on a remote host
nc -zv 10.0.1.50 3306
# Output: Connection to 10.0.1.50 3306 port [tcp/mysql] succeeded!
# Test an HTTP endpoint
curl -I https://yourdomain.com
5.3 Port and Socket Inspection
# Show all listening TCP ports and which process owns them
ss -tlnp
# Show all established connections
ss -tnp state established
# Show UDP ports
ss -ulnp
# Find what process is using a specific port
ss -tlnp | grep ':443'
lsof -i :443
# Count connections to port 80 grouped by TCP state
ss -tan | awk 'NR>1 {print $1}' | sort | uniq -c | sort -rn
5.4 Firewall Management
Most modern Linux servers use either ufw (Ubuntu/Debian) or firewalld (RHEL-based distributions). Both are front-ends to the kernel's netfilter packet filtering system.
UFW β Ubuntu and Debian
# Always allow SSH before enabling ufw, or you will lock yourself out
sudo ufw allow 22/tcp
sudo ufw enable
sudo ufw status verbose
# Allow HTTP and HTTPS
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
# Restrict SSH to a specific IP address only
sudo ufw allow from 203.0.113.10 to any port 22
# Remove a rule
sudo ufw delete allow 8080
Firewalld β RHEL, AlmaLinux, Rocky Linux
# Check status
sudo firewall-cmd --state
sudo firewall-cmd --list-all
# Allow a service permanently
sudo firewall-cmd --permanent --add-service=https
# Allow a specific port
sudo firewall-cmd --permanent --add-port=8080/tcp
# Apply changes (required after --permanent flags)
sudo firewall-cmd --reload
5.5 Secure Remote Access with SSH
# Connect to a remote server
ssh user@192.168.1.100
# Connect on a non-standard port
ssh -p 2222 user@192.168.1.100
# Connect using a specific private key
ssh -i ~/.ssh/deploy_key user@192.168.1.100
# Copy files securely to a remote server
scp localfile.tar.gz user@192.168.1.100:/opt/app/
# Sync directories β only transfers what has changed
rsync -avz --progress /local/dir/ user@remote:/remote/dir/
# Port forwarding: access remote MySQL locally on port 3307
ssh -L 3307:127.0.0.1:3306 user@192.168.1.100 -N
# Generate a modern ed25519 SSH key pair
ssh-keygen -t ed25519 -C 'deploy@leoservers.com'
# Install your public key on a remote server
ssh-copy-id user@192.168.1.100
6. Disk Management and Storage
Running out of disk space on a production server causes crashes, data corruption, and service outages. A full /var/log directory can crash a web server as effectively as a hardware failure. Knowing how to monitor, investigate, and manage storage is not optional.
6.1 Disk Usage and Space Monitoring
# How full are all mounted filesystems?
df -h
# Which directories are eating the most space?
du -sh /var/log/* | sort -rh | head -20
du -sh /home/* | sort -rh | head -10
# Quick scan for all files over 500MB
find / -xdev -size +500M -exec ls -lh {} + 2>/dev/null
# Check inode usage β a full inode table causes 'No space left on device'
# even when df shows free space
df -i
# Monitor disk read/write I/O in real time
iostat -xz 2
iotop -o # Show only processes actively doing I/O
6.2 Partitioning, Formatting, and Mounting
# List all block devices
lsblk
# Detailed partition info for a specific disk
sudo fdisk -l /dev/sdb
# Format a new partition as ext4
sudo mkfs.ext4 /dev/sdb1
# Or as XFS (common on RHEL-based systems, better for large files)
sudo mkfs.xfs /dev/sdb1
# Mount it temporarily
sudo mount /dev/sdb1 /mnt/data
# To make it permanent across reboots, add to /etc/fstab
# Get the UUID first:
sudo blkid /dev/sdb1
# Then add a line like this to /etc/fstab:
# UUID=xxxxxxxx-xxxx-xxxx /mnt/data ext4 defaults 0 2
6.3 LVM β Logical Volume Management
LVM is used extensively in enterprise environments. It lets you resize volumes online, take snapshots for backups, and span storage across multiple physical disks without downtime.
# Show physical volumes, volume groups, and logical volumes
pvs && vgs && lvs
# Extend a logical volume by 20GB
sudo lvextend -L +20G /dev/mapper/ubuntu--vg-ubuntu--lv
# Grow the ext4 filesystem to use the new space
sudo resize2fs /dev/mapper/ubuntu--vg-ubuntu--lv
# For XFS (XFS can only grow, not shrink):
sudo xfs_growfs /mnt/data
7. Package Management
Installing, updating, removing, and auditing software packages is something you do constantly. The correct command depends on which Linux distribution you are running.
7.1 APT β Debian and Ubuntu
# Refresh the package index from all configured repositories
sudo apt update
# Upgrade all installed packages to their latest versions
sudo apt upgrade
# Install one or more packages
sudo apt install nginx php8.3-fpm mariadb-server
# Remove a package but keep its configuration files
sudo apt remove nginx
# Remove package AND its configuration files
sudo apt purge nginx
# Remove automatically installed dependencies that are no longer needed
sudo apt autoremove
# Search for a package
apt search 'web server'
# Show detailed info about a package (version, deps, description)
apt show nginx
# List all installed packages
dpkg -l | grep '^ii'
7.2 DNF β RHEL, AlmaLinux, Rocky Linux
# Update all packages (dnf supersedes yum on RHEL 8 and later)
sudo dnf update
# Install a package
sudo dnf install nginx
# Remove a package
sudo dnf remove nginx
# Search and get info
dnf search php
dnf info nginx
# List all installed packages
dnf list installed
# Enable the EPEL repository (Extra Packages for Enterprise Linux)
sudo dnf install epel-release
8. Bash Scripting for Server Automation
One-liners are great. But once you find yourself running the same sequence of commands repeatedly, that is the signal to write a script. Bash scripting is not traditional programming β it is orchestrating commands. Even basic scripting skill dramatically multiplies your effectiveness as a sysadmin.
8.1 Script Structure and Best Practices
#!/bin/bash
# Script: disk_alert.sh
# Purpose: Send an alert if any filesystem is over 85% full
set -euo pipefail
# -e = exit immediately on error
# -u = treat unset variables as errors
# -o pipefail = catch failures in pipe chains
THRESHOLD=85
ALERT_EMAIL='admin@yourdomain.com'
HOSTNAME=$(hostname -f)
df -h | awk 'NR>1 {gsub(/%/,"",$5); if ($5+0 >= THRESHOLD+0) print $0}' \
THRESHOLD=$THRESHOLD | while read -r line; do
echo "DISK ALERT on $HOSTNAME: $line" | \
mail -s "Disk Space Warning: $HOSTNAME" "$ALERT_EMAIL"
done
echo 'Disk check complete.'
8.2 Variables, Conditionals, and Loops
#!/bin/bash
APP_DIR='/opt/myapp'
# Conditional: check if a directory exists, create it if not
if [ -d "$APP_DIR" ]; then
echo "App directory exists"
else
mkdir -p "$APP_DIR"
echo "Created $APP_DIR"
fi
# Check if a command succeeded or failed
if systemctl restart nginx; then
echo 'nginx restarted successfully'
else
echo 'ERROR: nginx failed to restart' >&2
exit 1
fi
# Loop over all enabled nginx config files and test each one
for config in /etc/nginx/sites-enabled/*; do
nginx -t -c "$config" && echo "$config: OK"
done
8.3 Scheduling Tasks with Cron
Cron runs scripts on a schedule. This is how backups happen at 2 a.m., how logs get rotated weekly, and how SSL certificates get renewed without anyone touching the server.
# Edit the crontab for the current user
crontab -e
# Crontab format:
# .ββββββββββββ minute (0β59)
# | .ββββββββββ hour (0β23)
# | | .ββββββββ day of month (1β31)
# | | | .ββββββ month (1β12)
# | | | | .ββββ day of week (0=Sunday)
# | | | | |
# * * * * * command to execute
# Run a backup every day at 2:00 AM
0 2 * * * /opt/scripts/backup.sh >> /var/log/backup.log 2>&1
# Attempt Let's Encrypt certificate renewal twice per day
0 */12 * * * /usr/bin/certbot renew --quiet
# Clear temp files older than 7 days every Sunday at 3 AM
0 3 * * 0 find /tmp -mtime +7 -delete
# System-wide cron jobs go in /etc/cron.d/
9. Advanced Commands for Power Users
These are the tools that separate administrators who react to problems from those who can prevent them, investigate them deeply, and automate their way out of recurring ones.
9.1 Text Processing Pipelines
The real power of Linux comes from combining simple commands with pipes (|). Understanding awk, sed, and grep means you can extract exactly what you need from any output without writing a separate program.
# Count HTTP response codes in your Nginx access log
awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn
# Find the top 10 IP addresses hitting your server
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -10
# Extract log lines from the last hour only
grep "$(date +'%d/%b/%Y:%H')" /var/log/nginx/access.log
# Replace a string in a config file in-place
sed -i 's/old_hostname/new_hostname/g' /etc/hosts
# Show only lines 100β200 of a large log file
sed -n '100,200p' /var/log/syslog
# Parse PHP-FPM errors and extract unique file paths
grep 'PHP Fatal error' /var/log/php-fpm/error.log | awk -F'in ' '{print $2}' | sort -u
9.2 Deep System Diagnostics
# strace: trace every system call a process makes (powerful for debugging)
strace -p 1234 # Attach to a running process by PID
strace -c php-fpm # Summary of system calls during execution
# lsof: list all open files for a specific process
lsof -p 1234
# What files are preventing unmounting of /mnt/data?
lsof /mnt/data
# Check for Out-Of-Memory kills in the kernel ring buffer
journalctl -k | grep -i 'oom\|killed process'
dmesg | grep -i 'out of memory'
# Benchmark raw disk write speed
dd if=/dev/zero of=/tmp/testfile bs=1G count=1 oflag=dsync
# Generate a CPU performance report with sar
sar -u 2 30
9.3 Security Checks
# Check who is currently logged into the server
w
# See recent logins
last -20
# See recent failed login attempts
lastb | head -20
# Find which IPs are brute-forcing SSH
grep 'Failed password' /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -rn | head
# Check for SUID files (can be used as privilege escalation paths)
find / -perm -4000 -type f 2>/dev/null
# Look for unexpected outbound connections
ss -tnp state established
# Check SSL certificate expiry for a domain
echo | openssl s_client -connect yourdomain.com:443 2>/dev/null | openssl x509 -noout -dates
# Generate a strong random password
openssl rand -base64 24
9.4 Useful One-Liners
| Command | What It Does |
|---|---|
history | grep rsync | Search command history for a specific command |
sudo !! | Re-run the last command with sudo (when you forget it) |
Ctrl+R | Reverse search through command history interactively |
watch -n 5 'df -h' | Re-run df -h every 5 seconds β real-time disk monitoring |
curl -s ifconfig.me | Get the public IP of your server |
openssl rand -base64 24 | Generate a strong random password |
column -t file.txt | Format a text file into aligned readable columns |
10. A Practical Troubleshooting Framework
When something breaks on a production server, panic is your enemy. A systematic approach means you find the problem faster and cause less collateral damage in the process.
10.1 The Layered Diagnostic Approach
Work from the outside in. Start with the most visible symptoms and drill down to the root cause:
Step 1 β Is the server reachable at all? (ping, ssh)
Step 2 β Is the service actually running? (systemctl status)
Step 3 β What do the service logs say? (journalctl -u service -n 100)
Step 4 β Are system resources exhausted? (df -h, free -h, uptime, top)
Step 5 β Is the process listening on the correct port? (ss -tlnp)
Step 6 β Is the firewall blocking the connection? (ufw status or firewall-cmd --list-all)
Step 7 β Is DNS resolving correctly? (dig domain.com +short)
Step 8 β Did anything change recently? (journalctl --since '1 hour ago')
10.2 Scenario: Website Returning 502 Bad Gateway
# 1. Is nginx running?
sudo systemctl status nginx
# 2. Is PHP-FPM running?
sudo systemctl status php8.3-fpm
# 3. What does the nginx error log say?
sudo tail -50 /var/log/nginx/error.log
# 4. Is PHP-FPM actually listening on the socket or port nginx expects?
ss -tlnp | grep php
ls -la /run/php/
# 5. Is the disk full? (full disk causes log write failures which crash PHP-FPM)
df -h
du -sh /var/log/* | sort -rh | head -10
10.3 Scenario: Server is Slow β High Load Average
# 1. What is the load average?
uptime
# Rule of thumb: load average should be below the number of CPU cores
nproc # How many CPU cores does this server have?
# 2. Which specific process is consuming the most CPU?
ps aux --sort=-%cpu | head -10
# 3. Is it I/O wait? Look for 'wa' percentage in iostat
iostat -xz 1 5
iotop -o
# 4. Is it memory exhaustion causing swap thrashing?
free -h
vmstat 1 5
# Look at 'si' (swap in) and 'so' (swap out) columns β high values are bad
Quick Reference Cheat Sheet
Filesystem Commands
| Command | What It Does |
|---|---|
ls -lah | List with permissions, sizes, hidden files |
find / -name 'file.txt' | Find a file by name across the filesystem |
grep -rn 'pattern' /dir | Search file contents recursively |
tail -f /var/log/syslog | Follow a log file in real time |
tar -czvf archive.tar.gz /dir | Create a compressed archive |
tar -xzvf archive.tar.gz -C /dest | Extract archive to a destination path |
Users and Permissions
| Command | What It Does |
|---|---|
useradd -m -s /bin/bash user | Create a user with home directory |
usermod -aG sudo user | Add user to the sudo group |
chmod 755 file | Standard permissions for executables |
chown user:group file | Change file owner and group |
visudo | Safely edit the sudoers file |
Processes and Services
| Command | What It Does |
|---|---|
ps aux --sort=-%cpu | head | Top CPU-consuming processes |
kill -9 PID | Force kill a process by PID |
systemctl status service | Check if a service is running |
systemctl restart service | Restart a service |
journalctl -u service -f | Follow live service logs |
Networking
| Command | What It Does |
|---|---|
ip a | Show all network interfaces and their IPs |
ss -tlnp | Show listening ports with process names |
dig domain.com +short | Quick DNS lookup |
nc -zv host port | Test if a TCP port is open |
rsync -avz src/ user@host:/dest/ | Efficient file sync to remote server |
Disk and Storage
| Command | What It Does |
|---|---|
df -h | Disk space on all mounted filesystems |
du -sh /path/* | Space used by each item in a directory |
lsblk | List all block devices and mount points |
iostat -xz 1 | Real-time per-disk I/O statistics |
df -i | Check inode usage (not just disk space) |
