How to Optimize and Secure a Virtual Private Server (VPS) and Virtual Dedicated Server (VDS)

This article covers a wide range of methods for securing and optimizing your VPS/VDS. We do not provide any warranty for this article, so if you are not sure what you are doing please make sure you research before you do it.

Secure cPanel/WHM and the Root User on VDS:

Checking for formmail:

Form mail is used by hackers to send out spam email, by relay and injection methods. If you are using matts script or a version of it, you may be in jeopardy.

Command to find pesky form mails:
find / -name "[Ff]orm[mM]ai*"

CGIemail is also a security risk:
find / -name "[Cc]giemai*"

Command to disable form mails:
chmod a-rwx /path/to/filename
(a-rwx translates to all types, no read, write or execute permissions).

(this disables all form mail)

If a client or someone on your vps installs form mail, you will have to let them know you are disabling their script and give them an alternative.

Root kit Checker (rkhunter or chkrootkit)

Check for a root kits via a cron job, by doing this you will regularly check if your server is comprised, and you will be sent regular reports.

To install chkrootkit, login to the server as root and on the command line interface type:

cd /root/
wget ftp://ftp.pangeia.com.br/pub/seg/pac/chkrootkit.tar.gz
tar xvzf chkrootkit.tar.gz
cd chkrootkit-0.47
make sense

To run chkrootkit, type the following on the CLI:

/root/chkrootkit-0.47/chkrootkit

To ensure the highest level of security setup a cronjob which emails you the results on a regular basis.

Setup Email on Root Login (to detect breaches)

If an unauthorized person gains access to root, you want to be notified - you can do so by doing the following while logged into root:

cd /root
vi .bash_profile
Add the following line:
echo 'ALERT - Root Shell Access on:' `date` `who` | mail -s "Alert: Root Access from `who | awk '{print $6}'`" your@email.com

Where your@email.com is your email address.

Save an exit vi: :wq

To change the SSH Legal Message (displayed when you login via SSH), edit /etc/motd to display the message you wish to show.

Securing WHM and cPanel:

By default cPanel/WHM is not setup securely and efficiently, so you will want to optimize the cPanel/WHM settings by doing the following:

Go to: Server Setup -> Tweak Settings

Under Domains tick:
· Prevent users from parking/adding common internet domains (hotmail.com, aol.com, etc)

Under Mail tick:
· Attempt to prevent pop3 connection floods
· Default catch-all/default address behavior for new accounts - set this to FAIL

Under System tick:
· Use jailshell as default on new accounts

Go to: Server Setup -> Tweak Security
· Enable php open_basedir protection
· Enable mod_userdir protection
· Disable compilers for unprivileged users

Go to: Server Setup -> Shell Fork Bomb Protection
· Enable shell bomb/memory protection

When creating reseller packages, be sure to:
· Disallow creation of packages with shell acces
· Disallow creation of packages with full root access

Go to: Service Configuration -> FTP Configuration
· Disable anonymous FTP access

Go to: Account functions -> Manage Shell Acess
· Disable shell access for all users (except yourself)

Go to: MySQL -> Manage Root Password
· Change Root Password for MySQL

Go to: Security -> Quick Security Scan for Trojan Horses, and make sure you don't have any of the following infected:
· /sbin/depmod
· /sbin/insmod
· /sbin/insmod.static
· /sbin/modinfo
· /sbin/modprobe
· /sbin/rmmod

Update OS and Software:

If you are running cPanel:
· Update cPanel: /scripts/upcp
· Update Apache: /scripts/easyapache

If you are not running cPanel:
· Update OS and software: yum upgrade

General OS Security (do not need to be running cPanel):

Restict SSH access:
For improved security

Parts of this article were obtained from forum postings on WHT.