SUDO has long since been removed from Dreamhost making this invalid

If you are a Dreamhost customer, you may have popular enough websites that you switched from their stellar shared hosting to a Virtual Private Server (VPS). They offer both your main Apache/PHP server as well as a MySQL VPS. I switched to these shortly after it became available, to ensure maximum control over restarting my servers, troubleshooting problems, and scalable resources. The latter option is a real perk: You can change the memory/CPU allocation for your VPS as activity grows on your websites. On the down side, once those resources are used up, specifically memory, your server will determine that memory was exceeded and all processes will be killed.

When this happens, for a minute or two, all of your websites are returning a 500 error code; This is bad. Luckily, you can simply up your resources for a few more dollars a month and go on your merry way. However, if you are trying to write the best code possible, you probably aren’t satisfied with throwing money at the problem. That’s where another perk of a VPS comes in: Root access and SSH access to your server.

Really quickly, I’m going to back up and add a caveat. For my busiest websites, I’ve pushed my static resources (CSS, JavaScript, and image files) to Amazon Web Services (AWS) S3 and CloudFront, serving as a very fast Content Delivery Network (CDN). This added only $2/month to my expenses, and improved website loading speeding by more than 50%, as well as reduced the load on my Dreamhost VPS server (where the costs are much higher for less benefit). I recommend this 100%, and coming soon, a full tutorial on setting up a CDN will be posted on SeoMoz.org (link as soon as it is posted).

The Problem

Okay, back to the point. Even after all the previous tweaks and improvements, I found myself hitting my memory maximum multiple times in the day. I had just added a new domain to my growing list of websites, which means another chunk of memory for PHP5.cgi, which put me over the top. However, I wasn’t going to be idle: I went to work to find out how often this happening and what else might be causing it. The side effect of this exploration is not only this post, but also 30 hours of research and coding over a 48 hour period. To use all of the tools mentioned below, you will need to start by making a root user through the Dreamhost panel.

Super or Admin User

Login to the panel and head over to the Admin Users section of the VPS menu. You will want to create a root user with a different username and password as your main account, and preferably with a unique password that is not even close to a password used by you anywhere else on the web. You’ll receive an email confirmation when the user has been created. Excellent! Now, a quick primer on using this new root user through SSH (If you are on Windows, check out the program called PuTTY).

When you want to run a command that is restricted, you will typically see an error message about Permission Denied. The solution is to proceed that command with ‘sudo‘ (no quotes), which is short for SuperUser DO. This is standard practice on Debian Linux, which is the Linux software flavor of Dreamhost VPS. I recommend using sudo when needed, but there is one more, very dangerous option: ‘sudo -s‘. This switches you to a root shell, which CAN BE VERY RISKY. Please note that if you logged into your server with a username that does not have root privileges, you can switch by typing ‘su {username}‘, where {username} represents the root user (Admin User) you setup through the Dreamhost Panel. Whenever possible, if a command requires root privileges, I’ve included it in the command.

The Solution

First and foremost, it’s important to know what your memory usage is at any given time. To do that, use a command called ‘free‘. More specifically, I recommend ‘free -m‘ to specify the usage in MBs, or to see the results on a continuous scrolling basis, ‘free -m -s 10‘. This tells free to use MBs and update every 10 seconds. This allows you to have a scrolling list for monitoring changes. If you prefer a non-scrolling method, use ‘watch -interval=10 free -m‘ or ‘watch -d -interval=10 free -m‘ to have a refreshed view of the data, with the latter -d addition highlighting differences each change. If you remove the -interval=10 option, the default 2 second interval is used. To quickly understand what you are seeing, pay attention to the FREE column. Here is an example of the output:

		total	used	free	shared	buffers	cached
Mem:		633	553	79	0	0	240
-/+ buffers/cache:	298	334
Swap:		0	0	0

The Mem/total (row/column) is the total memory you have setup for your VPS through the Dreamhost panel. The Mem/free (79) represents the amount of unused memory. Linux likes to use memory to make things more efficient, so this one isn’t as worrisome if it is low as the next number: in the next row, buffers/cache, the free column (334) represents the amount of memory that is free after the buffers/cache has been considered. If that number is low, you’re likely on your way to a temporary crash of your processes. This StackExchange post adds some extra info.

Okay, so you have monitoring down, but next you need to figure out what is causing all the usage! I’ll assume that for most websites, the usage will be driven by Apache and PHP. That means you need to find out what is happening, in real-time, with requests to your web server (Apache) when your memory values dip low. It’s time to install an additional piece of software on your VPS: ApacheTop.

Thanks to a recommendation from a friend and two posts on the software, here is how you install the software:

sudo apt-get install apachetop

Yes, it’s that simple. Type ‘Y’ to any prompts you receive. Congratulations, it’s now installed! Now, to run it is a little trickier. Dreamhost stores your access.log file in a different place, and it will be different for every domain you host. Thus, if you host myserver.com and myotherone.com on your Dreamhost VPS, you must monitor them separately. Here is the command, using the domain name myserver.com:

sudo apachetop -f /home/{username}/logs/myserver.com/http/access.log

Like previously, replace {username} with the username for your account (not the ROOT username, but the username you setup in the panel under Users > Manage Users. Also, change the myserver.com to your actual domain (no www). If successful, and after entering your password for sudo, you should see a new window (to exit,
use CTRL+c). As soon as anyone visits your website, you will see data flow in. The default viewing time is 30 seconds, and you can see an extensive listing of all of the pages, scripts, images… anything being pulled from your server. This is usually where your jaw will drop to the ground, as you go, “WTF, why is that loading,” or “That’s a lot of request for xxx!” Don’t forget to get some sleep.

If you wish to dig in further, such as with CPU usage, processes, and more, here are three more commands of interest:

top
 
pstree -p
 
sudo pmap -x <processID> | grep total

I hope this is helpful for you. I intend to create a small PHP file that can be used as an Active System Monitor for CPU and Memory usage in your browser, and distribute it freely. I say Active, because it will not store long-term data, and only display statistics while the page is open, graphing (if I have time) history only as long as the browser page is open. Till then, you will need to use SSH and the above tips.

Side note: If you don’t currently use Dreamhost, but want to, this coupon will save you $50:
FOOL50 (that is the word fool and the number 50)