More photos here, courtesy of Gillespie… http://www.facebook.com/album.php?aid=118955&id=523076809&ref=nf 

When optimizing performance the objective is always the same – get the complete page with all assets downloaded in the shortest possible time. This goal can be split into a number of distinct tasks – frontend code, backend code  and environment optimization.

The Front End

Traditionally this means the HTML developer will concentrate on compressing image assets as much as possible, be they gifs, jpgs, pngs or swfs. But with increasingly complex web pages, the story doesn’t just stop there. We also have to consider the overhead of downloading the html, css and javascript files required. This bottleneck  can be broken down in a few distinct parts.

• The actual amount of data that needs to be downloaded limits performance. You can reduce this by pre-compressing files sent from the server using tools like mod-deflate on Apache. You can also try removing redundant white space from html, css and javascript files with tools like Yahoo’s YUI Compressor,
• It takes time to make a GET request for each individual asset on the page. So the fewer individual assets on the sever you call, the quicker the page load. Yahoo’s YSlow can give you a few suggestions for speeding things along, but typical improvements would be combining multiple CSS or Javscript files into one file, combining images into a large tiled image and using CSS Sprite techniques to display the required image. We do this to a certain degree by making rollover state sprites for each navigation element, but this could be extended to have the entire navigation as a single image, then use the background-position property to display the part of the image we need.
• Eliminating unnecessary page reloading by using AJAX. As long as this is implemented using progressive enhancement, this can lead to impressive improvements in the perceived responsiveness of the application.

The Back End

Big gains can be found in the code that generates your dynamic content. Most web sites, be they blog, CMS, e-Commerce or web application, will talk to a database at some point.

• If your application is data driven, then you really need to look at query optimization first and foremost. Adding indexes to columns used in joins, or searched on, can produce easy wins. Making sure columns used in joins are of the same type and length can help, as does tweaking columns lengths to the absolute minimum. Then once you’ve got the data from the database, why keep re-running the query?
• Cache the query results to disk, or preferably to memory. Even better, cache the entire HTML output, so that the next request just pulls the page from an html cache. Many high volume blogs use WordPress SuperCache to ensure that the vast majority of visitors get served static HTML. It’s difficult to appreciate how much of a difference caching can make without some figures. For reference, my trusty MacBook Pro when run under Siege, spits out ~800 hits/sec when serving the static html of a WordPress page. But it struggles to achieve ~20 hits/sec when running from a default WordPress 2.7.1 install.
• Optimize your loops. Don’t do for( var; count(someothervar); var++)!
• Look at any includes paths that might be used. I’m not sure the situation on .NET, but in PHP having ill-defined include_path can crucify your application. Larger PHP Frameworks can consist of 100s or 1000s of individual files, and when you try to include these files, a hell of a lot of disk thrashing can occur unless it knows exactly were to look. So make sure you use absolute URIs when defining your paths.
• Clean your code up to remove any warnings you might see in the log files. This includes missing files, depreciated calls or the like.

The Environment

I really hadn’t appreciated quite how much speed can be found from the environment your application runs in. By environment I mean the combination of physical hardware and software that makes up your platform. My testing has been on a LAMP system, but I’m sure similar gains can be found on Windows if you know where to look. Obviously throwing more RAM, faster processors or just more boxes can speed things up. But more subtle changes can yield significant gains.

My current PHP build has a number of useful extensions, but was missing a PHP byte-code cache like APC. For years the Java and .NET crowd have said PHP can’t perform comparably, because PHP scripts have to be interpreted (compiled) every time a script runs. APC negates this argument. APC stores the compiled code generated when a PHP page runs in memory until it’s next needed, which means that your html output is generated more quickly. Does all this theoretical stuff work? Oh yes…

The Results

I ran Siege with my default application setup and achieved ~7.8 pages/sec,  about 670,000 pages view per day. That’s a lot of traffic. Then I cleaned up my include paths, setup some simple caching to disk of the most complex database query on the page and enabled APC. A quick reboot of Apache and suddenly I was getting ~22.2 pages/sec, about 1.9 million page views per days and much less database load. Disabling the query cache  gave me ~14.1 pages/sec. With APC disabled, but query caching enabled I got 9.33 pages/sec.

From this we can see a 3x improvement in performance for limited effort. We can also see that in this simple application, query caching gains us little in comparison to APC. However this should be caveated with the fact that the database server was running on the same machine. This is often not the case and hence there will be a higher overhead connecting to the database server. Also the load on the computer dropped significantly with query caching enabled, as  before MYSQL was chewing in excess of 20% CPU whilst under Siege. This dropped to nothing with caching enabled.

Conclusions

Some basic tweaks to the code and environment produced a ~3x improvement in performance and a reduce in CPU load. That’s huge!

All this thinking came about as a result of getting Siege installed on my MacBook. It allows you to actually quantify impericially how fast your application is, rather than thinking it feels a bit more snappy. I tested a simple CMS based on Zend Framework 1.8.2, running with PHP Version 5.2.9 www.entropy.ch Release 7 and MYSQL 5.1.34. The hardware was a MacBook Pro running OS X 10.5.7 with 2Gb RAM. As performance web servers go, a bad choice!

They don’t get out of the office much so if you see them, make sure you give them a big cheery wave.

http://www.guardian.co.uk/technology/blog/2009/apr/28/wolfram-alpha-demo
http://news.bbc.co.uk/1/hi/technology/8026331.stm