PHP Framework benchmarking

For the past several months I have been in the process of creating a PHP framework. My framework, ZombiePHP, is nearing the point of release, and so I decided I ought to benchmark it against other popular frameworks to see how I stack up in terms of speed. I tested ZombiePHP, Symfony, Yii, CodeIgniter, CakePHP, and DooPHP.

All tests were performed on my laptop. Here are the specs:


Processor: intel i7 2.0Ghz
RAM: 8G
OS: Ubuntu 11.04
PHP: Version 5.3.5 with APC
Database: MySQL 5.1


My good friend, ab, performed the benchmarking. The same command was used for all benchmarks:


ab -t 30 -c 30 <url>


Test 1:


This test consists of unpacking the framework, getting a hello world application running, and checking the requests per second, stack size, and number of files included.

All of the frameworks, with the exception of DooPHP, came with a default layout. These were all included. This probably gives Doo a slight advantage in speed, but we won't worry about it for this benchmark.

The stack size and files included were checked at the time the view was rendered. These statistics were not calculated during the RPS benchmark.

The stack size and files included are just informational, however, the number of files included relates closely to the performance of the framework.

Hello world with default template
	Stack Size	Files Included	RPS
DooPHP	6	13	4500
ZombiePHP	9	19	3060
CodeIgniter	6	22	2570
Yii	13	21	1080
CakePHP	8	38	520
Symfony	12	81	430

Test 2:


The second test required the framework to retrive the contents of a mysql table and display them on the page. The mysql table consisted of two rows and two columns.

I added the column below showing what percent slower the frameworks are when adding database access. This is kind of important considering that database access is a key element of any web application.

Symfony was the woe of my life for a previous job I had, so I was able to get it working in a few minutes due to previous experience. I does make me want to stick my head in boiling oil every time it wants to drop my database. Who's idea was that? Overall I don't like it.

CodeIgniter appeared to be simple, but the tutorial on the homepage DID NOT WORK. Seriously? Who has a flagship tutorial that doesn't work? I decided not to waste any more time with CodeIgniter.

CakePHP had a plethora of tutorials, but I couldn't find any on getting data from a database. Again, WTH?

Yii came with a nice tool called "Gii", which had all sorts of code generators. It took a minute to figure out and I had to change a bunch of permissions in the yii folder, but overall it was pretty easy and I didn't mind using it.

DooPHP made me do everything myself (no code generator), but has a very clever ORM which only took a couple minutes to figure out. The tutorial was a little vague, and maybe a bit incomplete, but I forgive them on account of it not being hard to figure out.

ZombiePHP is my baby, and it comes with database interaction as a default module, so that was pretty easy.

DB Access
	Files Included	RPS	% Slower
DooPHP	16	3850	14%
ZombiePHP	24	2530	17%
Yii	63	390	63%
Symfony	138	281	34%
CodeIgniter	How can your flagship tutorial not work!?
CakePHP	I could not find a simple tutorial on getting data from a database.

Conclusions:

Symfony is slower than a turtle out of Arkansas towing a sailboat up a mountatin. It includes a bajillion files to do anything. I also hate it bitterly and will smite it down in my wrath.....maybe not, but I really don't like it.

CakePHP seems OK, but documentation is lacking. The logo is cool, but doesn't make up for the lack of performance. I don't plan on revisiting it.

CodeIgniter seems like a really cool framework, but I don't have the time or desire to figure out how to use it without proper documentation. It's performance is up there, but I didn't get a DB access benchmark. If they update the tutorials and docs I might come back to it.

Yii, pretty cool. Not blazing fast, but it's still quick. I like the code generator and it's pretty lightweight. It suffered the most from adding database access, dropping 63% in performance. The yii homepage shows a benchmark where it is faster than any other framework. This is obviously false, and considering it's lack of performance with database access, irrelevant.

DooPHP is extremely fast. They aren't kidding when they say it might be the "fastest PHP framework available." Adding database access only slowed it by 14%, very impressive. Documentation isn't great, but not bad either. Will probably revisit this one.

ZombiePHP stood up pretty well to it's competition. It came in second for speed, only behind DooPHP.  It's still young, almost ready for public announcement, so currently there is zero documentation. Considering my frustration with these other frameworks lack of good documentation, I better put some time into that.

If you have some more info on any of these frameworks, would like to contribute to ZombiePHP, or want to see another benchmark up here, let me know.

Until next time,
Adam