Linux and PHP web application support and development (Bromsgrove, UK)
Normally, profiling a PHP application is pretty straight forward thanks to the great xDebug . Normally, we’re looking at profiling a slow page (taking 5-15 seconds to load) and improving it’s performance down to a sub-second response.
Such profiling can produce relatively large output files – e.g. in the order of a few hundred megabytes.
Unfortunately a few weeks ago, we found ourselves profiling an application that took 4-5 hours to run, and the output profile (cachegrind) file was 70-80Gb.
At this point we have a problem – as the file is too large for kCachegrind to process and our desktops will just spend hours swapping if we try and load it.
So we needed an alternative way of getting an idea of finding areas within the codebase that needed optimisation / refactoring.
Thankfully, xDebug also provide execution/function tracing. This effectively just records the order of functions running within the program, and how long they took. While the output from this was still >70Gb in size, it did lend itself to being more human readable and easier to parse / analyse.
So, we wrote some python code (GitHub) to do the below :
1. Filtered the output file using the following quick Python script (filter.py) – this obviously requires you to have some idea as to where the bottleneck is likely to be (i.e. we suspect it’s in Module/File.php).
python filter.py Module/File.php tracefile.xt > filtered_tracefile.xt
2. Merged the filter output with the source code using this –
python report.py filtered_tracefile.xt ./path/to/Module/File.php > report.html
3. Read the output HTML report, where it’ll show you a cumulative count for time taken on each call and how often the function was called — like :
99 ....
100 foreach ($x as $y) {
101 $qty = $allocationRepo->getSomethingByXAndY($y, $location);
{'....->getSomethingByXAndY': {'avg_time': 0.018434527744934828, 'count': 774700, 'total_time': 14281.22864400101} }
102 if( ! $qty ) { continue ; }
103 ....
Hopefully this will be of use to others!.
‹ Speeding up subsequent SSH connections to a host Filesystem Magic on AWS with EBS volumes and BTRFS ›
RT @PalePurpleLtd: xdebug – profiling a long running PHP app (when the cachegrind file becomes too big to handle) http://t.co/s8oEUeuHhX
RT @PalePurpleLtd: xdebug – profiling a long running PHP app (when the cachegrind file becomes too big to handle) http://t.co/s8oEUeuHhX
RT @PalePurpleLtd: xdebug – profiling a long running PHP app (when the cachegrind file becomes too big to handle) http://t.co/s8oEUeuHhX
RT @PalePurpleLtd: xdebug – profiling a long running PHP app (when the cachegrind file becomes too big to handle) http://t.co/s8oEUeuHhX
RT @PalePurpleLtd: xdebug – profiling a long running PHP app (when the cachegrind file becomes too big to handle) http://t.co/s8oEUeuHhX