PHP garbage
Monday, March 24th, 2008I’ve been having a hell of a time with a data load script I’m writing at work. It’s something of the form:
$data_rows = get_a_bunch_of_data();
foreach ($data_rows as $data_row) {
$data_row = do_something1($data_row);
$data_row = do_something2($data_row);
$data_row = do_something3($data_row);
$data_row = do_something4($data_row);
}
and it consumes memory like cerebral plaque. PHP has some garbage collection mechanism through which it releases memory when a script is done running. We looked for some way to trigger this behavior at the end of each foreach iteration, but no dice. After hours of trying things and getting nowhere (it takes about 40 minutes just to see if there is improvement), we decided to make the innards of the loop it’s own function on the theory that garbage collection might be triggered when a function exits:
$data_rows = get_a_bunch_of_data();
foreach ($data_rows as $data_row) {
$data_row = do_it_all(data_row);
}
function do_it_all(
$data_row = do_something1($data_row);
$data_row = do_something2($data_row);
$data_row = do_something3($data_row);
$data_row = do_something4($data_row);
return $data_row;
}
and it worked!