Since PHP does not offer native threads, we have to get creative to do parallel processing. I will introduce 3 fundamentally different concepts to emulate multithreading as good as possible.<\/p>\n
If you have some basic linux knowledge, you will know that a background process can be started by adding ampersand to the systemcall (in Windows, it’s the start<\/a>-command)<\/p>\n The PHP script is running silently in the background. What is being printed to the shell (3229<\/i>) is the process id, so that we are able to kill the process using<\/p>\n A problem with this approach is, that any output of the script is lost, so we have to redirect the output stream to a file, just like this:<\/p>\n The purpose of the scary 2>&1<\/i> is to redirect stderr to stdout<\/b>, so when your script produces any kind of php error, it will also get caught by the output-file. Putting everything together, we get<\/p>\n Looks confusing, right? We’ve added echo $! >> %s<\/i> to the command, so that the process id of the background script gets written to a file. This proves to be useful to keep track of all running processes.<\/p>\n If you want to kill all php-processes, the following command will do:<\/p>\n Needless to say that when you add the php shebang<\/a> #!\/usr\/bin\/php<\/b> to the top of your script and make it executable using chmod +x script.php<\/b>, the system command needs to be modified to .\/script.php<\/b> instead of php script.php<\/b>.<\/p>\n To check if a process is still running, you might use some variation of the ps command<\/a> as done here (stolen from Steffen<\/a>):<\/p>\n Using the pnctl<\/a>-functions of php, you get the ability to fork<\/a> a process (pcntl_fork<\/a>, not availible on Windows). Before you get too excited, read to following quote from a comment written on php.net that exactly reflects my experience with forking in php:<\/p>\n \nYou should be _very_ careful with using fork in scripts beyond academic examples, or rather just avoid it alltogether, unless you are very aware of it’s limitations. You have been warned! Look at the following example:<\/p>\n Output:<\/b><\/p>\n As you can see, we get 2 ^ fork count<\/i><\/sup> processes. Somewhere in the middle of the output, the original script is finished but some forks are still running. It’s even possible to communicate<\/a> with processes that you forked. Forking is a very interesting area of computer science, nevertheless i don’t recommend using fork in real-world php applications.<\/p>\n The last way to process multiple scripts in parallel is to abuse the webserver and curl<\/a>. With curl, we are able to execute multiple requests in parallel (inspired by Gonzalo Ayuso<\/a>).<\/p>\n Here, we call the script calc.php<\/i> 15 times. The content of calc.php is:<\/p>\n The output<\/b> is as follows:<\/p>\n Interesting to see, that we see the same process id a few times. Keep in mind, that you trigger an http-request, so you are losing performance because a webserver has to do some work. Furthermore, the called script will be working with the ordinary php.ini<\/i>, and not<\/b> php-cli.ini<\/i>.<\/p>\n What would you take away from this post, when you didn’t know which parallel processing method is the fastest? I’ve written a little benchmark script using the 3 methods described above, did 3 runs and calculated the average. Basically, this is my benchmark scipt calc.php<\/b>:<\/p>\n My system:<\/p>\n I’m fully aware that this benchmark is in no way representative, because writing the result files to harddisk might influence other processes, that are still running and my time comparison may also be slightly inaccurate. Ah, before you ask: I haven’t used set_time_limit<\/a> because it sucks. So bring on the results!<\/b><\/p>\n As you can see, the more parallel processes, the more iterations in total. I haven’t tested 64 processes and more because my system almost froze (memory usage and cpu utilization). Feel free to interpret the results in any way you want but in the end, it boils down to the exec<\/i> – method because fork<\/i> is evil and curl<\/i> is not a serious alternative.<\/p>\n Finally, if you want to do some testing on your own, here is my benchmark file. Place it in the same folder with the calc.php<\/b> from above, give the file execute rights and create a folder results<\/i>. The file is invoked by using .\/bench.php method processcount<\/i>, so possible calls are<\/p>\n The file itself:<\/b><\/p>\n Since PHP does not offer native threads, we have to get creative to do parallel processing. I will introduce 3 fundamentally different concepts to emulate multithreading as good as possible. Using systemcalls If you have some basic linux knowledge, you … Weiterlesen \r\ndav@david:\/var\/www$ php index.php &\r\n[1] 3229\r\n<\/pre>\n
\r\nkill 3229\r\n<\/pre>\n
\r\nphp index.php > output.txt 2>&1 &\r\n<\/pre>\n
\r\n$cmd = "php script.php";\r\n\r\n$outputfile = "\/var\/www\/files\/out.";\r\n$pidfile = "\/var\/www\/files\/pid.";\r\n\r\nfor ($i = 0; $i < $process_count; $i++)\r\n exec(sprintf("%s > %s 2>&1 & echo $! >> %s", $cmd, $outputfile.$i, $pidfile.$i));\r\n<\/pre>\n\r\nkillall php\r\n<\/pre>\n
\r\nfunction is_running($pid)\r\n{\r\n\t$c = "ps -A -o pid,s | grep " . escapeshellarg($pid);\r\n\texec($c, $output);\r\n\r\n\tif (count($output) && preg_match("~(\\d+)\\s+(\\w+)$~", trim($output[0]), $m))\r\n\t{\r\n\t\t$status = trim($m[2]);\r\n\t\tif (in_array($status, array("D","R","S")))\r\n\t\t{\r\n\t\t\treturn true;\r\n\t\t}\r\n\t}\r\n\t\r\n\treturn false;\r\n}\r\n<\/pre>\n<\/p>\nUsing fork()<\/h2>\n
\nThe problem is that it just forks the whole php process, including not only the state of the script, but also the internal state of any extensions loaded.
\nThis means that all memory is copied, but all file descriptors are shared among the parent and child processes.
\nAnd that can cause major havoc if some extension internally maintains file descriptors.
\nThe primary example is ofcourse mysql, but this could be any extensions that maintains open files or network sockets.\n<\/p><\/blockquote>\n\r\nfor ($i = 0; $i < 4; $i++)\r\n{\r\n pcntl_fork();\r\n}\r\n\r\necho "hi there! pid: " . getmypid() . "\\n";\r\n<\/pre>\n\r\ndav@david:\/var\/www$ php script.php\r\nhi there! pid: 3534\r\nhi there! pid: 3536\r\nhi there! pid: 3538\r\nhi there! pid: 3539\r\nhi there! pid: 3540\r\nhi there! pid: 3541\r\nhi there! pid: 3542\r\nhi there! pid: 3537\r\nhi there! pid: 3543\r\ndav@david:\/var\/www$ \r\nhi there! pid: 3544\r\nhi there! pid: 3545\r\nhi there! pid: 3546\r\nhi there! pid: 3548\r\nhi there! pid: 3547\r\nhi there! pid: 3549\r\nhi there! pid: 3550\r\n<\/pre>\n
Using curl<\/h2>\n
\r\n$url = "http:\/\/localhost\/calc.php";\r\n$mh = curl_multi_init();\r\n$handles = array();\r\n$process_count = 15;\r\n\r\nwhile ($process_count--)\r\n{\r\n $ch = curl_init();\r\n curl_setopt($ch, CURLOPT_URL, $url);\r\n curl_setopt($ch, CURLOPT_HEADER, 0);\r\n curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);\r\n curl_setopt($ch, CURLOPT_TIMEOUT, 30);\r\n curl_multi_add_handle($mh, $ch);\r\n $handles[] = $ch;\r\n}\r\n\r\n$running=null;\r\n\r\ndo \r\n{\r\n curl_multi_exec($mh, $running);\r\n} \r\nwhile ($running > 0);\r\n\r\nfor($i = 0; $i < count($handles); $i++) \r\n{\r\n $out = curl_multi_getcontent($handles[$i]);\r\n print $out . "\\r\\n";\r\n curl_multi_remove_handle($mh, $handles[$i]);\r\n}\r\n\r\ncurl_multi_close($mh);\r\n<\/pre>\n\r\n<?php\r\necho "my pid: " . getmypid();\r\n?>\r\n<\/pre>\n
\r\ndav@david:\/var\/www$ php script.php\r\nmy pid: 1401\r\nmy pid: 1399\r\nmy pid: 1399\r\nmy pid: 1403\r\nmy pid: 1403\r\nmy pid: 1398\r\nmy pid: 1398\r\nmy pid: 1402\r\nmy pid: 3767\r\nmy pid: 3768\r\nmy pid: 3769\r\nmy pid: 3772\r\nmy pid: 3771\r\nmy pid: 3773\r\nmy pid: 3770\r\n<\/pre>\n
What about the speed? Benchmarks!<\/h2>\n
\r\n$starttime = time();\r\n$duration = 10;\r\n\r\n$filename = "\/var\/www\/results\/" . getmypid() . ".out";\r\n\r\n$loops = 0;\r\n\r\nwhile (true)\r\n{\r\n for ($i = 0; $i < 10000; $i++)\r\n {\r\n sqrt($i);\r\n }\r\n \r\n $loops++;\r\n \r\n if ($starttime + $duration <= time())\r\n break;\r\n}\r\n\r\nfile_put_contents($filename, $loops);\r\n<\/pre>\n\r\nUbuntu 10.10 (Kernel 2.6.35-28)\r\n4 gig Ram\r\nIntel Core 2 Duo T7500 (2 * 2.2GHz)\r\n<\/pre>\n
\r\nMethod Proc. Iterations\r\n\r\nexec 1 2183 \r\nexec 2 3953 \r\nexec 4 4283 \r\nexec 8 4378\r\nexec 16 4586\r\nexec 32 4868\r\n\r\ncurl 1 2203\r\ncurl 2 2843\r\ncurl 4 3029\r\ncurl 8 3556\r\ncurl 16 3986\r\ncurl 32 4373\r\n\r\nfork 1 2274\r\nfork 2 4299\r\nfork 4 4245\r\nfork 8 4309\r\nfork 16 4177\r\nfork 32 4577\r\n<\/pre>\n
\r\n.\/bench.php exec 16\r\n.\/bench.php curl 8\r\n.\/bench.php fork 32\r\n.\/bench.php -> no parameter to display results\r\n<\/pre>\n
\r\n#!\/usr\/bin\/php\r\n<?php\r\n$mode = isset($argv[1]) ? $argv[1] : "results";\r\n$process_count = isset($argv[2]) ? $argv[2] : 1;\r\n\r\n\/\/cleanup\r\nif ($mode != "results" && count(glob("\/var\/www\/results\/*")))\r\n{\r\n exec("rm \/var\/www\/results\/*");\r\n}\r\n\r\nif ($mode == "exec")\r\n{\r\n $cmd = "php calc.php";\r\n\r\n $outputfile = "\/var\/www\/results\/out.";\r\n $pidfile = "\/var\/www\/results\/pid.";\r\n\r\n for ($i = 0; $i < $process_count; $i++)\r\n exec(sprintf("%s > %s 2>&1 & echo $! >> %s", $cmd, $outputfile.$i, $pidfile.$i));\r\n}\r\nelseif ($mode == "curl")\r\n{\r\n $url = "http:\/\/localhost\/calc.php";\r\n $mh = curl_multi_init();\r\n \r\n while ($process_count--)\r\n {\r\n $ch = curl_init();\r\n curl_setopt($ch, CURLOPT_URL, $url);\r\n curl_setopt($ch, CURLOPT_HEADER, 0);\r\n curl_setopt($ch, CURLOPT_NOBODY, true);\r\n curl_setopt($ch, CURLOPT_RETURNTRANSFER, false);\r\n curl_setopt($ch, CURLOPT_TIMEOUT, 30);\r\n curl_multi_add_handle($mh, $ch);\r\n }\r\n \r\n $running=null;\r\n \r\n do \r\n {\r\n curl_multi_exec($mh, $running);\r\n } \r\n while ($running > 0);\r\n}\r\nelseif ($mode == "fork")\r\n{\r\n for ($i = 0; $i < log($process_count, 2); $i++)\r\n {\r\n pcntl_fork();\r\n }\r\n \r\n include "calc.php";\r\n}\r\nelse\r\n{\r\n $total = 0;\r\n\r\n foreach (glob("\/var\/www\/results\/*.out") as $f)\r\n {\r\n $runtime = file_get_contents($f);\r\n $total += $runtime;\r\n echo $runtime . "\\r\\n";\r\n }\r\n\r\n echo "Total: " . $total . "\\r\\n";\r\n}\r\n<\/pre><\/p>\n","protected":false},"excerpt":{"rendered":"