rolling curl outer loop can terminate prematurely leaving many requests unprocessed

[GoogleCodeExporter]

The outer do-while loop of the rolling_curl function terminates prematurely if 
the $running flag is false, even if new requests have since been added to 
$master and not started yet.

The conditional in RollingCurl.php:325 needs to check whether there are still 
requests outstanding.

Original issue reported on code.google.com by mcamm...@gmail.com on 3 Apr 2013 at 6:00

[GoogleCodeExporter]

The following patch will address this issue as well as cleanup memory in the 
case where the callback is not defined or not callable.

--- RollingCurl.php : Working Base, Revision 3686   2013-07-01 12:09:46.000000000 
-0500
+++ RollingCurl.php : Working Copy  2013-07-03 09:33:43.000000000 -0500
@@ -290,30 +290,34 @@
                 // get the info and content returned on the request
                 $info = curl_getinfo($done['handle']);
                 $output = curl_multi_getcontent($done['handle']);

                 // send the return values to the callback function.
                 $callback = $this->callback;
+                $key = (string) $done['handle'];
                 if (is_callable($callback)) {
-                    $key = (string) $done['handle'];
                     $request = $this->requests[$this->requestMap[$key]];
-                    unset($this->requestMap[$key]);
                     call_user_func($callback, $output, $info, $request);
                 }
+                unset($this->requestMap[$key]);

                 // start a new request (it's important to do this before removing the old one)
                 if ($i < sizeof($this->requests) && isset($this->requests[$i]) && $i < count($this->requests)) {
                     $ch = curl_init();
                     $options = $this->get_options($this->requests[$i]);
                     curl_setopt_array($ch, $options);
                     curl_multi_add_handle($master, $ch);

                     // Add to our request Maps
                     $key = (string) $ch;
                     $this->requestMap[$key] = $i;
                     $i++;
+                    // Reset the running flag so even if all requests in the
+                    // window size completed, we continue to process the
+                    // newly added requests.
+                    $running = true;
                 }

                 // remove the curl handle that just completed
                 curl_multi_remove_handle($master, $done['handle']);

             }

Original comment by k...@korl.com on 3 Jul 2013 at 2:41

[GoogleCodeExporter]

This patch will stop issue 25 from being fixed, perhaps branching logic to 
allow achieving both?

Original comment by lewis.cowles.1986@gmail.com on 22 Sep 2013 at 9:32