codemasher / rolling-curl

Automatically exported from code.google.com/p/rolling-curl
0 stars 0 forks source link

[PATCH] Several patches #13

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
Hi,

This includes several patches I needed, which are attached.

[PATCH 1/3] Add $request as parameter to callback function:

Currently it is not possible to know, which request finished in the callback as 
the URL might not be the same due to 301 and is not unique anyway.

By providing the request to the callback this is changed and allows a very 
flexible implementation.

I also changed Request class name to be more RollingCurl specific as it is else 
cluttering the namespace.

[PATCH 2/3] Add rolling curl group:

This is an implementation using the power of OO programming by extending 
rolling curl to allow finishing groups of requests, which allows priotizing and 
also giving back feedback, when each group is finished.

The usage is really easy:

Inheriting from the base class allows processing of groups and requests 
directly from the class:

Test Class
======

class TestCurlRequest extends RollingCurlGroupRequest
{
        public $test_verbose = false;

        function process($output, $info)
        {
                echo "Processing " . $this->url . "\n";
                if ($this->test_verbose)
                        print_r($info);
                parent::process($output, $info);
        }
}

class TestCurlGroup extends RollingCurlGroup {

        function process($output, $info, $request)
        {
                echo "Group CB: Progress " . $this->name . " (" . ($this->finished_requests+1) . "/" . $this->num_requests .  ")\n";
                parent::process($output, $info, $request);
        }

        function finished()
        {
                echo "Group CB: Finished " . $group->name . "\n";
                parent::finished();
        }
}

Main function:
=========

        $group = new TestCurlGroup("High");
        $group->add(new TestCurlRequest("www.google.de"));
        $group->add(new TestCurlRequest("www.yahoo.de"));
        $group->add(new TestCurlRequest("www.newyorktimes.com"));
        $reqs[] = $group;

        $group = new TestCurlGroup("Normal");
        $group->add(new TestCurlRequest("twitter.com"));
        $group->add(new TestCurlRequest("www.bing.com"));
        $group->add(new TestCurlRequest("m.facebook.com"));
        $reqs[] = $group;

        $reqs[] = new TestCurlRequest("www.kernel.org");

        $rc = new GroupRollingCurl(); /* Note: No callback here, as its done in Request class*/

        foreach ($reqs as $req)
                $rc->add($req);

        $rc->window_size = $window_size;
        return $rc->execute();

---------

Due to the power of polymorphism, the same function (add) can be used for 
adding requests and groups of requests.

The "callback" in request and groups is:

process($output, $info)

and

process($request, $output, $info)

Also finished is available for groups.

[PATCH 3/3] Allow custom options to overwrite default ones

Issue #12 fixed in a patch. Dependent on patches before.

Hope you enjoy!

Please apply.

The patches are also available in the master branch of my fork of rolling-curl 
at github:

http://github.com/LionsAd/rolling-curl

Best Wishes,

Fabian (LionsAd)

Original issue reported on code.google.com by UFabi...@googlemail.com on 5 Aug 2010 at 7:29

Attachments:

GoogleCodeExporter commented 9 years ago
[PATCH] Add __destruct function to RollingCurlRequest.

Ooops, forgot one patch.

Here it is. Else it gives an error that the __destruct function is not defined.

Original comment by UFabi...@googlemail.com on 5 Aug 2010 at 7:47

Attachments:

GoogleCodeExporter commented 9 years ago

Original comment by alexander.makarow on 12 Sep 2010 at 8:42