examples needed

[brikou]

As this library is brand new... capabilities need to be discovered, I wish some basic simple example where provided, through some gist for example.

NB: What I'm trying to achieve is parallel computation (~batch processing) with N job running at the same time... when on is finished append a new one until all job are done...

Cheer

[kriswallsmith]

This would constitute a new batch strategy, one that I would like to support out of the box but haven't written yet. The createBatches() method would return a special iterator whose current() method waits until one of the first four batches exits before returning the fifth, for example.

[seyfer]

Is there timeout for Chunk strategy? If one fork work in infinity loop, will it die after some timeout?

I made process() with new ChunkStrategy(10) but it still generate only 3 forks like default value. How i can make 10 or more forks ?

[kriswallsmith]

@seyfer how many data are you passing into the batch job? If you configure 10 chunks but only pass 3 data, there will only be 3 forks.

[seyfer]

I have high load system and always have requests in database. I take from db 10 tasks and give it to Spork. All this happen in daemon. I made debug, and give 8 tasks. Then in wait() method of manager i print count($this->forks) and it prints 8.

But on server, or on local machine i see only 3-4 forks by "ps aux | grep minion".

Previosly i use AzaThread (https://github.com/amal/AzaThread) and when i configure 10 forks in Aza, i see 10 in system, when 8 - i see 8. And Aza faster process my tasks. Was forced to withdraw from this library due to a bug in libevent.

I think Spork don't create 10 forks. If you can - please test.

My example:

$requests_iterator = new ArrayIterator($requests_to_process);

$fork = $this->sporkManager->process($requests_iterator, function($request) {

                        try
                        {
                            $this->model->reconnect();

                            $asbresult = $this->asbProcessor->run($request);

                            if ($asbresult)
                            {
                                $id = $this->asbProcessor->getId();
                                $this->asbProcessor->update($id, $asbresult);
                                return TRUE;
                            }

                            return FALSE;
                        }
                        catch (\Exception $exc)
                        {
                            return $exc->getMessage();
                        }

                    }, new ChunkStrategy($this->maxThreads));

            $this->sporkManager->wait();

$requests_to_process is array with 8 or 10 tasks.

[seyfer]

$this->maxThreads = 10 in example

[seyfer]

watch --interval=1 'ps aux | grep minion'

I have monitoring and see only 3 forks maximum, but configured 10.

[seyfer]

Made better monitoring. There no erros, all works fine.

I think issue may be closed, becouse i write example for issue author. :) But there still need more examples for Strategies.

Thank for support!