search

kakserpom / phpdaemon

Asynchronous server-side framework for network applications implemented in PHP using libevent
http://daemon.io/
GNU Lesser General Public License v3.0
1.53k stars 231 forks source link

IPCManager->sendSingleCall #82

Closed dcb closed 11 years ago

dcb commented 12 years ago

The sendSingleCall is actually doing a broadcast call. In the function 'op' => 'broadcastCall', should probably be 'op' => 'singleCall'

Even with that fixed the function is not very helpful. In my test the singleCall just sent the call to the first declared worker, which was busy. When the worker became available it did process the call. If you issue multiple calls they will all be executed sequentially by the first declared worker. In my usage scenario I wanted to issue multiple singleCalls to be exectuted in parallel by the available idle workers.

Is it possible to modify the sendSingleCall function to find an idle worker instead of just sending the call to the first declared worker?

kakserpom commented 12 years ago

First problem has been fixed. Thank you for helping us making phpDaemon better.

I do not think that it shoud scan shared memory to select worker. Workers may (should) become idle and busy too fast, and shared memory segment has been implemented primarily for statistics purposes rather than selection of workers. Thereby, I suggest to implement key-based distribution (randomised). What do you think?

dcb commented 12 years ago

I see. That probably is good enough for most of the projects, but my situation is a bit different. Let me explain what I'm trying to achieve. To put it simply it is a specialized cron daemon. For that I have created one ApplicationInstance (with 1 instance limit) that executes every second and retrieves the job definitions and should launch them. This is where I got stuck and I finaly came with the idea of creating another ApplicationInstance that implements the actual jobs. I have tried to use the sendSingleCall method to launch the jobs, which take several seconds to execute and that's how I discovered the problem with the single call method. Now, if the workers are to be picked at random some jobs will still have to wait if the worker is busy and I would like to avoid that situation. Ideally the function should launch new workers as need because the highest priority is to execute each job at the scheduled time. Also I expect the application to have a high number of jobs to run each second.

Maybe I'm trying to solve this the wrong way, any advice would be greatly apreciated. But if using sendSingleCall is the way to go I need to change it to pick an idle worker or spawn a new one if all are busy. So any pointers on how to do that would be great. Of course I will share back the modified code.

Thanks

kakserpom commented 12 years ago

I'm going to implement both mechanisms. How much time does one task take (avg) ?

2012/11/24 Bogdan Dumitru notifications@github.com

I see. That probably is good enough for most of the projects, but my situation is a bit different. Let me explain what I'm trying to achieve. To put it simply it is a specialized cron daemon. For that I have created one ApplicationInstance (with 1 instance limit) that executes every second and retrieves the job definitions and should launch them. This is where I got stuck and I finaly came with the idea of creating another ApplicationInstance that implements the actual jobs. I have tried to use the sendSingleCall method to launch the jobs, which take several seconds to execute and that's how I discovered the problem with the single call method. Now, if the workers are to be picked at random some jobs will still have to wait if the worker is busy and I would like to avoid that situation. Ideally the function should launch new workers as need because the highest priority is to execute each job at the scheduled time. Also I expect the application to have a high number of jobs to run each second.

Maybe I'm trying to solve this the wrong way, any advice would be greatly apreciated. But if using sendSingleCall is the way to go I need to change it to pick an idle worker or spawn a new one if all are busy. So any pointers on how to do that would be great. Of course I will share back the modified code.

Thanks

— Reply to this email directly or view it on GitHubhttps://github.com/kakserpom/phpdaemon/issues/82#issuecomment-10668588.

dcb commented 12 years ago

Well, it can be anything between 20ms to 1sec, and in special situations up to 30secs. They are network related tasks.

kakserpom commented 12 years ago

Would you please describe the task? Is it calculating something? Worker cannot be "busy" of network-related tasks longer than ~1 ms at worst case.

2012/11/24 Bogdan Dumitru notifications@github.com

Well, it can be anything between 20ms to 1sec, and in special situations up to 30secs. They are network related tasks.

— Reply to this email directly or view it on GitHubhttps://github.com/kakserpom/phpdaemon/issues/82#issuecomment-10669152.

kakserpom commented 12 years ago

Also, I offer you to discuss it more suitable way if you please. You may find me in gtalk (kak.serpom.po.yaitsam@gmail.com) or irc.freenode.net#phpdaemon .

2012/11/24 Vasily Zorin kak.serpom.po.yaitsam@gmail.com

Would you please describe the task? Is it calculating something? Worker cannot be "busy" of network-related tasks longer than ~1 ms at worst case.

2012/11/24 Bogdan Dumitru notifications@github.com

Well, it can be anything between 20ms to 1sec, and in special situations up to 30secs. They are network related tasks.

— Reply to this email directly or view it on GitHubhttps://github.com/kakserpom/phpdaemon/issues/82#issuecomment-10669152.

dcb commented 12 years ago

I've added you on gtalk (dcbinsomniac@gmail.com). In the tasks I'm using curl to get detailed information about the load times (connection time, dns lookup time, wait time etc) of different URLs, and also analize the returned content.

kakserpom commented 11 years ago

Fixed (a long time ago).