Non time-based loop? - Githubissues

Hi,

congrats on php-daemon! I've just started looking into the framework and find it quite cool and complete! Especially good the cluster-based functionality, very useful. A couple of questions/comments maybe you can help steer me:

my first daemon needs to poll a particular webservice queue which wasnt developed by us. Nonetheless, the webserver is "blocking", and when it reaches the last even it will "wait" 10 seconds... basically, I need a non-time-looped daemon. Would this be something possible to implement? Basically, whenever it reaches the end, start again... I could just put 1 second interval, but that would flood my e-mail box, and potentially give other complications? (Is there somewhere to disable these catches? would this be the best way?)
on another note, do you see any implications in porting the memcache portion of your code to use Redis instead? We have changed from memcache to redis for most of our use, and were thikning of disabling the memcache cluster, but we could always keep it for this daemon control (although i'd prefer to just port a lock plugin for redis).

Anyways, great work. I think this will be very useful in several jobs, and when combined with a message queue such as php-resque or whichever one anyone prefers, gives us a complete toolbox for work...

Hi,

Glad you found it useful!

First, about Redis... I think you should be able to create a Redis lock adapter class very easily using the memcache one as a template. And you won't need to worry about replicating what's in Memcache.php -- that is there to work around bugs in memcache that I've never experienced in redis anyway (auto-retry). Sure, the auto namespacing and stuff would be useful so if you wanted to implement that, great, but just to use redis as a lock provider, just create a redis.php class in the /Lock directory. If you submit that in a pull request i'll merge it in and help maintain it.

Now, about the non time-base thing. I think the easiest way to do this is going to be to create your own very basic fork of the code base. Then, in the Core_Daemon class, change the timer() method from private to protected.

Then, in your daemon class, use your own timer() method. Make it so it keeps a count internally, and when it gets to 10, sleep 10 seconds. That is -- make it behave in whatever way the webservice forces you to. I won't want to merge that into trunk (i want to keep the method private) but this is what GitHub is great for. You can make a simple fork and anybody that needs non time-based daemon can clone from your repo!

Thanks for the compliments, let me know if you run into anything strange! Shane

On Mon, Dec 19, 2011 at 8:47 AM, RVN-BR < reply@reply.github.com

wrote:

Hi,

congrats on php-daemon! I've just started looking into the framework and find it quite cool and complete! Especially good the cluster-based functionality, very useful. A couple of questions/comments maybe you can help steer me:

my first daemon needs to poll a particular webservice queue which wasnt developed by us. Nonetheless, the webserver is "blocking", and when it reaches the last even it will "wait" 10 seconds... basically, I need a non-time-looped daemon. Would this be something possible to implement? Basically, whenever it reaches the end, start again... I could just put 1 second interval, but that would flood my e-mail box, and potentially give other complications? (Is there somewhere to disable these catches? would this be the best way?)

on another note, do you see any implications in porting the memcache portion of your code to use Redis instead? We have changed from memcache to redis for most of our use, and were thikning of disabling the memcache cluster, but we could always keep it for this daemon control (although i'd prefer to just port a lock plugin for redis).

Anyways, great work. I think this will be very useful in several jobs, and when combined with a message queue such as php-resque or whichever one anyone prefers, gives us a complete toolbox for work...

Reply to this email directly or view it on GitHub: https://github.com/shaneharter/PHP-Daemon/issues/5

Hi Shane,

Thanks for your comments!

I will (eventually) try and get around to the Redis lock, and post it here for others... For now I'll try file based locks during dev, and if we need to push it into trials before i get time, use memcache a bit, and then hopefully Redis by the beginning of 2012....

About your idea, I dont really follow... In our soapcall, the soapcall is basically "blocked", so what would I be doing exactly in my timer code?

One thing that came to mind would be since I saw somewhere you mentioned changing loop-interval in execution... can that happen in the middle of a loop? For example, I'd set the loop time for 25 seconds (so I still get alerted if something goes very out of control), however when the webservice finishes its thing (usually after 10.5 - 11 seconds) i'd "revise" the loop for X seconds (tapping into the execution timer)...

a few doubts:

what variable stores the current function running time? So I could use that to set the current loop?
I'd also need to set the interval back to 25 at the very beginning of the execute, so I dont have an issue if the next run takes a bit more...
would this sort of approach even work? does setting the interval "in the middle of execution" work?

I dont really want to fork any processes, unless maybe I fork, and then only fork another one if I got a message from the initial one that its "done"? with this I could do some sort of count of how many loops it takes for it to receive the "fork" and even crap out if it takes too many loops? Is there any way of forcing a quit on a forked function? Any way to check if a forked function is finished? (iirc there is an example for this)...

anyways, would any of these approaches work and be efficient enough in your opinion? At this moment i dont feel confident enough to start hacking through your core code, I'm afraid i may undo something important and have it bight me in the butt later on... one of the reasons i chose phpdaemon was to start with something somewhat stable as a daemonizer "framework".

thanks for your help thus far!

Sorry to confuse -- first, when I said "fork" i meant "fork the project on github" not "fork a process."

You CAN change loop intervals during execution by changing the loop_duration property. But I think that would be a hack for you.

I think you should look at the timer() method. It's not too hard to follow. The daemon runs on a main while() loop: At the top of the iteration, it calls timer(true) and at the bottom it calls timer().

You should change the timer() function for your own use. Otherwise you're going to have constant issues trying to make your problem work on a timer.

On Mon, Dec 19, 2011 at 10:50 AM, RVN-BR < reply@reply.github.com

wrote:

Hi Shane,

Thanks for your comments!

I will (eventually) try and get around to the Redis lock, and post it here for others... For now I'll try file based locks during dev, and if we need to push it into trials before i get time, use memcache a bit, and then hopefully Redis by the beginning of 2012....

About your idea, I dont really follow... In our soapcall, the soapcall is basically "blocked", so what would I be doing exactly in my timer code?

One thing that came to mind would be since I saw somewhere you mentioned changing loop-interval in execution... can that happen in the middle of a loop? For example, I'd set the loop time for 25 seconds (so I still get alerted if something goes very out of control), however when the webservice finishes its thing (usually after 10.5 - 11 seconds) i'd "revise" the loop for X seconds (tapping into the execution timer)...

a few doubts:

what variable stores the current function running time? So I could use that to set the current loop?

I'd also need to set the interval back to 25 at the very beginning of the execute, so I dont have an issue if the next run takes a bit more...

would this sort of approach even work? does setting the interval "in the middle of execution" work?

I dont really want to fork any processes, unless maybe I fork, and then only fork another one if I got a message from the initial one that its "done"? with this I could do some sort of count of how many loops it takes for it to receive the "fork" and even crap out if it takes too many loops? Is there any way of forcing a quit on a forked function? Any way to check if a forked function is finished? (iirc there is an example for this)...

anyways, would any of these approaches work and be efficient enough in your opinion? At this moment i dont feel confident enough to start hacking through your core code, I'm afraid i may undo something important and have it bight me in the butt later on... one of the reasons i chose phpdaemon was to start with something somewhat stable as a daemonizer "framework".

thanks for your help thus far!

Reply to this email directly or view it on GitHub: https://github.com/shaneharter/PHP-Daemon/issues/5#issuecomment-3207552

Thanks!

Yeah, changing intervals works pretty well! I was quite amazed :)

(I was actually trying it all out before you replied)

I changed the timer method and am including a "custom" one for this particular daemon... I'll make it always return straight away (leaving your suggested safety usleep), but i'll include a "safety" log if it runs for over X... so instead of a interval loop i'll just have the timer check against a safety timeout for warning... I think that will be good!

Thanks for the help!

I can post my revised timer function here if it is worth it for anyone, but I've never used github (:ashamed:) and its basically just a simple change as you described... anyways, thanks for the help so far!

PS: congratulations on your commenting of code! impressive and makes me jealous!

Wanted to close this out: This feature is now fully supported and trivially easy starting with the new 2.0 release available now in master.

Updating apps written for 1.1.1 to 2.0 will be fairly straight forward. There are name changes to some methods (load_plugins is now setup_plugins, fork is now task, etc), and the interfaces have been renamed (but remain the same) in case you're implementing custom plugins.

There's also far far more documentation in the Wiki now.

Shane

shaneharter / PHP-Daemon

Non time-based loop? #5