search

jgehrcke / gipc

gevent-cooperative child processes and inter-process communication
https://gehrcke.de/gipc
MIT License
83 stars 13 forks source link

support for gevent>=1.5.0 please #98

Closed koreno closed 3 years ago

koreno commented 4 years ago

It has been out for a couple of weeks, and we're getting:

ERROR: gipc 1.1.0 has requirement gevent<1.5,>=1.2, but you'll have gevent 1.5.0 which is incompatible.
jgehrcke commented 4 years ago

@koreno I will release a new version of gipc soon. Thanks for raising this issue. The preparational work was all done, tracked in https://github.com/jgehrcke/gipc/issues/86.

ifiddes commented 4 years ago

Is this going to be released as a new version soon?

jgehrcke commented 4 years ago

Thanks for asking @ifiddes -- I'll look into this soon again. Most of the work was done in https://github.com/jgehrcke/gipc/pull/91 and I am not quite sure anymore why I hesitated releasing. Sorry for the delay!

jgehrcke commented 4 years ago

Also see #101, about supporting gevent versions newer than 1.5.0.

koreno commented 3 years ago

@jgehrcke - any update? we're biting our fingernails!

jgehrcke commented 3 years ago

Thanks for the ping @koreno. Is gevent 1.5.0 itself a big win for you? There are new challenges with gevent 20.x compatibility, 1.5.0 support is easier to do. A gipc version that supports gevent 1.5.0 but not 20.x would be a win for you -- yes?

jgehrcke commented 3 years ago

@koreno as I am always curious about what people use gipc for: what's your use case (roughly, if you can talk about it!)? : ) (having a cool use case in mind is of course additional huge motivation for doing work!).

koreno commented 3 years ago

Sure! We're maintaining a large testing infrastructure for an enterprise storage system. We generally use a monkey-patched python-gevent environment, but some of our tests need to spawn a child process to perform the IOs, as part of some functions tests. Without gipc, these forked processes inherit various monitoring greenlets from the parent process, which is highly undesirable. It seems to inherit various file descriptors as well.

(In fact we're running into such an issue now, even with gipc, so I'm wondering if there's some rare edge-case or race-condition that allows it, or maybe this has to do with changes in python/gevent)

On Thu, Dec 10, 2020 at 4:01 PM Dr. Jan-Philip Gehrcke < notifications@github.com> wrote:

@koreno https://github.com/koreno as I am always curious about what people use gipc for: what's your use case? : ) (having a cool use case in mind is of course additional huge motivation for doing work!).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jgehrcke/gipc/issues/98#issuecomment-742538800, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABMCR7O4DFUBD46P7FVB6TSUDIEFANCNFSM4MTFAFHA .

jgehrcke commented 3 years ago

Thank you for elaborating!

There's hope by the way that we can do gevent 20.x: https://github.com/gevent/gevent/pull/1728/files

(In fact we're running into such an issue now, even with gipc, so I'm wondering if there's some rare edge-case or race-condition that allows it, or maybe this has to do with changes in python/gevent)

With gevent 1.4.x? If you can repro with a minimal working example, I'd be happy to take a look. I also understand that it's always super tough to create a minimal working example. A lot of care and testing went into past gipc releases to prevent these things from happening, but I can only ever make statements about specific combinations of versions of course (gipc, CPython/PyPy, gevent)

jgehrcke commented 3 years ago

Just released gipc 1.1.1, adding official support for gevent 1.5.0. https://pypi.org/project/gipc/1.1.1/ https://github.com/jgehrcke/gipc/blob/master/CHANGELOG.rst#version-111-jan-03-2021

Supporting gevent 20.x is a little more work.

Happy new year everyone! :balloon:

kiarn commented 3 years ago

Thanks @jgehrcke ! I will test to upgrade the requirements in Ajenti.

jgehrcke commented 3 years ago

@kiarn did things work out, did gipc 1.1.1 unblock you?

kiarn commented 3 years ago

@jgehrcke : thanks for asking, and sorry for the late answer, I'm under heavy load.

I still have a problem with the signal handler I'm not able to solve, but I think it belongs to another discussion, I will open an issue for this, even if I'm not sure it's an issue. But all the rest is working fine, many thanks for your update.

Edit : I was writing the issue, and then I understood my dummy error ... The rubber duck debugging still has pretty future.