search

collective / collective.taskqueue

Asyncore-based asynchronous task queue for Plone
https://pypi.org/project/collective.taskqueue
8 stars 7 forks source link

Add support for Python 3 #22

Closed staeff closed 3 years ago

staeff commented 5 years ago

I had a first look at the possibilities to port this addon to support Plone 5.2 with Python 3. I found the PR of https://github.com/collective/collective.taskqueue/pull/21 by @petschki and the branch 'python3_compatibility' by @cekk. So there is also some work in progress?

@datakurre stated on the merge request, that the dependency of ZServer would make the porting difficult. Can you describe on a high level, what the issue is and how a possible solution could look like?

Is someone about to put more work into it soon?

Thanks!

cekk commented 5 years ago

I only fixed imports, but didn't tested if it works because project stopped.

datakurre commented 5 years ago

Currently there is now way to support WSGI based Plone on Python 3. My own goal is to migrate ZServer to Python 3 with Twisted and upgrade all my async work to work on that. (Unless someone convinced me why eg ASGI is superior to Twisted+AsyncIO.)

I’m coming to Sorrento sprint to work on this.

staeff commented 5 years ago

Hi @datakurre, this sounds great!

MrTango commented 5 years ago

@datakurre if I understand this correctly, the only tricky thing here is the consuming part, when the worker is consuming the redis queue and has to do the task then. Also the worker needs access to the database, in most cases. Is the dependency to the ZServer important for that part or is it more for the standalone version when you don't have Redis running? I guess most people can live without this feature.

datakurre commented 5 years ago

@MrTango

Both with and without redis rely on a co-opting main loop asyncore server polling queue (local in-memory or a shared redis one) and dispatching ZPublisher worker requests from those. Integration with ZServer async loop and avoiding external service process (similarly to "zope clock server") was always the key idea of this add-on.

This add-on simply makes no sense with WSGI as such. Also I don't know if it is possible to do transaction aware task queues with Plone WSGI deployment at all (as WSGI middlewares)...

I hope to have a better answer after Sorrent sprint.

datakurre commented 5 years ago

https://community.plone.org/t/moonshot-for-migrating-zserver-to-python-3-at-plog-2019/8191/19

datakurre commented 5 years ago

As promised I've been looking into Python 3 port at Beethoven Sprint 2019.

I can now say that the Python 3 port will happen, but it will only support Plone running on my Twisted based ZServer version. I have volatile queue ported and Redis one should be just matter of time. But it is too early to say that my Python 3 ZServer will be merged back to Zope.

gforcada commented 3 years ago

More or less this can be closed? 🤔

datakurre commented 3 years ago

Yes. This is "won't fix" as such. Unortunately. I believe, I based my other branch on top of this.