search

faust-streaming / mode

Python AsyncIO Services
https://faust-streaming.github.io/mode/
Other
43 stars 16 forks source link

mode not ready for 3.11 #19

Closed AntonOfTheWoods closed 1 year ago

AntonOfTheWoods commented 1 year ago

In the spirit of https://github.com/faust-streaming/mode/issues/9

The issues were highlighted in the above issue but mitigated not fixed. The error below is because of asyncio.wait, which has had passing coroutines deprecated since 3.8 and now removed in 3.11.

As an aside, it looks like aiokafka also has plenty of these and hasn't released in over a year (with a few blockers apparently), so it looks like a ton of work to get things working with 3.11.

Checklist

Steps to reproduce

Try to run with python 3.11

Expected behavior

Run as with <3.11

Actual behavior

Error

Full traceback

faustworker_1  |     [2022-10-25 09:01:53,673] [10] [INFO] [^---AIOKafkaConsumerThread]: Stopping... 
faustworker_1  |       ^^^^^^^^^^^^^^^                                                                                              
faustworker_1  |   File "/usr/local/lib/python3.11/site-packages/mode/threads.py", line 253, in _serve
faustworker_1  |     await self.wait_until_stopped()                                                                                
faustworker_1  |   File "/usr/local/lib/python3.11/site-packages/mode/services.py", line 1005, in wait_until_stopped
faustworker_1  |     await self.wait()                                                                                              
faustworker_1  |   File "/usr/local/lib/python3.11/site-packages/mode/services.py", line 722, in wait
faustworker_1  |     await self._wait_stopped(timeout=timeout)                                                                      
faustworker_1  |   File "/usr/local/lib/python3.11/site-packages/mode/services.py", line 794, in _wait_stopped
faustworker_1  |     done, pending = await asyncio.wait(                                                                            
faustworker_1  |                     ^^^^^^^^^^^^^^^^^^^                                                                            
faustworker_1  |   File "/usr/local/lib/python3.11/asyncio/tasks.py", line 415, in wait
faustworker_1  |     raise TypeError("Passing coroutines is forbidden, use tasks explicitly.")
faustworker_1  | TypeError: Passing coroutines is forbidden, use tasks explicitly.

Versions

wbarnha commented 1 year ago

We're having related issues porting this project to support 3.11. I initially opened #20 to tackle this problem but it seems we're going to need a major overhaul. There's another fork called mode-ng, and we've brought in its maintainer to assist us in supporting this project.

Regarding aiokafka, it seems a great deal of effort will be required to get it ready for 3.11.

wbarnha commented 1 year ago

Opened up a PR for aiokafka in https://github.com/aio-libs/aiokafka/pull/863, support should come soon hopefully.

AntonOfTheWoods commented 1 year ago

@wbarnha , that PR was merged but it looks like it just updated some build settings in preparation for something. Actually, it looks like it is now building with 3.11, so if there is still a lot of work to do, it's not clear at all how it is passing the tests... Or am I missing something?

wbarnha commented 1 year ago

I figure if it's passing builds and tests, aiokafka is fine. Now the main bottleneck in our efforts is updating mode-streaming for 3.11.

lqhuang commented 1 year ago

I will involve this ASAP. Sorry for busy work recently 😢