search

lbryio / lbry-sdk

The LBRY SDK for building decentralized, censorship resistant, monetized, digital content apps.
https://lbry.com
MIT License
7.19k stars 484 forks source link

Bump to Python 3.9 attempt 2. #3623

Closed moodyjon closed 1 year ago

moodyjon commented 2 years ago

Based on work by @eukreign in branch py39.

I am still not sure what issues in the tests are truly due to Python upgrade vs. pre-existing timing issues. I have some hacks in there adding asyncio.sleep().

Other risky areas are related to Python 3.8 changing the base class of asyncio.CancelledError from Exception to BaseException. I fixed some places that assumed the old base class, but there may be more. See 8bb780538817956c32d7f762b8b33ab62d06c1f1 and 528a37a2b1beb2f6fdf1ad88dd93a075485fe560

I am excited about upgrading to 3.9 because it lets me run more things natively on MacOS arm64.

Change a36922855046ba0e23f47038f55b8674e1d76bd9 should be temporary, and lets me repeat tests manually to assess reliability.

moodyjon commented 2 years ago

Putting this in draft status, pending fix for some (probably) unrelated test failures:

1) (pending fixes) https://github.com/lbryio/lbry-sdk/pull/3624 2) (hub? fixes needed) https://github.com/lbryio/hub/issues/55

eukreign commented 2 years ago

@moodyjon I'm really excited about your work on this! Really looking forward to be able to upgrade to 3.9. I will review all of this as soon as you get all of the tests passing and give me a heads up.

coveralls commented 2 years ago

Coverage Status

Coverage increased (+0.4%) to 58.134% when pulling fcffe1b610a476a0e9ae63c6dc18786467814a4f on moodyjon:py39_2 into 01cd95fe460094b7a9e29ecfd61864e705b2e2c1 on lbryio:master.

moodyjon commented 1 year ago

I have synced up to head-of-line, and am running tests to asses where the next pain point is in the tests. Most of this is going on in moodyjon/lbry-sdk actions using the manual trigger.

As test_setting_stream_fields against (go) herald has been disabled, that is no longer a deterministic failure.

moodyjon commented 1 year ago

Out of 20 runs, I got 5 failures. test_connection_drop_still_receives_events_after_reconnected was seen 2 times, the others 1 time.

py39_2 ci failures.txt

test_file_save_stop_before_finished_streaming_only_wait_for_start is one I had been modifying heavily, though I don't really know what to make of the way it failed. A bunch of debugging info was dumped out and coverage run .... integration.datanetwork exited abruptly with SIGSEGV during the test.

180
ERROR: InvocationError for command /home/runner/work/lbry-sdk/lbry-sdk/.tox/datanetwork/bin/coverage run -p --source=/home/runner/work/lbry-sdk/lbry-sdk/.tox/datanetwork/lib/python3.9/site-packages/lbry -m unittest discover -vv integration.datanetwork (exited with code -11 (SIGSEGV)) (exited with code -11)

I think I've seen the SIGSEGV before in other workspaces. Any insight into whether Python code can do this? or could it be the coverage tool combined with some feature of the code?

moodyjon commented 1 year ago

Closing in favor of https://github.com/lbryio/lbry-sdk/pull/3711