search

Bitmessage / PyBitmessage

Reference client for Bitmessage: a P2P encrypted decentralised communication protocol:
https://bitmessage.org/wiki/Main_Page
Other
2.81k stars 578 forks source link

python 3 port plan #1712

Open PeterSurda opened 3 years ago

PeterSurda commented 3 years ago

Using the principles of continuous delivery, port to python 3 should progress as following:

g1itch commented 3 years ago

the first PR will add allow travis to run python3 while successfully doing nothing. The pybitmessage binary will check if it's running python 3 and if yes, exit with return code 0. Unit tests will be decorated to skip all tests when using python 3. This should allow travis to succeed in python 3.

In my ci branch I have python "3.6" in allow_failures and build itself succeeds if python3 part fails. But we all see the failures.

g1itch commented 3 years ago

Not sure what to do if requirements.txt diverge between python 2 and python 3.

Unlikely

PeterSurda commented 3 years ago

In my ci branch I have python "3.6" in allow_failures and build itself succeeds if python3 part fails. But we all see the failures.

This is interesting, unfortunately it doesn't help me here. I need something that allows a gradual compatibility with python 3 with as little human intervention as possible. If all the errors are just ignored, then a PR reviewer needs more time to figure out what's working and what not by looking at the detailed output. The creator of the PR also needs to spend more time evaluating whether he still needs to continue working or wait for a review. Whereas in my proposal, a failure means the PR is rejected and the PR creator has immediate feedback that the isn't finished, and a success means it's ready for a review. This way, PR after PR, python 3 support will be implemented.

I already tried several approaches for getting python 3 support intergrated but none produced a PR that can be merged. One of the developers produced a branch which has most of the functionality as well as most unit tests working (or at least it looks like it's working), but they don't have the proper quality and aren't chunked nicely for PRs. My proposal allows this to happen (refactor or write from scratch, depending on which makes more sense).

Then there are other minor issues, like travis2bash not understanding the full travis syntax, the non-developers not having proper feedback. In my proposal, pybitmessage could simply print "python3 version not ready yet, use python2" when run under python3.

ghost commented 3 years ago

@g1itch I don't have python2! It is not supported in any new OS. Not even in Debian 11!

I need a Debian 10 virtual machine with Gnome Boxes to run PyBitmessage. Unfortunately.

g1itch commented 3 years ago

@g1itch I don't have python2! It is not supported in any new OS. Not even in Debian 11!

I need a Debian 10 virtual machine with Gnome Boxes to run PyBitmessage. Unfortunately.

Well, for me Ubuntu Focal is recent enough distro and it has python2. Proper flatpack packaging will probably be enough to run current PyBitmessage code on Linux distros with no python2, though I would prefer merging qt5 code first.

g1itch commented 3 years ago

Then there are other minor issues, like travis2bash not understanding the full travis syntax, the non-developers not having proper feedback.

It seems that python: - "3.6" is OK for your buildbot tasks - it's simply ignored: buildbot/travis_focal succeed.

PeterSurda commented 3 years ago

That particular feature is supported. However the VMs don't have python 3.6 so the tests can't run. I'll add python 3.6 into the VMs and then rerun your tests, but it may take a couple of days.

g1itch commented 3 years ago

I'll add python 3.6 into the VMs and then rerun your tests, but it may take a couple of days.

There is no rush. If you take a close look into python 2.7 job of the PR you will see that it's not fully correct. Now I'm discovering some strange behavior of test-mode.

ghost commented 3 years ago 
Python 3.9.1 (default, Dec  8 2020, 07:51:42) 
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Well, I only have Python 3.9 maybe I will flip that 9 to 6 somehow.

qt5 should be priority, just as to make sure that PyBitmessage works above Python 3.5 at least.

g1itch commented 3 years ago

just as to make sure that PyBitmessage works above Python 3.5 at least.

PyBitmessage currently does not support python3.

PeterSurda commented 3 years ago

@g1itch The VMs I use for buildbot now should support python3.6 through python3.10. No need to change anything, just put the desired version(s) into .travis.yml like you're doing in #1725. It will run all the tests for all the python versions listed.

g1itch commented 3 years ago

It seems I read here something you didn't write and maybe didn't even mean. Do you agree that if somebody wants to participate in the porting process, he should:

  1. chose an entity: a structure, a module, a thread or a couple of them (for example objectProcessor works together with addressGenerator)
  2. cover it by tests
  3. when the tests pass with python2, edit the code to support python3

Furthermore, it's not related to the python3 porting process but it used to be so - in order to pass a review PR should fit the following criteria:

  1. comply with PEP8
  2. have one commit per change with descriptive messages
  3. do not include unrelated changes
  4. do not introduce unneeded extra files

The latest merge violates all the points. I'm surprised that it even passed pylint check.

PeterSurda commented 3 years ago

@g1itch Your proposals make sense. I appreciate you giving feedback to the other developers.

Unfortunately for personal reasons I have very little time to code review nowadays. I also didn't have any help from the developer that normally helps me in this process as she had time off for personal reasons too (for about 1.5 months). At the same time however, I'm paying developers to work on the project, and they either need feedback or their code merged, or I pay them to do nothing. The project is in an annoying phase when python2 compatibility is being dropped by more and more distros and dependencies, but python3 code isn't working sufficiently well to have the whole program run.

For these reasons I approved PRs I otherwise wouldn't have. Hopefully, now more python3 code can be worked on, and some tests that require full app can be refactored or created from scratch using mocks. Existing code can be cleaned up. And developers can work without having to wait for each other's PRs to be merged, so development can work in parallel.

I should have more time starting in about 3-4 weeks, around the end of May. Perhaps then we can make the process more strict again.

g1itch commented 2 years ago

Meanwhile the official doc explicitly mentions test coverage and the bad practice of using sys.version_info: https://docs.python.org/3/howto/pyporting.html#use-feature-detection-instead-of-version-detection

As for requirements, you can separate them by version using environment markers e.g.

diff --git a/requirements.txt b/requirements.txt
index 899753cb..65fafe75 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,5 +1,6 @@
 coverage
 python_prctl;platform_system=="Linux"
+PyQt5;python_version>="3.8"
 psutil
 pycrypto
 six
tuxayo commented 2 years ago

Is there a branch/repo/milestone to follow the python3 port?

cornfeedhobo commented 1 year ago

@g1itch are you still working on this?

Neustradamus commented 1 year ago

Have you progressed on it?

cornfeedhobo commented 1 year ago

After revisiting this 9 months later, I think it's safe to say that bitmessage is dying. There are few distros that still package pyqt4, leaving this to be deployable to few. Unless someone decides to take this on, probably best to archive your keys and move your chats elsewhere.

PeterSurda commented 1 year ago

The development is currently paused but should resume in a couple of months. While running from source may be problematic, you can use the binaries (like appimage) until the python3 port is done.

cornfeedhobo commented 1 year ago

@PeterSurda Is there a link to the AppImage that's published from an official build source, preferably signed?

PeterSurda commented 1 year ago

This is the latest one: https://appimage.bitmessage.org/releases/20230116/ . I understand it's a bit of indirect. Automated signing isn't implemented at the moment.

cornfeedhobo commented 1 year ago

@PeterSurda Thanks for following up. Good luck with the python3 migration. I'll check back in at the end of the year :)