Next step in Python 3 compatibility

[cclauss]

See https://docs.python.org/3/howto/pyporting.html#update-your-code

• We need to use one of these two tools... Which one? (My inclination would be Futurize but either way)
• See the second paragraph... Can you please pass just the test suite through whichever tool makes the most sense. After that we can make sure that the test suite still passes before doing the rest of the code.

[operatorequals]

OK. I will create a new branch with the nasty code the Futurize created. As you can expect both Py2 and Py3 tests fail before even beggining... I used entropy package for Py3 and found that hilarious screenshot that summarizes Python Packaging System quite well:

Using pytest package both for Py2 and Py3 tests (just to be in the same page). It's also a good idea to change it in travis-ci too (it prevents STDOUT and gets a lot faster).

Working on it right now...

Can't thank you enough for your effort on that!

[operatorequals]

https://github.com/operatorequals/covertutils/tree/py3_compatibility

Let's have fun!

[operatorequals]

Oh and a line like the following in your shells rc (_profile?)would be really handy:

alias pytest3='python3 -m pytest'

[cclauss]

Aggressive... The trick was to Futurize just the Test directory first and get all tests to pass before Futurizing the code. But let's try it this way. I do not know entropy -- have to read up. The way we have Travis working is that we have a Py2 env and a Py3 env that test in parallel. You can do that locally with tox but I have never done that. Let's see how we progress.

[operatorequals]

entropy is the package that is used to test the homebrew hashing algorithm. It just ensures that it returns "seemingly" random data. The joke was on the "collecting entropy" pip's quote, that implies that Python Packaging is a bit random (have you tried to upload to PyPI lately?)...

[operatorequals]

If you find it difficult to proceed with the full futurized mess I can recreate the branch from master

[operatorequals]

This list (posted in the link you posted) sums it all up nicely:

To summarize:

• Decide which of your APIs take text and which take binary data
• Make sure that your code that works with text also works with unicode and code for binary data works with bytes in Python 2 (see the table above for what methods you cannot use for each type)
• Mark all binary literals with a b prefix, textual literals with a u prefix
• Decode binary data to text as soon as possible, encode text as binary data as late as possible
• Open files using io.open() and make sure to specify the b mode when appropriate
• Be careful when indexing into binary data

The problem is that almost all the API work with bytes but accept strings...

[cclauss]

Second paragraph was...

Regardless of which tool you choose, they will update your code to run under Python 3 while staying compatible with the version of Python 2 you started with. Depending on how conservative you want to be, you may want to run the tool over your test suite first and visually inspect the diff to make sure the transformation is accurate. After you have transformed your test suite and verified that all the tests still pass as expected, then you can transform your application code knowing that any tests which fail is a translation failure.

[operatorequals]

yes, I firgured out later. I thought you were initially referring to the "Use feature detection instead of version detection" paragraph. Want me to scrap and recreate?

[cclauss]

Yes. I think you should recreate with:

1. future added to requirements.txt
2. futurize only run on the tests directory

We need to have Python 2.7 running in a stable way against the futurized tests before futurizing the rest of the codebase and then finally making Python 3 changes again.

[operatorequals]

The new branch is ready at "https://github.com/operatorequals/covertutils/tree/py3_compatibility"

[adamchainz]

Hi, just came to this project from Python Weekly. FYI I've seen six used much more often in the python ecosystem and thus modernize is my preferred approach.

[cclauss]

Hi @operatorequals I have been away from this project but could come back to it now if you want to make Py3 support mainstream.

I see that currently the Python 3 tests are failing on TabErrors. This is because Python 3 has zero tolerance when space and tab indentation are mixed in the same code block. If you want to stick with tab indentation, that needs to be used everywhere. We could fix the TabErrors and then edit .travis.yml to take Python 3 out of allow_failures mode to enforce discipline on all pull requests.

Your thoughts?