Update supported Python versions

jugmac00 commented 4 years ago

drop support for Python versions < 3.6
add support for Python versions 3.7, 3.8 (if missing)
fix linting errors (if present)

If you are a beginner, and need to help for this issue, especially during Hacktoberfest, please comment here or reach out to me via https://jugmac00.github.io/

Aniruddha-Shriwant commented 4 years ago

Hey, @jugmac00 sir, I am very beginner in coding I also cant assure you that I can solve this issue for you but I want to try to solve the issue I just know little JavaScript and just learned Git and Github. I'm interested in solving this issue can you please tell me what should I have to do to solve this

Aniruddha-Shriwant commented 4 years ago

Hey sorry to comment out again, I basically don't know about python but I want to solve this issue, I want to learn something new. I had searched for some information and then I found out this: https://packaging.python.org/guides/dropping-older-python-versions/ As python is literally new to me I first didn't understand your issue but then I understand what exactly you mean As the link should I need to do like that?

jugmac00 commented 4 years ago

Hi @Aniruddha-Shriwant great to hear you want to contribute!

As you have no experience with Python, this will be a lot of ground to cover.

Please note, this issue is not about Python programming, but about testing, linting and packaging.

I see you are very motivated and I certainly will help you to get this done.

I have planned to have some time in October - as the https://hacktoberfest.digitalocean.com/ happens then. You do some open source work, like here, and then you get a nice T-Shirt :-D

I hope you are patient - as this will take some time, as there is much to learn.

As a starter, which operating system do you use? Windows, Linux or Mac?

Aniruddha-Shriwant commented 4 years ago

Well, thanks for telling me about Hacktoberfest, I really liked that, I will surely participate in the event Currently, my Operating system is Windows 10 with Intel i5 8th Gen and 8 gigs of Ram. I basically came here seeing the Javascript Language in the repo I know JS but this Python is very new to me I am planning to contribute in Open Source with Javascript, also want to try https://summerofcode.withgoogle.com/ As I am in the first year I am not familiar with all this but I can try to solve your issue, I will definitely learn something new from it.

jugmac00 commented 4 years ago

I am running Linux, so I have no experience with running Python on Windows, which - from what I heard - can be a bit problematic from time to time.

Do you use Windows Subsystem for Linux? (WSL)

Either way, at first you need to have a working Python installation on your PC.

If you use WSL with Ubuntu, I could help you, but I do not know what is the best way to install one, or even several Python versions on a Windows machine. That would be something you'd have to figure out yourself, ie read about it.

Aniruddha-Shriwant commented 4 years ago

Hey, @jugmac00 I have Python installed on my Windows machine and I have some experience of using WSL on windows, As you said I will figure out how to install several python versions from WSL by my own. Can I contact you using your email-address ?

jugmac00 commented 4 years ago

@Aniruddha-Shriwant Which Linux distribution do you use with WSL? Ubuntu? Fedora? Depending on the distribution, installing Python works differently.

Feel free to write to my email address (I saw you did already), but I think it would be good to keep the conversation here at one place - this is also a reference for others who want to learn to contribute.

Aniruddha-Shriwant commented 4 years ago

Well, I use Ubuntu 20.04 LTS

jugmac00 commented 4 years ago

Awesome!

Now, you can install Python 3 via the following documentation (option 1). https://phoenixnap.com/kb/how-to-install-python-3-ubuntu

Python comes in different versions.

There was a very big leap between the now legacy Python 2 and the current Python 3.

Python keeps getting developed, and nowadays every 12-18 months a new minor Python version gets released, but the previous ones still get supported.

For this project we want to support Python 3 in the minor versions 3.6, 3.7 and 3.8.

Please install all of them via the above linked article.

You can make sure everything works by entering e.g.

python3.8

in your shell.

When this is done, you could clone this repository.

The step after that would be to setup tox - but before you do that, I will try to explain more about what tox is and what kind of problems it tries to solve.

Aniruddha-Shriwant commented 4 years ago

Hey @jugmac00 , Sorry for the late reply I had installed Python from reading that article, and also I had fork the repository and clone it into my local system.

jugmac00 commented 4 years ago

Did you also install the other two Python versions?

For this project we want to support Python 3 in the minor versions 3.6, 3.7 and 3.8.

Once this is done, I try to explain what tox is and how to get it working.

If you are curios, you can read a bit at https://tox.readthedocs.io/en/latest/

Aniruddha-Shriwant commented 4 years ago

Done @jugmac00 , I had installed the three Python versions required also I had read the article you provided. From that, I came to know about what is tox and some information regarding it

jugmac00 commented 4 years ago

Ok, let's recap.

This library uses Python, Python 3.

Python 3 gets a new version every 12-18 months. Library authors usually support a couple of Python versions. Here Python 3.6 - 3.8.

Supporting means, one makes sure the library is compatible with those Python versions.

Writing tests helps a lot.

Usually, when you install a Python package or library, you do not install it globally, as the dependencies could interfere with other packages.

Usually, you install a Python package in a so called "virtual environment", or short venv. This has nothing to do with a virtual machine. Actually, it is just a "copy" of a Python interpreter, and isolated site-packages (this are the installed packages).

As a library author, you would have to create a separate virtual env for each supported Python version - and then run the tests in each. This is cumbersome, that is why there tox.

tox creates virtual environments and then runs e.g. tests or also linters (which make sure the code is well formatted and free of some kind of errors).

So, now we are in a kind of dilemma - we want to install tox, but not in the global Python installation, but actually we'd like tox so we don't have to manually create virtual environments... this makes the head spin :-D

There are several ways to do this. What I like:

create a folder in your user account, e.g. /home/youruser/opt/
within this folder, create a virtual environment; here, the Python version does not matter much, let's use Python 3.8; python3.8 -m venv tox; -m venv means use the virtual env module, and tox is just the name
activate the environment by entering the following command source /home/youruser/opt/tox/bin/activate
now, if you use Python, or pip (the python package installer), you do not use the global ones, but the ones from your newly created virtual environment
in most shells now the prompt changes and shows that you have the environment activated
if you ever want to deactivate the environment, just enter deactivate followed by enter
make sure the environment is active; enter which python - this should now show you the interpreter in your venv
now you can install tox by entering pip install tox

This is a one time thing - and you almost made it.

Now, tox is installed in this virtual environment, but I and maybe you too like to have tox available globally.

If you know what Linux symlinks are, you can create a symlink from /home/youraccount/opt/tox/bin/tox to a directory which is on your path, on my pc this is e.g. /home/jugmac00/bin/. You can find out your path via entering echo $PATH in your terminal (not in a Python repl).

If symlinks are new, and you do not want to read about them, no worries, just activate your virtual environment so tox as a command is available, or enter the complete path to tox, whenever you want to call it e.g. /home/youruser/opt/tox/bin/tox - you can always arrow up or arrow down in your terminal to use command again - or use CTRL+r and start typing your command to search in your command history.

This was a lot of stuff - and maybe confusing too. But now the fun part could begin - if this all worked out.

While in the cloned repository, you can now enter tox - tox will read the tox.ini file within the repository and then creates a separate virtual environment for each Python version, and also for the linter and one for measuring the test coverage, and then runs all the tests / linter.

And don't be disappointed! Some tests will fail - but we are here to fix the tests, and then add support for the newer Python versions.

Do not hesitate to ask if something is unclear! I will either try to explain it more thoroughly, or link to a text or video, or create a video to explain it.

Aniruddha-Shriwant commented 4 years ago

Hey @jugmac00 , I'm done with all the setup and Thanks for explaining me in so much detailed way ;) I was having some troubles with Windows and WSL in installing Python, So I changed my OS to Ubuntu 20.04, I'm using it on Virtual box. So now I had installed everything without any errors. Basically as you said at last run tox in the cloned repository, I got the following output So, what's this all, can you please tell?

jugmac00 commented 4 years ago

Ok, as explained above, tox tried to create a virtual environment for each configured entry in the envlist in tox.ini. https://github.com/morepath/morepath_cerebral_todomvc/blob/c8d69a09916e9ec4237ac43239c5bd04602e7917/tox.ini#L2

As you do not have installed Python 3.4 and 3.5, you got a skipped: interpreter not found message, but that is ok, as we do not want to support those old Python versions.

The error message for Python 2.7 is one I never saw before - after googling it, I found these two issues:

https://github.com/pypa/virtualenv/issues/1873 https://github.com/pypa/virtualenv/issues/1875

But as we also do not plan to support Python 2.7, let's skip it for a moment. Maybe it works out for Python 3.

Ok, as we plan the first code changes, you should create a new branch, e.g. git checkout -b update-python-versions or similar.

Once this is done, you could replace the existing, outdated Python versions with new ones, e.g. in the envlist delete py27, py34, py35 and add py36, py37, py38 - and in the other environments replace the basepython 3.5 with 3.8.

Then you could run tox again (fingers crossed, you do not see the above error message again).

I try to record a quick video tomorrow, where I explain how tox works and how to understand the tox.ini file.

jugmac00 commented 4 years ago

Here we go.. the promised video: https://youtu.be/JoOsVGqsehM

Aniruddha-Shriwant commented 4 years ago

Great explanation in the video 🔥 I will now make a branch and do the further things.

Aniruddha-Shriwant commented 4 years ago

Hey, @jugmac00 I had done the specified things, I had also installed some other versions of python but there was some different type of error : So what exactly this line means :

ERROR: py36: InvocationError for command /usr/bin/python3 -m virtualenv --no-download --python /usr/bin/python3.6 py36 (exited with code 1) ERROR: py37: InvocationError for command /usr/bin/python3 -m virtualenv --no-download --python /usr/bin/python3.7 py37 (exited with code 1) ERROR: py38: InvocationError for command /usr/bin/python3 -m virtualenv --no-download --python /usr/bin/python3 py38 (exited with code 1)

Did I miss some python tools or anyother setup to install ? But when I run this commands it gives me the output: aniruddha@aniruddha-VirtualBox:~$ python2.7 --version Python 2.7.18rc1 aniruddha@aniruddha-VirtualBox:~$ python3.5 --version Python 3.5.10 aniruddha@aniruddha-VirtualBox:~$ python3.6 --version Python 3.6.12 aniruddha@aniruddha-VirtualBox:~$ python3.7 --version Python 3.7.9 aniruddha@aniruddha-VirtualBox:~$ python3.8 --version Python 3.8.2 So my machine is having python versions installation correctly right ?

jugmac00 commented 4 years ago

Ok, now it is getting a bit rough...

You have installed the correct Python versions, but there seems to be a problem with the virtualenv package.

It is the problem you had with Python 2.7 from above, where I already listed the corresponding issues on the virtualenv tracker: pypa/virtualenv#1873 pypa/virtualenv#1875

I never experienced those problems before, so I have to read the issues myself.

Maybe also try to read them and think about how you installed virtualenv?

Did you sudo apt-get install virtualenv?

You could show me the output of dpkg -l | grep env.

Python packaging is a thing, and especially packaging with Debian/Ubuntu is a bit of a mess.

Maybe @asottile knows how to proceed?

I'd probably go for a which virtualenv and then try to deinstall it, and as @gaborbernat mentioned in the above linked issues, you probably also need to deinstall the second virtualenv - though I have no clue where this second one would be coming from.

Anthony has made a couple of videos about virtualenv, see here https://www.youtube.com/results?search_query=anthony+virtualenv

It is a bit a of a pity that you, @Aniruddha-Shriwant , as a complete Python beginner encounter so many problems.

gaborbernat commented 4 years ago

@jugmac00 yeah, probably is https://www.youtube.com/watch?v=O390_abzo08 😢

Aniruddha-Shriwant commented 4 years ago

Ya, @jugmac00 I had installed virtualenv with the way you had mentioned, I will read the 2 issues link and try to figure out what's happening wrong in my case and sure I will watch that video and will show you the output of dpkg -l | grep env afterward. At last, a lot of thanks that you are helping me with this silly issue :) I had learned a lot of things than I was knowing before

Aniruddha-Shriwant commented 4 years ago

Hey @jugmac00 , I had solved the previous issue but there was one problem the same you got for fakedb.py and setup.py saying W504 error. As you said by there is syntax of writing 2 lines of space after class, So that error was solved but the one of binary operator was not yet solved :| then as you did in tox.ini, I too similarly added [flake8] ignore = W504 and then the error gets ignored and I got the following output after running tox, As in your video you had passed py35 tests similarly I had passed the tests for python versions from 3.6, 3.7&3.8 Here I had just edited tox.ini and fakedb.py. So after this what's the next step?

Aniruddha-Shriwant commented 4 years ago

Ok, as explained above, tox tried to create a virtual environment for each configured entry in the envlist in tox.ini.

https://github.com/morepath/morepath_cerebral_todomvc/blob/c8d69a09916e9ec4237ac43239c5bd04602e7917/tox.ini#L2

As you do not have installed Python 3.4 and 3.5, you got a skipped: interpreter not found message, but that is ok, as we do not want to support those old Python versions.

The error message for Python 2.7 is one I never saw before - after googling it, I found these two issues:

pypa/virtualenv#1873 pypa/virtualenv#1875

But as we also do not plan to support Python 2.7, let's skip it for a moment. Maybe it works out for Python 3.

Ok, as we plan the first code changes, you should create a new branch, e.g. git checkout -b update-python-versions or similar.

Once this is done, you could replace the existing, outdated Python versions with new ones, e.g. in the envlist delete py27, py34, py35 and add py36, py37, py38 - and in the other environments replace the basepython 3.5 with 3.8.

Then you could run tox again (fingers crossed, you do not see the above error message again).

I try to record a quick video tomorrow, where I explain how tox works and how to understand the tox.ini file.

Hey @jugmac00 ,here you mention replacing basepython 3.5 woth 3.8 in other environments So can you tell me what exactly means by the other environments ? I had updated tox.ini but don't know about this other environments

henri-hulski commented 4 years ago

So that error was solved but the one of binary operator was not yet solved :| then as you did in tox.ini, I too similarly added [flake8] ignore = W504 and then the error gets ignored and I got the following output after running tox,

@Aniruddha-Shriwant I would recommend to fix the binary errors and ignore W503 instead, which is deprecated.

See https://www.flake8rules.com/rules/W503.html and https://www.flake8rules.com/rules/W504.html

Aniruddha-Shriwant commented 4 years ago

@henri-hulski Ok, I will definitely look at it

Aniruddha-Shriwant commented 4 years ago

Hey, @henri-hulski I had seen those above links, I had then edited the setup.py like this : Before :

long_description = (
    io.open('README.rst', encoding='utf-8').read() + '\n\n' +
    io.open('CHANGES.rst', encoding='utf-8').read())

After :

long_description = (`
    io.open('README.rst', encoding='utf-8').read() + '\n\n'
    + io.open('CHANGES.rst', encoding='utf-8').read())

And instead of ignoring W504, I had now ignored W503. Well I don't know much about Python but I can see In setup.py near the last lines of code there is this part:

classifiers=[
        'Intended Audience :: Developers',
        'Environment :: Web Environment',
        'License :: OSI Approved :: BSD License',
        'Topic :: Internet :: WWW/HTTP :: WSGI',
        'Programming Language :: Python :: 2.7',
        'Programming Language :: Python :: 3.4',
        'Programming Language :: Python :: 3.5',
    ]

So here I can see the old versions of Python, should I edit those with new(3.6, 3.7, 3.8) So what should I do?

jugmac00 commented 4 years ago

@Aniruddha-Shriwant

I had solved the previous issue

Awesome! Would you mind to share what exactly you did? So I learn something new, and when somebody searches for a solution to this problem, and finds this discussion, there is the solution. There is nothing more frustrating when you have a problem, find a helpful discussion, and the somebody writes "I solved it!" but doesn't tell how :-D

there was one problem the same you got for fakedb.py and setup.py saying W504 error.

Yes, this was to be expected. In the video, I showed you how to handle those problems, but I did not commit my solution. This issue here is about that you learn how to handle it yourself :-)

W503 or W504

While I may heard of it, I did not remember that one should prefer one over the other. This is the great thing about open source - many help together and you learn something new every day! Thanks @henri-hulski

I had passed the tests for python versions from 3.6, 3.7&3.8

Awesome! We are very close to the finish!

So can you tell me what exactly means by the other environments ?

I don't see your tox.ini as you have not created a pull request yet, but I was referring to testenv:pep8 and testenv:coverage sections - and I guess you already updated the basepython from 3.5 to 3.8, otherwise the envs would not successully run.

So here I can see the old versions of Python, should I edit those with new(3.6, 3.7, 3.8) So what should I do?

Great catch!

Actually, tox and setup.py are not the only places, where the supported Python versions are defined.

We also have them in .travis.yml - this is a configuration file for Travis, which is a CI service. Everytime you create a pr or push to a branch, CI runs the test suite. This makes it easier for maintainers to review pull requests and to ensure that only tested code gets merged.

From here, there are several ways how to approach this.

I guess it is very fine to just update both setup.py and .travis.yml, and then create a pull request!

Having a look at setup.py, I also noticed there is one key-value pair missing.

You could add python_requires=">= 3.6", maybe just above the classifiers key.

P.S.: As there are up to three or more files which all define the Python version, there is a tool out there to check whether these values fit together - it is called check-python-versions. I have installed this on my pc, but there are other ways to run the check.

One, you already know now - you could create a tox environment which installs check-python-versions and runs it.

Another widely used way is pre-commit - similarily to tox, pre-commit runs other tools, especially linters and formatters (and checkers :-) ).

As I said - just manually edit those files, and create a PR. If you really want to dive deep and also learn more about how tox, travis and pre-commit fit together, I created another short video ( https://www.youtube.com/watch?v=0405xzlycD8&ab_channel=J%C3%BCrgenGmach ) which I wanted to present as a Lightning Talk at PyIreLand meetup.

Aniruddha-Shriwant commented 4 years ago

Well Thanks a lot @jugmac00 for this info, I will do all that you had mentioned and yet I had not Committed my changes, I will soon do this and make a PR and as you said:

This is the great thing about open source - many help together and you learn something new every day!

This is very true in my case when I was thinking about trying to solve this issue at the beginning without knowing Python :D, I was a bit scared but later with all the links you provided I learned very new things which I may never learn if I don't comment here

Aniruddha-Shriwant commented 4 years ago

Hey, @jugmac00 I had done with the changes you may see that in #3 and to talk about the previous issues that I had faced with virtualenv, the issues link you had given ( pypa/virtualenv#1873) had helped me, form there I see that If I would degrade my virtualenv version then the problem gets solved I know @gaborbernat had talked about there that this was not the actual solution, and this may just temporarily solve the issues but I had tried many other ways but it hadn't worked :(

jugmac00 commented 4 years ago

@Aniruddha-Shriwant Awesome! That's it! Your pull request looked great, so I merged it.

Thanks to you the tests pass again, see https://travis-ci.org/github/morepath/morepath_cerebral_todomvc

The morepath projects have "some" more broken CI jobs, see https://travis-ci.org/github/morepath?page=1&tab=repositories&timeInterval=month

You should know now how to fix them pretty much by yourself.

Don't forget - in October there is the #hacktoberfest - I can't recall - either you need 3 or 5 pull requests, then you get a nice t-shirt - so maybe spare some easy tasks for then.

Other than that - if you are interested in open source work, try to find something you really like and maybe use yourself, and just start contributing - either by code or documentation or replying to issues or comment on pull requests...

You got my email address - do not hesitate to contact me!

morepath / morepath_cerebral_todomvc

Update supported Python versions #2