Python conventions improvements ideas

src-d / guide

Aiming to be a fully transparent company. All information about source{d} and what it's like to work here.

Creative Commons Attribution Share Alike 4.0 International

294 stars 101 forks source link

Python conventions improvements ideas #331

Open se7entyse7en opened 5 years ago

se7entyse7en commented 5 years ago

The current status of python conventions is a bit poor. Here are some of the things that I think could be decided and that would help to improve consistency across python projects:

supported versions: from some slack discussions it seems to be >=3.5, they could be tested only on travis or using for example tox
deps management, packaging and publishing: pip + requirements.txt, pipenv, poetry (I personally like poetry as it also handles packaging and publishing)
packaging and publishing: setup.py + twine, poetry
virtualenvs compatibility: should the project be compatible only with virtualenvs or maybe even conda?
best practices: a common configuration for flake, usage of other tools such as isort for imports sorting

smacker commented 5 years ago

/cc @vmarkovtsev

vmarkovtsev commented 5 years ago

This is the current state of all our ML projects:

We support 3.5+. Generally, we support any Python version equal to or newer than in Ubuntu LTS -2 years ago. That is 16.04 and 3.5. Hence we will drop 3.5 in 2020.
We use pip and requirements.txt and are totally happy with them, apart from a few minor complaints. The rationale is that the ML folks have never heard about poerty, myself included, and it is completely not widespread in the Python (ML/DS) community. Anaconda is, but no.
setup.py + twine. It works - that's all we need from packaging atm.
We do not use virtual environments. We always run shared stuff in Docker.
This is something we are investigating. There is some common configuration in the latest projects, and the older ones do not use flake at all in favor of vanilla pycodestyle.
Type hinting guide is under development. I am not going to force it over the whole company though.

se7entyse7en commented 5 years ago

@vmarkovtsev thank you for the explanation!

smola commented 5 years ago

We support 3.5+. Generally, we support any Python version equal to or newer than in Ubuntu LTS -2 years ago. That is 16.04 and 3.5. Hence we will drop 3.5 in 2020.

This seems sensible. We should add this already, since it's our current minimum.

deps management, packaging and publishing: pip + requirements.txt, pipenv, poetry (I personally like poetry as it also handles packaging and publishing)

We use pip and requirements.txt and are totally happy with them, apart from a few minor complaints.

Using pipenv for applications would make sense to me. This has been discussed a few times on Slack, but never got a clear picture.

Type hinting guide is under development. I am not going to force it over the whole company though.

It can be a good recommendation for everyone though.

se7entyse7en commented 5 years ago

Type hinting guide is under development. I am not going to force it over the whole company though.

I think that really helps to understand the code faster, and it should be a really very nice to have at least for "public" methods. Just as an example we started using type hinting in lookout-sdk and along with sphinx docstring makes the code easy to follow (example).