OpenSenseAction / poligrain

Simplify common tasks for working with point, line and gridded sensor data, focusing on rainfall observations.
https://poligrain.readthedocs.io
BSD 3-Clause "New" or "Revised" License
2 stars 10 forks source link

Fix problem with mypy check of import for matplotlib, numpy etc #3

Open cchwala opened 5 months ago

cchwala commented 5 months ago

During #2 I had to exclude the imports of matplotlib, xarray and numpy from mypy checking, see here. The problem was I got mypy errors like this

error: Cannot find implementation or library stub for module named "numpy"

If I understand correctly, excluding these package imports from mypy checking means we can also not use the internal types for type hinting. Hence, if we really want to make use of type hints and mypy (which seems to be recommended nowadays), we should try to resolve this problem. It should actually be easy to fix since it is most likely just a problem of the virtual env that is used when running the linting. Info from mypy on that problem is here.

cchwala commented 5 months ago

This might have been an issue with my local environment, because the virtual environments that nox and poetry build and use can access packages from my conda base env since I run everything in a terminal which has my conda base env switched on, i.e. the PYTHONPATH or PATH in general is probably adjusted so that conda packages are visible. The virtual env just adjust the PATH in addition, but conda packages would still be accessible from with the virutal env.

I got mypy working for the new functions for spatial.py developed in #16, see the linting in CI here. I tried different things, e.g. installing mypy and pre-commit in poetry and add it to the nox install command for the pre-commit run. I removed some of the changed in nox.py and pyproject.toml so I am not sure why it still works on my local machine, maybe because the packages are still installed. But, most importantly it works in CI.

cchwala commented 5 months ago

...the problem described at the top of this issue is there again when removing another package (I now tried xarray in 732f637ade94dc5adedac684ea80b6664c40c8b0) from

[[tool.mypy.overrides]]
module = "numpy,num.....

where I put all of them to not have the error at each import.

If I recall correctly, strangely the problem did also not appear with the first run of pre-commit, but only after I change something to force a mypy error because of wrong return type.

cchwala commented 5 months ago

The issue with evns for mypy during pre-commit runs seems to be a known problem. There are solutions, but they require custom hooks or things like that for pre-commit. For now, I disabled mypy in #18 and just document the info I found and then will later tackle this issue.

The solution has to be that nox installs all package dependencies for the mypy run. Maybe this has to be separated from a normal pre-commit linting run to not slow down the process, i.e. mypy would not be run with each commit. Or maybe if the env is cached it could be fast enough.