Build Conda package

[nmaarnio]

We should create the necessary files and workflow for building a Conda package soon, and certainly before the April release. Many end users may opt for Conda for it's easy GDAL installation.

[nmaarnio]

@nialov you have built conda packages before if I recall correctly. Could you help out with this task?

[nialov]

For release to conda-forge you want to use a "feedstock" repository. See e.g. https://github.com/conda-forge/fractopo-feedstock or https://github.com/conda-forge/geopandas-feedstock. The fractopo uses the pypi package as the source while the geopandas one uses a GitHub release. The pypi approach might be easier as the feedstock automatically gets a pull request when the pypi version is incremented.

Been a while since I created fractopo-feedstock so it is not that fresh in my memory. You can check the guides yourself or I can do it in a week or two (https://conda-forge.org/docs/maintainer/adding_pkgs.html).

[nmaarnio]

Okay. I think in a couple weeks time is just fine, so if you could do this that'd be great. I'll focus on other things for now at least.

[nialov]

Okay the packaging is going ok, see https://github.com/conda-forge/staged-recipes/pull/25419. There are a few dependency problems which need to be solved and a new release made. As part of checks during the conda build process it uses pip check to see if dependencies in pyproject.toml match those installed based on meta.yaml. See log below:

 + pip check
 eis-toolkit 0.1.0 has requirement gdal==3.4.3, but you have gdal 3.6.3.
 eis-toolkit 0.1.0 has requirement seaborn==0.13.0, but you have seaborn 0.13.2.
 momepy 0.0.0 has requirement shapely>=2, but you have shapely 1.8.5.post1.
 pysal 23.7 has requirement giddy>=2.3.3, but you have giddy 0.0.0.
 pysal 23.7 has requirement momepy>=0.5.3, but you have momepy 0.0.0.
 WARNING: Tests failed for eis_toolkit-0.3.0-pyhd8ed1ab_0.conda - moving package to /home/conda/staged-recipes/build_artifacts/broken
 TESTS FAILED: eis_toolkit-0.3.0-pyhd8ed1ab_0.conda

Problem is strict dependency constraints for gdal and seaborn in pyproject.toml. I will make a pull request with the needed changes.

[nialov]

Also @nmaarnio I should probably add you as maintainer alongside myself? Anyone else? You will get an invite to the conda-forge team after the pull request on staged-recipes is accepted and the feedstock repository is generated.

[nmaarnio]

Awesome! I think gdal was originally locked to 3.4.3 since there was not any newer compatible and official binary for libgdal for Ubuntu 22.04, but it should be relaxed for the release now. I think there was no particular reason to lock seaborn version, might have been just a mistake.

Right now running the unit tests also generate a lot of warnings (2499 in Poetry env for me currently) of which many seem to be caused by certain package versions. Should we try to adjust some dependencies to reduce them?

And yes, please add me as a maintainer too. I don't think anyone else from Gispo at least for now.

[nialov]

Awesome! I think gdal was originally locked to 3.4.3 since there was not any newer compatible and official binary for libgdal for Ubuntu 22.04, but it should be relaxed for the release now. I think there was no particular reason to lock seaborn version, might have been just a mistake.

Cool, will relax the constraints.

Right now running the unit tests also generate a lot of warnings (2499 in Poetry env for me currently) of which many seem to be caused by certain package versions. Should we try to adjust some dependencies to reduce them?

E.g. deprecation warnings are not that dangerous when using e.g. poetry as the dependencies are locked. If you want to clean warnings I recommend setting pytest to error on warning and then going over the code to squash them. In pyproject.toml:

[tool.pytest.ini_options]
filterwarnings =[
    # this error line causes pytest to error on ALL warnings except those explicitly ignored below
    "error",
    # can also filter warnings that should be ignored
    # "ignore:The Shapely:UserWarning:geopandas",
    ]

But not as part of this conda packaging work.

And yes, please add me as a maintainer too. I don't think anyone else from Gispo at least for now.

👍

[nialov]

Done, eis_toolkit is installable from conda-forge:

conda install -c conda-forge eis_toolkit

https://github.com/conda-forge/eis_toolkit-feedstock

[nmaarnio]

Awesome, thanks!

[nialov]

Closing as the package is built! Some closing notes:

The conda-forge bot automatically creates pull requests whenever a new release (or tag?) of eis_toolkit is released, as seen here: https://github.com/conda-forge/eis_toolkit-feedstock/pull/1. They are pretty safe to automatically merge IF the GitHub workflows (on Azure pipelines) pass in the pull request.

Because the entire test suite is ran as part of the CI there is probably lots of potential for minor test failure. We might have to relax the test suite in the feedstock at some point by e.g. only running a subset of important tests.

I can take care of debugging and merging but if there is a situation where you have released a new version of eis_toolkit and you need the conda-forge package updated fast, please tag me in the feedstock pull request! And of course @nmaarnio you can also fix and merge the pull requests.