search

conda-forge / pyogrio-feedstock

A conda-smithy repository for pyogrio.
BSD 3-Clause "New" or "Revised" License
0 stars 9 forks source link

Change dependency from libgdal to gdal #49

Closed theroggy closed 2 weeks ago

theroggy commented 2 months ago

GDAL recently reorganised its conda packages as discussed in https://github.com/conda-forge/gdal-feedstock/issues/722. In short: the libgdal package was split up in a core package, libgdal-core, and a list of seperate packages for many GDAL plugins, to be able to avoid the (large) footprint of the entire libgdal. For pyogrio this is also very relevant, as it depends on libgdal and most pyogrio users only need libgdal-core.

However, as libgdal-core only exists for GDAL 3.9.1, just depending on libgdal-core would mean - as far as I know - that users of future versions of pyogrio could only install that new version in combination with GDAL 3.9.1, not with older versions of GDAL, as for those only the full libgdal package exists.

Because of this, this PR at the moment "proposes" to changed the dependency to the gdal package. This is the GDAL package with python bindings (which aren't needed for pyogrio), but the advantage is that for old versions of GDAL this depends on libgdal, but starting from GDAL 3.9.1 it depends on libgdal-core, so it can offer a workaround for the issue above.

If conda would offer an option to depend on libgdal-core, but if not available (for the needed version of GDAL), fallback to libgdal, that would be even better.

Closes #48.

Checklist

conda-forge-webservices[bot] commented 2 months ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

theroggy commented 2 months ago

@conda-forge-admin please rerender

theroggy commented 2 months ago

@ocefpaf as described in the PR description above, we wonder if conda offers an option to depend on libgdal-core, but if (another dependency in) the conda environment needs an older version of GDAL, it would fallback to libgdal. Could you shed some light, or is depending on the gdal package the way to go?

ocefpaf commented 2 months ago

@ocefpaf as described in the PR description above, we wonder if conda offers an option to depend on libgdal-core, but if (another dependency in) the conda environment needs an older version of GDAL, it would fallback to libgdal. Could you shed some light, or is depending on the gdal package the way to go?

If something requests older GDAL, an older build of pyogrio would also be selected. If you want to keep that as an option for future versions, then a variant would need to be built. IMO it is not worth the effort. Even more so given that the current pyogrio will be available for both for a while if we merge this one.

With that said. you are depending on gdal only here, that means conda will solve for whatever libgdal(-core or not) the gdal version requested is available.

Does that make sense?

theroggy commented 2 months ago

If something requests older GDAL, an older build of pyogrio would also be selected. If you want to keep that as an option for future versions, then a variant would need to be built. IMO it is not worth the effort. Even more so given that the current pyogrio will be available for both for a while if we merge this one.

With that said. you are depending on gdal only here, that means conda will solve for whatever libgdal(-core or not) the gdal version requested is available.

Does that make sense?

Yes. This is indeed the reason I created this PR to depend on gdal. I expected this to be the only/best (practical) solution to be able to have recent versions of pyogrio use libgdal-core for new GDAL versions and libgdal for older GDAL versions.

Based on your answer, I think you agree?

jorisvandenbossche commented 2 months ago

But (and that's something I forgot when we chatted about this in our meeting), pyogrio binaries are built against a specific version of GDAL anyway. So it's not actually possible to install the latest pyogrio (assume the resulting binary of this rerendering) with an older version of GDAL. In that case the concern is a bit moot and it is actually fine to just directly depend on libgdal-core?

theroggy commented 2 months ago

Yes, indeed... in the corresponding rasterio issue I created (https://github.com/conda-forge/rasterio-feedstock/issues/303) this also was just mentioned...

I already encountered the case that with global pinning a rebuild was needed to use the latest version, but apparently the pinning is really on a specific version... Kind of a weird situation, as in pyogrio we try to support keep support older gdal versions as well and the GDAL C API is binary compatible as far as I know...

theroggy commented 2 months ago

Another point raised in the rasterio topic: maybe better to wait for the next version (0.10 for pyogrio) as it can be seen as it can break an installation, even if it is easy to solve it?

olivier-lacroix commented 1 month ago

Hello there, I am looking forward to this change being merged to slim down our environments. I see very few, if any, open issues are linked to the 0.10 milestone: is there any target release date for 0.10 and the change in feedstock?

jorisvandenbossche commented 4 weeks ago

@olivier-lacroix indeed, we hope to release shortly. I opened an issue to track the release at https://github.com/geopandas/pyogrio/issues/470

olivier-lacroix commented 4 weeks ago

Thanks @jorisvandenbossche ! Looking forward to it!