openssl reference in GDAL recipe?

[hobu]

In 2021, #543 had us add an openssl to the host section. It can be seen at https://github.com/conda-forge/gdal-feedstock/blob/main/recipe/meta.yaml#L274C11-L274C66

I have a situation where I want to run Conda Forge stuff on Google Colab, which has OpenSSL 3.0. I'd like to be able to install that, but it seems the GDAL package has a pin:

mamba install pdal=2.6.2 openssl=3.0

The following packages are incompatible ├─ openssl 3.0 is requested and can be installed; └─ pdal 2.6.2 is not installable because it requires ├─ libgdal >=3.8.3,<3.9.0a0 , which requires │ └─ openssl >=3.2.0,<4.0a0 , which conflicts with any installable versions previously reported; └─ openssl >=3.2.0,<4.0a0 , which conflicts with any installable versions previously reported

Do we really still need openssl in the host section?

[xylar]

@hobu, I think it would be an important decision for the maintainers here to make if we want to drop OpenSSL support. It seems from this section in the documentation that some functionality would be lost if we didn't have OpenSSL available during the package build: https://gdal.org/development/building_from_source.html#openssl So the question would be if we think users expect this functionality. I would hesitate to remove it.

The next question would be if we can use an older OpenSSL in our builds to retain the compatibility you're looking for. The OpenSSL pin on conda-forge is just 3: https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/conda_build_config.yaml#L667 This suggests that we should be able to build with OpenSSL 3.0 without falling out of sync with the conda-forge package ecosystem. It seems that cryptography is doing something similar: https://github.com/conda-forge/cryptography-feedstock/blob/main/recipe/conda_build_config.yaml#L10

@conda-forge/gdal, what do the rest of you think?

[rouault]

It seems from this section in the documentation that some functionality would be lost if we didn't have OpenSSL available during the package build:

We need OpenSSL for Google Cloud Storage (/vsigs) authentication when a OAuth2 service account (https://developers.google.com/identity/protocols/oauth2/service-account)

The weird thing is to have openssl in the host section of "gdal" (the python bindings) (that was probably the workaround part of the recipe), instead of just in "libgdal" where it is actually the place where it is logically needed. But here @hobu issue seems to be related with then openssl dependency coming from libgdal, and that one is required if not wanting to drop functionality from the build.

That said an alternative would be to use the cryptopp dependency (https://gdal.org/development/building_from_source.html#cryptopp), which offers slightly the same functionnality, except that cryptopp (or at least the way we use it in GDAL) doesn't understand all formats of private keys (it only accepts the ones starting with "-----BEGIN PRIVATE KEY-----" , that is PKCS#8, and not ones starting with "-----BEGIN RSA PRIVATE KEY-----" and "-----BEGIN ENCRYPTED PRIVATE KEY-----"), and I don't remember which type of private keys OAuth2 service account return.

[akrherz]

Lets see what the openssl dep is set to once main branch builds with #885 now in. To me it shows a "openssl": "3", pin being set and not >=3.2. Eh, that isn't likely right, sigh.

[akrherz]

Hi @hobu, openssl is a rather basic requirement of conda-forge as python itself has a dependency on it. It would seem difficult to get anything close to current builds of conda-forge packages and have an old openssl included?

[xylar]

@akrherz, unless we explicitly ask for openssl 3.0 in host, we'll get 3.2.1, the latest version and the pin at runtime will be >=3.2.1,<4.0.

[xylar]

@rouault, okay, thanks for the reply. It sounds like dropping openssl is not a good option.

[xylar]

The weird thing is to have openssl in the host section of "gdal" (the python bindings) (that was probably the workaround part of the recipe), instead of just in "libgdal" where it is actually the place where it is logically needed.

We can certainly try dropping that. But it's unrelated to this issue as you said.

[xylar]

I opened #887. We can at least see how it goes. I think it should be fine.

[xylar]

@hobu, bad news. #887 is failing because there are upstream dependencies of libgdal that have constraints on openssl, too. So I don't think we're going to be able to relax that constraint without this being a massive headache.

[hobu]

Thanks for the investigation. I wasn't sure about GDAL linking against openssl directly, but it seems it needs to. Maybe it should be put in the build section too?

Anyway, I'll close for now, as there's nothing we need to do here. I'll find another workaround.

[xylar]

Maybe it should be put in the build section too?

I don't think that's needed. The build section is basically for build tools, while the host section is more for libraries and such to link to. I think openssl is used as the latter.