search

locationtech-labs / geopyspark

GeoTrellis for PySpark
Other
179 stars 59 forks source link

Unstable shapely dependency #712

Closed soxofaan closed 2 years ago

soxofaan commented 2 years ago

geopyspark currently has a dependency on shapely>=1.6b3: https://github.com/locationtech-labs/geopyspark/blob/97bcb17a56ed4b4059e2f0dbab97706562cac692/setup.py#L20

Yesterday a first alpha version of shapely 2.0 was released: https://github.com/shapely/shapely/issues/962

Because of the b3 in the version constraint of shapely, pip thinks it is ok to install alpha/beta versions in general.

For example

$ docker run --rm -it python /bin/bash
root@3c6cef8bb5a2:/# pip install geopyspark
...
Collecting shapely>=1.6b3 (from geopyspark)
  Downloading https://.../shapely-2.0a1.tar.gz (261kB)

So if you have an app that depends on geopyspark, you will blindly drag in an alpha version of shapely.

soxofaan commented 2 years ago

As development on geopyspark has stalled, this probably won't be fixed soon here.

Obvious workaround of course is constraining or pinning down the shapely version in your app/env dependencies (requirements.txt/setup.py/setup.cfg/...)

jpolchlo commented 2 years ago

As noted, we're not maintaining this package any longer,and so the standard open-source caveats apply: feel free to fork the repo to make this change if it suits you. I would, however, point you in the direction of pyrasterframes if you're looking for a replacement that is likely to be maintained for a while. It's basically GPS, but for Spark SQL, so it's more efficient, and generally more featureful (even if it may lack support for focal operations, though some of those ops have been added). RF also targets much more modern versions of Spark, so you don't have to be stuck on a three year old release.

I'm going to close this issue for now, since even if you make a PR, the publishing mechanism is broken, so there won't be any artifacts to draw on after the change.