search

RDFLib / prez

Prez is a data-configurable Linked Data API framework that delivers profiles of Knowledge Graph data according to the Content Negotiation by Profile standard.
BSD 3-Clause "New" or "Revised" License
23 stars 10 forks source link

Reducing dependencies #259

Open ashleysommer opened 2 months ago

ashleysommer commented 2 months ago

I notice Shapely is listed in the dependencies of Prez.

One big issue with having Shapely as a dependency is shapely subsequently depends on Numpy, which is a very large dependency and often needs to be compiled on installation (if the target platform does not use binary wheels).

After searching through the codebase, I see Shapely is only used in one location, that is to convert cql->Polygon->wkt String.

I recently made a PR in the rdf2geojson repo to replace the use of Shapely with a vendorised copy of GeoMet. https://github.com/Kurrawong/rdf2geojson/pull/1 Note, it is vendorised so we can use it without the sub-dependency on "click", that we don't need. As well as applying some out-of-tree patches to fix multiple WKT serialization errors.

GeoMet is pure-python, is (relatively) small and is great at converting from GeoJSON to WKT and back again (and some other formats). It is compatible with the Python Geo Interface convention (__geo_interface__) so should work for our use case in Prez.

Secondly, uvicorn is included as a non-optional requirement in the dependencies list. Again, the main issue is the sub-dependency on uvloop that again is a rather large dependency that also often needs to be compiled on installation (if the target platform does not use binary wheels).

Those who are familiar with the codebase will know that uvicorn is used in exactly one place in the codebase, that is at the bottom of main.py, if you are running prez as a standalone application it will spawn uvicorn to run the app.

There are lots of other ways of running the FastAPI ASGI app other than uvicorn (eg, in a Azure Function App, a AWS Lambda app, Apache ASGI wrapper, another ASGI runner like hypercorn or daphne). So I believe uvicorn should be an optional dependency, because not everyone is going to want Prez to supply its own server.

The motivation for these changes is to reduce the size of the requirements.txt file generated by poetry:

poetry export --format=requirements.txt > requirements.txt"

This is used when generating the requirements for building the Azure Function App bundle, and I feel the total built bundle size could be smaller.

ashleysommer commented 2 months ago

Note, this is the whole requirements.txt that is currently generated when exporting requirements from poetry.

aiocache==0.12.2 ; python_version >= "3.11" and python_version < "4.0"
annotated-types==0.7.0 ; python_version >= "3.11" and python_version < "4.0"
anyio==4.4.0 ; python_version >= "3.11" and python_version < "4.0"
cachetools==5.4.0 ; python_version >= "3.11" and python_version < "4.0"
certifi==2024.7.4 ; python_version >= "3.11" and python_version < "4.0"
click==8.1.7 ; python_version >= "3.11" and python_version < "4.0"
colorama==0.4.6 ; python_version >= "3.11" and python_version < "4.0" and (platform_system == "Windows" or sys_platform == "win32")
dnspython==2.6.1 ; python_version >= "3.11" and python_version < "4.0"
email-validator==2.2.0 ; python_version >= "3.11" and python_version < "4.0"
fastapi-cli==0.0.5 ; python_version >= "3.11" and python_version < "4.0"
fastapi==0.111.1 ; python_version >= "3.11" and python_version < "4.0"
frozendict==2.4.4 ; python_version >= "3.11" and python_version < "4.0"
h11==0.14.0 ; python_version >= "3.11" and python_version < "4.0"
httpcore==1.0.5 ; python_version >= "3.11" and python_version < "4.0"
httptools==0.6.1 ; python_version >= "3.11" and python_version < "4.0"
httpx==0.27.0 ; python_version >= "3.11" and python_version < "4.0"
idna==3.7 ; python_version >= "3.11" and python_version < "4.0"
isodate==0.6.1 ; python_version >= "3.11" and python_version < "4.0"
jinja2==3.1.4 ; python_version >= "3.11" and python_version < "4.0"
lxml==5.3.0 ; python_version >= "3.11" and python_version < "4.0"
markdown-it-py==3.0.0 ; python_version >= "3.11" and python_version < "4.0"
markupsafe==2.1.5 ; python_version >= "3.11" and python_version < "4.0"
mdurl==0.1.2 ; python_version >= "3.11" and python_version < "4.0"
numpy==2.0.1 ; python_version >= "3.11" and python_version < "4.0"
oxrdflib==0.3.7 ; python_version >= "3.11" and python_version < "4.0"
pydantic-core==2.20.1 ; python_version >= "3.11" and python_version < "4.0"
pydantic-settings==2.4.0 ; python_version >= "3.11" and python_version < "4.0"
pydantic==2.8.2 ; python_version >= "3.11" and python_version < "4.0"
pygments==2.18.0 ; python_version >= "3.11" and python_version < "4.0"
pyld==2.0.4 ; python_version >= "3.11" and python_version < "4.0"
pyoxigraph==0.3.22 ; python_version >= "3.11" and python_version < "4.0"
pyparsing==3.1.2 ; python_version >= "3.11" and python_version < "4.0"
python-dotenv==1.0.1 ; python_version >= "3.11" and python_version < "4.0"
python-multipart==0.0.9 ; python_version >= "3.11" and python_version < "4.0"
pyyaml==6.0.2 ; python_version >= "3.11" and python_version < "4.0"
rdflib==7.0.0 ; python_version >= "3.11" and python_version < "4.0"
rich==13.7.1 ; python_version >= "3.11" and python_version < "4.0"
shapely==2.0.5 ; python_version >= "3.11" and python_version < "4.0"
shellingham==1.5.4 ; python_version >= "3.11" and python_version < "4.0"
six==1.16.0 ; python_version >= "3.11" and python_version < "4.0"
sniffio==1.3.1 ; python_version >= "3.11" and python_version < "4.0"
sparql-grammar-pydantic==0.1.2 ; python_version >= "3.11" and python_version < "4.0"
starlette==0.37.2 ; python_version >= "3.11" and python_version < "4.0"
toml==0.10.2 ; python_version >= "3.11" and python_version < "4.0"
typer==0.12.3 ; python_version >= "3.11" and python_version < "4.0"
typing-extensions==4.12.2 ; python_version >= "3.11" and python_version < "4.0"
uvicorn==0.30.6 ; python_version >= "3.11" and python_version < "4.0"
uvicorn[standard]==0.30.6 ; python_version >= "3.11" and python_version < "4.0"
uvloop==0.19.0 ; (sys_platform != "win32" and sys_platform != "cygwin") and platform_python_implementation != "PyPy" and python_version >= "3.11" and python_version < "4.0"
watchfiles==0.23.0 ; python_version >= "3.11" and python_version < "4.0"
websockets==12.0 ; python_version >= "3.11" and python_version < "4.0"
ashleysommer commented 2 months ago

Additional note, if Prez adds the rdf2geojson dependency (which it may do when implementing geojson serialization support), you can probably use the GeoMet version that is bundled with rdf2geojson.