search

allenai / cached_path

A file utility for accessing both local and remote files through a unified interface.
https://cached-path.readthedocs.io/
Apache License 2.0
35 stars 12 forks source link

Optional dependencies for GCS/S3/HuggingFace-Hub? #223

Open sidharthrajaram opened 7 months ago

sidharthrajaram commented 7 months ago

Hi, big fan of the project!

cached_path is certainly very capable because of the support for GCS/S3/HuggingFace out of the box, but one issue is that it seems to bring a lot of dependency baggage for situations that really only require HTTP/local path functionality.

For example, one project I am working with relies on the HTTP and local cache path functionality, but there are conflicts due to the boto3 package. It's kind of like a printer refusing to print black-and-white when there is no magenta ink.

To that end, I'm curious whether there's interest in making the HF/S3/GCS dependencies optional? I can make a PR if needed! Thank you!

epwalsh commented 7 months ago

Hey @sidharthrajaram yes we'd be happy to accept a PR for this.