search

fsspec / s3fs

S3 Filesystem
http://s3fs.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
884 stars 273 forks source link

Is it possible to disable globbing? #895

Open polm-stability opened 1 month ago

polm-stability commented 1 month ago

I'm trying to copy files from s3 to a local system. The files were downloaded from a variety of sources and kept their original filenames, some of which seem to have included ** or other wildcard patterns. Is there a way to disable globbing or escape the filenames? I am using filenames directly as supplied by an s3fs ls call.

martindurant commented 1 month ago

The top level user methods like cat expect to expand paths. However, there are other methods like cat_file that work one file at a time and don't expand.

Currently, only recursive= is passed on by the bulk functions to .expand_path. There has been talk of disabling or being able to configure this process more, but these is no such option right now.

polm-stability commented 1 month ago

Thanks, get_file is exactly what I needed.

Maybe it would be good to mention this in the documentation? Something like adding "does not expand paths" to the get_file documentation, and something like "You can use get_file to get a single file" in the get documentation. Now that you've explained it it's a clear convention, but being new to the library I never would have realized it.

martindurant commented 1 month ago

Updating docstring to make things clearer is always a good idea. Would you like to contribute an update? The change would be in fsspec.spec.AbstractFileSystem , which is S3FileSystem's superclass.