fsspec / universal_pathlib

pathlib api extended to use fsspec backends
MIT License
211 stars 36 forks source link

storage_options doesn't work for local paths #156

Closed normanrz closed 8 months ago

normanrz commented 8 months ago

For local paths (PosixUPath and WindowsUPath), accessing the storage_options property fails because there is no _kwargs attribute. See https://github.com/fsspec/universal_pathlib/blob/main/upath/core.py#L243C36-L243C43 @ap-- is that something you're fixing already in #152? Otherwise, I can make a fix.

ap-- commented 8 months ago

Hey @normanrz, which version are you running?

# pip install universal_pathlib
>>> import upath
>>> upath.__version__
'0.1.3'
>>> upath.UPath("abc").storage_options
{}
>>> upath.UPath("abc")
PosixUPath('abc')

_kwargs is set for these pathlib-local paths in https://github.com/fsspec/universal_pathlib/blob/7e3836d0291539cb9876f3d5590c30eec9fbd228/upath/implementations/local.py#L62

normanrz commented 8 months ago

This is how it breaks:

from upath import UPath
(UPath("C:/Users/foo/") / "bar").storage_options

So, it is probably caused by the path concatenation.

ap-- commented 8 months ago

Interesting, I assumed there was a test for this.

Okay, so we need to override ~_format_parsed_parts~ _from_parsed_parts in the subclasses too to add _kwargs. I can roll out a fix later tonight. But, if you have time feel free to add the fix ❤️

I'm a lot happier with the 3.12 based implementations in #152, and am looking forward to being able to backport those. It'll be a bit bumpy to do that transition though, but 🤷 ... Just need to find a bit of time to write more tests for that.