search

fsspec / universal_pathlib

pathlib api extended to use fsspec backends
MIT License
251 stars 44 forks source link

Fix serialization of UPath #47

Closed normanrz closed 2 years ago

normanrz commented 2 years ago

Currently, pickled instances of UPath will be un-pickled as pathlib.PosixPath. This PR fixes that by implementing __reduce__ and __setstate__ methods.

Fixes #44

andrewfulton9 commented 2 years ago

Thanks for the PR, @normanrz! I made a comment on how to get the CI/CD tests passing. If you make that change, I get this merged ASAP.

normanrz commented 2 years ago

I fixed the tests.

I also refactored the / (i.e. __truediv__) operator so that the following two expressions produce the same representation. Previously, there were differences in _parts and _url.path, which are cloaked in _format_parsed_parts.

path_a = UPath("gcs://bucket/folder")
path_b = UPath("gcs://bucket") / "folder"

Also, the mock: tests gave me a bit of trouble on Windows, because the file path that is used is Windows-style path, but UPath implements _posix_flavour, which expects a / at the beginning in some methods.

andrewfulton9 commented 2 years ago

I also refactored the / (i.e. __truediv__) operator so that the following two expressions produce the same representation. Previously, there were differences in _parts and _url.path, which are cloaked in _format_parsed_parts.

path_a = UPath("gcs://bucket/folder")
path_b = UPath("gcs://bucket") / "folder"

Great! Thank you!

Also, the mock: tests gave me a bit of trouble on Windows, because the file path that is used is Windows-style path, but UPath implements _posix_flavour, which expects a / at the beginning in some methods.

Interesting. I'm surprised that hasn't shown up on the CI/CD windows tests. Thanks for adding the fix though. I'll go ahead and merge this.

normanrz commented 2 years ago

Thanks for merging! Could you please also create a new pypi release?