fsspec / universal_pathlib

pathlib api extended to use fsspec backends
MIT License
251 stars 44 forks source link

UPath constructor does not take UPath object #25

Closed brl0 closed 3 years ago

brl0 commented 3 years ago

This isn't really a big problem, but below is an issue I ran into, and I am hoping a little fix might simplify my usage.

from pathlib import Path

from upath import UPath

Path(Path("."))  # this works
UPath(UPath("."))  # this does not work
AttributeError: 'PosixPath' object has no attribute 'decode' ```python --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) in ----> 1 UPath(UPath(".")) ~/anaconda3/envs/py38/lib/python3.8/site-packages/upath/core.py in __new__(cls, *args, **kwargs) 13 args_list = list(args) 14 url = args_list.pop(0) ---> 15 parsed_url = urllib.parse.urlparse(url) 16 for key in ["scheme", "netloc"]: 17 val = kwargs.get(key) ~/anaconda3/envs/py38/lib/python3.8/urllib/parse.py in urlparse(url, scheme, allow_fragments) 370 Note that we don't break the components up in smaller bits 371 (e.g. netloc is a single string) and we don't expand % escapes.""" --> 372 url, scheme, _coerce_result = _coerce_args(url, scheme) 373 splitresult = urlsplit(url, scheme, allow_fragments) 374 scheme, netloc, url, query, fragment = splitresult ~/anaconda3/envs/py38/lib/python3.8/urllib/parse.py in _coerce_args(*args) 122 if str_input: 123 return args + (_noop,) --> 124 return _decode_args(args) + (_encode_result,) 125 126 # Result objects are more helpful than simple tuples ~/anaconda3/envs/py38/lib/python3.8/urllib/parse.py in _decode_args(args, encoding, errors) 106 def _decode_args(args, encoding=_implicit_encoding, 107 errors=_implicit_errors): --> 108 return tuple(x.decode(encoding, errors) if x else '' for x in args) 109 110 def _coerce_args(*args): ~/anaconda3/envs/py38/lib/python3.8/urllib/parse.py in (.0) 106 def _decode_args(args, encoding=_implicit_encoding, 107 errors=_implicit_errors): --> 108 return tuple(x.decode(encoding, errors) if x else '' for x in args) 109 110 def _coerce_args(*args): AttributeError: 'PosixPath' object has no attribute 'decode' ```

Due to these lines: https://github.com/Quansight/universal_pathlib/blob/dbc57f7ec4c8bb8b2e22975f6b62e25b2b82ecff/upath/core.py#L14-L15

fsspec has a stringify_path utility function that might be appropriate:

fsspec.utils.stringify_path ```python from fsspec.utils import stringify_path def stringify_path(filepath): """Attempt to convert a path-like object to a string. Parameters ---------- filepath: object to be converted Returns ------- filepath_str: maybe a string version of the object Notes ----- Objects supporting the fspath protocol (Python 3.6+) are coerced according to its __fspath__ method. For backwards compatibility with older Python version, pathlib.Path objects are specially coerced. Any other object is passed through unchanged, which includes bytes, strings, buffers, or anything else that's not even path-like. """ if isinstance(filepath, str): return filepath elif hasattr(filepath, "__fspath__"): return filepath.__fspath__() elif isinstance(filepath, pathlib.Path): return str(filepath) elif hasattr(filepath, "path"): return filepath.path else: return filepath ```

I think this would fix it:

from fsspec.utils import stringify_path
...
            url = stringify_path(args_list.pop(0))
            parsed_url = urllib.parse.urlparse(url)

Happy to submit a small PR and test if you'd like.

Also, thanks for the awesome project!

andrewfulton9 commented 3 years ago

That looks like a good addition to me. Would gladly accept a PR.