This PR adds .storage_options and .protocol to UPath.
The "non-pathlib" interface of UPath is then:
UPath.protocol: str the fsspec protocol
UPath.storage_options: dict[str, Any] the storage_options for the fsspec class
UPath.path: str the fsspec compatible path for use with the fsspec class
UPath.fs: AbstractFileSystem convenience attribute to access a instantiated fsspec class
the first three provide a public interface to access a file via fsspec as follows:
pth = UPath(..., **sopts)
from fsspec import filesystem
fs = filesystem(pth.protocol, **pth.storage_options)
with fs.open(pth.path) as f:
data = f.read()
This closes #129, and closes #91.
additional changes:
To support correct behavior for cloud filesystems .path had to be overwritten to return a path that includes the bucket (consistent with fsspec).
Since the webdav filesystem doesn't serialize to string well (which is why we use the webdav+http and webdav+https protocol workaround) the WebdavPath.protocol returns "webdav" and the base_url is added to the .storage_options. Which then allows to support the usecase from the code mentioned above again.
HTTPPath().path returns a full url, in line with fsspec behavior.
extra note regarding local filesystems
# protocol is empty string for non-uri localpaths:
p = UPath("/path/file.txt")
assert isinstance(p, PosixUPath)
assert p.protocol == ""
# and it's "file" for uri localpaths
u = UPath("file:///path/file.txt")
assert isinstance(p, LocalPath)
assert p.protocol == "file"
Both "file" and "" return LocalFileSystem when using fsspec.get_filesystem_class(protocol).
This PR adds
.storage_options
and.protocol
toUPath
.The "non-pathlib" interface of
UPath
is then:UPath.protocol: str
the fsspec protocolUPath.storage_options: dict[str, Any]
the storage_options for the fsspec classUPath.path: str
the fsspec compatible path for use with the fsspec classUPath.fs: AbstractFileSystem
convenience attribute to access a instantiated fsspec classthe first three provide a public interface to access a file via fsspec as follows:
This closes #129, and closes #91.
additional changes:
.path
had to be overwritten to return a path that includes the bucket (consistent with fsspec).webdav+http
andwebdav+https
protocol workaround) the WebdavPath.protocol returns"webdav"
and thebase_url
is added to the.storage_options
. Which then allows to support the usecase from the code mentioned above again.HTTPPath().path
returns a full url, in line with fsspec behavior.extra note regarding local filesystems
Both
"file"
and""
returnLocalFileSystem
when usingfsspec.get_filesystem_class(protocol)
.