fsspec / universal_pathlib

pathlib api extended to use fsspec backends
MIT License
251 stars 44 forks source link

Path loses leading `//` after `joinpath` operation #18

Closed kcpevey closed 2 years ago

kcpevey commented 3 years ago

I can do this:

from upath import UPath
import pandas as pd

pd.read_csv(str(UPath('gs://bucket/filename.csv')))

but not this:

up = UPath('gs://') 
pd.read_csv(str(up.joinpath('bucket', 'colombia/filename.csv')))

This is because

str(UPath('gs://bucket/filenamecsv'))

gives 'gs://bucket/colombia/filename.csv' which maintains the leading //
while

str(UPath('gs://').joinpath('bucket', 'filename.csv'))

gives 'gs:/bucket/filename.csv' which is invalid.

This is important because I need to be able to programmatically construct the path separate from the bucket. The idea behind this tool is to keep me from using f strings, but it appears that is my only option here.

andrewfulton9 commented 3 years ago

Thanks for raising this, @kcpevey. I'll think of how to fix this. In the meantime, can you check if you can pass pd.read_csv(UPath(gs://).joinpath('bucket', 'filename.csv')). UPath is an os.PathLike object, so should be able to be passed directly.

kcpevey commented 3 years ago

@andrewfulton9 Nope, that fails with No such file or directory: 'gs:/ae.... i.e. its losing the second /

andrewfulton9 commented 3 years ago

Thanks for checking that. I'll fix this as soon as I get a chance.

grisaitis commented 2 years ago

Can this issue be closed? Looks like that commit and merged PR might have resolved this.