Closed theogaraj closed 4 months ago
While continuing to work with UPath
I noticed another problem related to trailing slashes. I don't know if this has the same underlying problem as the issue I described above or if it should be its own separate thing, but here's what I'm seeing...
Attempting to glob over a directory (either local, or S3). I've defined a UPath called files_location
, and I'm attempting to iterate over all the files with for filepath in files_location.glob('*.json')
files_location = UPath('data/myjsons')
or files_location = UPath('data/myjsons/')
(note the trailing slash in the second one)files_location = UPath('s3://mybucket/myjsons')
works, whereas file_locations = UPath('s3://mybucket/myjsons/')
(note the trailing slash) does not workThank you for reporting. Handling double slashes in s3 is still an open issue.
While it's supported from the s3 side, I've seen a few cases so far, where those keys were created unintentionally do to bugs in the scripts that copied the files to s3.
Nevertheless, we need to add better support for handling those cases where you want to access existing s3 buckets which are not under your control.
Would you be interested in creating a PR with a testcase for your specific issue?
Cheers, Andreas
Which operating system and Python version are you using? Windows 11, Python 3.9.6
Which version of this project are you using? 0.1.4
What did you do?
/
operator to create a new S3 path by joining a string to the original UPathWhat did you expect to see? I would expect to see a single slash between
s3://mybucket/withkey
andsubfolder/myfile.txt
What did you see instead? Resultant S3Path has a double slash:
S3Path('s3://mybucket/withkey//subfolder/myfile.txt')
Additional info This works as expected when I have just a bucket, or if I have bucket and key without a trailing slash Just a bucket:
Bucket and key but no trailing slash: