viant / afs

Abstract File Storage
Apache License 2.0
297 stars 34 forks source link

ZIP + afsc GCS fails to detect file if a key prefix of the same name exists #19

Open dchoi-specificmedia opened 1 year ago

dchoi-specificmedia commented 1 year ago

GCS seems to support both files AND directories (prefixes) with the same string. For example, gs://test_bucket/test_file.txt and gs://test_bucket/test_file.txt/sub_file.txt are both valid GCS objects.

In the case this occurs, it looks like the ZIP detection isn't handling this correctly. Specifically, github.com/viant/afsc/gs.(s *storager).List will return multiple files in the case that both a file and a prefix exist at the same path, and github.com/viant/afs/zip.newStorager() checks to make sure the List only returns one file https://github.com/viant/afs/blob/master/zip/storager.go#L241.

The currently coded behavior seems to work on the assumption that a file and a directory cannot be the same name (which is valid).

This may also coincide with the way that afsc/gs is handling the List method, as it seems to assume that the URL provided can be a prefix.