search

yandex-cloud / geesefs

Finally, a good FUSE FS implementation over S3
Other
686 stars 45 forks source link

Feature request: consider using `prefix` for directory listing #57

Closed Felixoid closed 1 year ago

Felixoid commented 2 years ago

Hello.

As a following for #56, is it possible to implement a switcher in S3 OPTIONS to use prefix instead of start-after/marker request attribute?

R2 received a bug report from us, but it's not clear when/if it will be resolved.

I understand that the change can be a bit more expensive compared to the current scheme, but, unfortunately, R2 is just broken with start-after/marker.

vitalif commented 1 year ago

Hi. I can try to add an option like --no-fast-listings :D but if their S3 doesn't work correctly with start-after/marker then some listings will anyway be broken - even with prefix start-after will still have to be used for continuations, i.e. for directories that have more than 1000 entries. I'd suggest you to move somewhere else. There are already at least 2 bugs that they don't want to fix: 1) 403 on throttling 2) broken listings

vitalif commented 1 year ago

If you don't want to use Yandex ;-) there's for example Backblaze B2

Felixoid commented 1 year ago

As I've mentioned, we're kind of stuck with R2 since it's the cheapest available option. We host our packages there, and for 3.5 TB of data and 45+ TB monthly traffic, it costs only €60. With the native integration of website workers, it looks like the best option for hosting.

On the other hand, the inconsistency and struggle to make a new release, probably I'll try to use another intermediate step to publish the repo metadata.

Felixoid commented 1 year ago

BTW, the API is fixed. I am not sure if the feature makes sense then.

I hope you haven't started working on it yet ^_^