Listing directory is slow with versioned bucket when looking for delete markers

[cyberduck]

b4af655 created the issue

Just updated to the latest version and it takes forever to login to my S3 bucket. When trying to upload files it is also very slow.

Before the update today everything worked super fast, as it should be.

[cyberduck]

@ylangisc commented

Do you have a default path set to a bucket in your bookmark? If yes, is there versioning enabled for this bucket? This might be a consequence of our improved versioning support (#10357).

[cyberduck]

b4af655 commented

Hi yla,

Yes, I have a default path set and yes, my bucked has versioning enabled.

[cyberduck]

@ylangisc commented

Is there any chance to post a transcript (Cmd-L) from the authentication process?

[cyberduck]

b4af655 commented

yla,

Unfortunately I don't think I can share that, however, I did notice that in the authentication process it makes HTTP requests for all folders I have there. And I have a lot of folders:

Here's an example request:

GET /?max-keys=1&versions&prefix=apps%2Fthirdparty%2Fumarket%2F&delimiter=%2F HTTP/1.1

And there are dozens of not hundreds of such requests in that log. I assume that's what started causing the delay. Thank you.

[cyberduck]

@ylangisc commented

Thanks, that's enough information. There reason for this behaviour is that in the latest version we try to properly decide if a folder is hidden or not for versioned buckets. For this decision we have to issue a list operation for each folder in order to see if there is at least one object without delete marker in the subtree.

[cyberduck]

b4af655 commented

Any updates here?

[cyberduck]

@ylangisc commented

In 0b255d28b03c2378584496de0bfd594c0fa40ebe.

[cyberduck]

@dkocher commented

Milestone renamed

[cyberduck]

18f6d8a commented

This seems to be back - a folder with nothing but a subfolder (and a ton of deleted/glaciar'd files) takes many seconds to load. (Version 7.3.0 (32621))