search

sapcc / elektra

An opinionated openstack Web UI for consumer self service and operations.
Apache License 2.0
73 stars 29 forks source link

Bug[ObjectStore]: limit for list at 10k #1398

Closed hgw77 closed 1 week ago

hgw77 commented 4 weeks ago

At the moment there is a limit to show max 10k objects.

example project can be found in eu-de-2 -> monsoon3/cc-demo/object-storage/os-api/swift/f5dumps?delimiter=%2F

see also https://docs.openstack.org/api-ref/object-store/?expanded=#show-container-details-and-list-objects we need to page through the objects with ?marker=

hgw77 commented 4 weeks ago

Idea: load the list first and if we have 10k count load the rest in a while loop until it's done

hgw77 commented 3 weeks ago

to prepare my own test case for container object list

  1. create 12k files for i in {1..12000}; do touch "file${i}.txt"; done
  2. upload with swift upload CONTAINERNAME .
hgw77 commented 3 weeks ago

1400 will handle the problem