IBM / aspera-cli

IBM Aspera CLI
Apache License 2.0
77 stars 17 forks source link

browse more than 100 files through aspera cli shares plugin #141

Closed BharatMundra123 closed 5 months ago

BharatMundra123 commented 6 months ago

we are trying to browse all files in the provided aspera folder. but able to fetch only 100 records at a time. need help in browse all the files present in respective folder we are using below command "ascli shares files browse "

BharatMundra123 commented 6 months ago

Hi @laurent-martin Could you please check this and suggest

laurent-martin commented 6 months ago

The shares plugin, internally uses the node plugin, as shares browsing/download API is similar to the node "gen3" API:

https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/api/API--aspera--ibm-aspera-node-api#post551762460

Currently full listing using multiple calls (paging) is not implemented in ascli for node gen3 (it is in other places, though)

Nevertheless, it's possible to pass additional API parameters with option query to influence the result:

So, you can ask for a large number of results with: --query=@json:'{"count":10000}'

It's possible that not all elements would be returned, but you can list the next ones with: --query=@json:'{"count":10000,"skip":10000}'

Not sure about the maximum value for count.

BharatMundra123 commented 6 months ago

Thanks @laurent-martin for fast response. unfortunately limit for count is 1000 we are looking for more than 2000 files

laurent-martin commented 6 months ago

If you want to try, you can get a beta with full browsing, including recursive, here: https://ibm.biz/aspera-cli-beta

BharatMundra123 commented 5 months ago

able to fetch with mentioned comment It's possible that not all elements would be returned, but you can list the next ones with: --query=@json:'{"count":10000,"skip":10000}'