search

cedadev / esgf-stac-client

A Python client to access ESGF STAC data
BSD 3-Clause "New" or "Revised" License
2 stars 1 forks source link

Are only 10,000 records coming back? It's a problem! #16

Open agstephens opened 1 year ago

agstephens commented 1 year ago

In this example, getting back 10,000 records only is a major problem. @agstephens 

result = client.search()
my_set = set()

for item in result.items():
     my_set.add(item.properties['variable'][0])

And the result was: {'pr', 'psl', 'tas', 'tasmax', 'tasmin', 'ua', 'zg500'}

Why there's no 'hus', while search with filter returns the item with 'hus'?

Adik8688 commented 1 year ago

I spotted that max_items=100 by default I suppose that caused this problem

agstephens commented 1 year ago

@Adik8688: where is the default set? And can you increase it to 10,000?

Adik8688 commented 1 year ago

I'm not sure where the default is set, but I can increase it to 10000, however I cannot go beyond it

agstephens commented 1 year ago

You can use other filtering options to search different parts of the record space - that will help you go beyond 10,000.