search

eurec4a / eurec4a-intake

Intake catalogue for EUREC4A field campaign datasets
17 stars 19 forks source link

find piñata pinned files #150

Closed observingClouds closed 10 months ago

observingClouds commented 11 months ago

set accelerated DHT client to find datasets faster

d70-t commented 10 months ago

I'm wondering if this really helps in our situation. As per the docs:

Currently, the DHT is not usable for queries for the first 5-10 minutes of operation as the routing table is being prepared. This means operations like searching the DHT for particular peers or content will not work initially.

So while finding data via DHT should be accelerated in the long term, the initial period after startup is supposed to not be usable (or at least be slow). In the CI setup, we'll (hopefully) be mostly retrieving data during this startup phase...

My guess would be, that if any, IPNI does help in our case.

observingClouds commented 10 months ago

I honestly just left this draft open as this was one suggestion I got from the Piñata team when I first pinned the files. However, it currently works very smoothly without (look at all the successful weekly tests 💪 ) so I guess their service is doing something right 🤣 I can close this for now and hopefully we get reminded about this when we face issues again.