Super low rate limits make checkpoints.testnet.sui.io infeasible for indexing

[fl0ydj]

Steps to Reproduce Issue

E.g. run the suins-indexer crate and log checkpoint lag for testnet. Observe 15-20s lag on average (most likely due to 429s and exponential backoff retries)

Expected Result

Being able to run the indexer on testnet with reasonable lag.

Happy to also take advice on alternative endpoints. I am not aware of any

[fl0ydj]

Feedback about this? Today the indexer doesn't seem to be working at all

[singhparshant]

The indexer is stuck for us as well.

[alexanderludwig]

+1

[phoenix-o]

thanks for the feedback! It's a known issue for testnet. We're migrating the bucket with checkpoints to a different remote store. The ETA is early next week

[fl0ydj]

Ok! What's up with the indexer right now though? Since 8pm UTC last night, we haven't been able to receive any data? Is this part of your migration and will stay like this until done or will this be resolved before?

[phoenix-o]

it's a different issue. Should be fixed now

[fl0ydj]

Ok perfect! Can confirm that it works again now :)

Keep us updated about the migration next week then. We got a customer demo on Friday (and we are not on mainnet yet), so would be great to know early if there are any delays on your end haha

[fl0ydj]

Also, what about these really low rate limits? On average we need like 20 requests to get any checkpoint using the sui-data-ingestion crate

Will this be fixed with the migration?

[phoenix-o]

on testnet? The situation should improve with the migration and be similar to the mainnet bucket. You can always switch to a different bucket(I'm not aware of an alternative for the testnet though atm) or even a FN (assuming it's sufficiently unpruned and has corresponding data). For instance, set remote_store_url to https://<address_of_your_full_node>:<port>/rest.

[fl0ydj]

Any updates on the migration? Also, we have been getting: remote reader transient error invalid value: integer 4, expected variant index 0 <= i < 4 Do you know the root cause of this? Will this be fixed with the migration?

[phoenix-o]

expected variant index 0 <= i < 4

please update the indexer code(i.e. rebase on top of main). There are some recent changes introduced in sui core, so indexer needs to be updated too

[phoenix-o]

Any updates on the migration?

we switched to a different store. You can retry now

[fl0ydj]

Looking solid so far. Thanks so much for the great work and timely delivery :) Props to the team!

Will close this accordingly