search

near / near-indexer-for-explorer

Watch NEAR network and store all the data from NEAR blockchain to PostgreSQL database
https://near-indexers.io/docs/projects/near-indexer-for-explorer
GNU General Public License v3.0
123 stars 56 forks source link

access_keys table almost empty #284

Closed baldator closed 2 years ago

baldator commented 2 years ago

Hi, I'm running an indexer instance on mainnet with the following parameters:

--store-genesis --stream-while-syncing --non-strict-mode --concurrency 1 sync-from-interruption

The indexer is running as expected and transactions are flowing to the db. However, I've noticed that the access_keys table is almost empty (I only have the 35 records from the genesis block). Am I missing something? How can I troubleshoot the problem?

Thanks

khorolets commented 2 years ago

It's because of --non-strict-mode it skips access keys. Indexer for Explorer is designed to work with the database that has all the blockchain data from the genesis. Non-strict mode was introduced for "emergency" situations like catching up the freshest data while another instance is indexing the history.

baldator commented 2 years ago

thank you for your explanation. Would only sync this table from the public indexer and remove the --non-strict-mode works, considering that we aren't willing to have a node in archive mode to sync the entire chain?

A suggestion for you: it would be super lovely to have a regular pg_dump of the db in order to make it much faster (and cheaper) to run an up-to-date indexer without setting up an archive node.

khorolets commented 2 years ago

Would only sync this table from the public indexer and remove the --non-strict-mode works, considering that we aren't willing to have a node in archive mode to sync the entire chain?

Should work. At least worth trying.

A suggestion for you: it would be super lovely to have a regular pg_dump of the db in order to make it much faster (and cheaper) to run an up-to-date indexer without setting up an archive node.

Thank you for the idea. We don't have neither resources nor capacity to do that.

baldator commented 2 years ago

Thank you for your feedback. I let you know if the manual sync will work.

frol commented 2 years ago

A suggestion for you: it would be super lovely to have a regular pg_dump of the db in order to make it much faster (and cheaper) to run an up-to-date indexer without setting up an archive node.

There is a major work being done in solving the root cause for this issue, which is inability to use the public shared access in the first place. We plan to expose the data warehouse with pay-as-you-go model. Unfortunately, it is currently on hold due to other priorities. Feel free to share your original problem here (I don't believe anyone would sacrifice themselves to only run a copy of indexer-for-explorer for no good reason behind it)