search

near / near-indexer-for-explorer

Watch NEAR network and store all the data from NEAR blockchain to PostgreSQL database
https://near-indexers.io/docs/projects/near-indexer-for-explorer
GNU General Public License v3.0
123 stars 56 forks source link

Failed to perform query to database after 10 attempts. Stop trying. #368

Closed yerycs closed 1 year ago

yerycs commented 1 year ago

I am getting following error when start near-indexer-for-explorer. ./target/release/indexer-explorer --non-strict-mode --concurrency 1 mainnet from-latest image I am using local postgresql. When I check postgresql db, I can see first block height is indexed. So, there isn't issue in db credentials env. What's the reason?

yerycs commented 1 year ago

When I remove --non-strict-mode argument, I don't see above error. But It seems it stuck like following. image I think it should synced and It should show each block logs?

khorolets commented 1 year ago

The reasoning behind this is fairly simple.

When you use the --non-strict-mode option, some data is skipped entirely, and there are additional limitations on retries. Specifically, when using the non-strict mode, the indexer will give up after 10 retries, as you can see in your first screenshot, whereas in strict mode, retries are infinite.

If you intentionally chose to use --non-strict-mode, you can safely disregard those messages. They should eventually stop appearing or at least occur less frequently.

However, it's important to note that the Indexer for Explorer is designed to provide all data starting from the genesis block. While we added the non-strict mode as an option, the indexer was not intended to function that way, and it may behave unpredictably in non-strict mode.

Let me know if you have any further questions!

yerycs commented 1 year ago

This means, I need to run indexer explorer from the genesis block, it will run without issue?

khorolets commented 1 year ago

@yerycs Ideally, yes. If your intention is to replicate the same functionality that our instance was designed to have, then it would be best to follow the recommended usage.

I understand that it can be frustrating to use the tool as intended, and I would be happy to hear more about your use case. Based on that information, I can offer suggestions on how to achieve your goals while avoiding the need to run the Indexer for Explorer.

yerycs commented 1 year ago

Thank you. I am going to get transactions in Near. I can see public postgresql database for query. But I noticed Danger caution about usage of production. So, I think it is better to run this database on my side. That's why I am going to run indexer-explorer.

khorolets commented 1 year ago

@yerycs I am sorry, the use case you've described is a bit ambiguous.

  1. Do you need Transactions only? What about the Receipts, Execution Outcomes?
  2. Do you need all the Transactions or only the ones related to specific accounts/contracts?
  3. Do you need something else?
yerycs commented 1 year ago

I need only transactions for specific accounts only.

khorolets commented 1 year ago

Okay, I have a few options that I can suggest:

  1. Have you considered using the nearblocks.io API? It may be a good fit for your needs. (Disclaimer: I am not affiliated with this project.)

  2. Alternatively, you could create a lightweight indexer to track only specific accounts and transactions. The NEAR Lake Framework is a great starting point for this. Here's a tutorial to get you started.

  3. You might also find these projects interesting, although you may have to wait for them to become available:

I hope these suggestions are helpful to you! Let me know if you have any further questions.

yerycs commented 1 year ago

I don't like using other third party apis. I think Option 2 is good. One question, this indexer-explorer is also based in Near Lake Framework?

khorolets commented 1 year ago

Yes, it is. You can fork it and cut off everything you don't need. If you prefer this approach :)

yerycs commented 1 year ago

Thank you for your suggestions. :)