near / near-indexer-for-explorer

Watch NEAR network and store all the data from NEAR blockchain to PostgreSQL database
https://near-indexers.io/docs/projects/near-indexer-for-explorer
GNU General Public License v3.0
123 stars 56 forks source link

fix: Action args_json must be extracted correctly whenever possible #313

Closed frol closed 1 year ago

frol commented 1 year ago

This PR fixes function args extraction (args->args_json) for actions (action_receipt_actions and transaction_actions tables). This bug affected all the records since 0.10.22 release. Consider removing and re-indexing all the records from the time you deployed 0.10.22+ release. I will also provide a patch that you can use to re-index without cleaning up the database and instead just run it over the existing data, so it will forcefully update the data.

frol commented 1 year ago

Here is a temporary patch that you can use to re-index the data in-place without removing the records from the database:

diff --git a/src/db_adapters/receipts.rs b/src/db_adapters/receipts.rs
index d327b8c..01838c9 100644
--- a/src/db_adapters/receipts.rs
+++ b/src/db_adapters/receipts.rs
@@ -572,7 +572,9 @@ async fn store_receipt_actions(
     crate::await_retry_or_panic!(
         diesel::insert_into(schema::action_receipt_actions::table)
             .values(receipt_action_actions.clone())
-            .on_conflict_do_nothing()
+            .on_conflict((schema::action_receipt_actions::receipt_id, schema::action_receipt_actions::index_in_action_receipt))
+            .do_update()
+            .set(schema::action_receipt_actions::args.eq(diesel::pg::upsert::excluded(schema::action_receipt_actions::args)))
             .execute_async(pool),
         10,
         "ReceiptActionActions were stored in database".to_string(),
diff --git a/src/db_adapters/transactions.rs b/src/db_adapters/transactions.rs
index 3e794fc..ac3c4ea 100644
--- a/src/db_adapters/transactions.rs
+++ b/src/db_adapters/transactions.rs
@@ -177,7 +177,9 @@ async fn store_chunk_transactions(
     crate::await_retry_or_panic!(
         diesel::insert_into(schema::transaction_actions::table)
             .values(transaction_action_models.clone())
-            .on_conflict_do_nothing()
+            .on_conflict((schema::transaction_actions::transaction_hash, schema::transaction_actions::index_in_transaction))
+            .do_update()
+            .set(schema::transaction_actions::args.eq(diesel::pg::upsert::excluded(schema::transaction_actions::args)))
             .execute_async(pool),
         10,
         "TransactionActions were stored in database".to_string(),

So the ultimate solution would be:

  1. Deploy plain 0.10.28 release to all your nodes, so all the new data is indexed correctly
  2. Deploy a new node with 0.10.28 release with the above patch applied and run from the block height that is the first date when you deployed 0.10.22+ release, and let it re-index those old blocks