Closed telezhnaya closed 3 years ago
@telezhnaya how long does it take to apply the migration?
@frol the honest answer is "too long anyway". 3 minutes if everything was already updated before. More than one hour to apply it till 31 December 2020 at once (killed this process because it's insane anyway).
I think the best idea is to comment UPDATE
statement in up.sql
so that we will never apply this migration fully. The comment will be used just to know the idea of how it should be done.
In the real world, we should apply it in pieces. I suggest creating a script that updates the data splitting it by timestamp.
I think the best idea is to comment UPDATE statement in up.sql so that we will never apply this migration fully. The comment will be used just to know the idea of how it should be done. In the real world, we should apply it in pieces. I suggest creating a script that updates the data splitting it by timestamp.
This sounds like a plan :+1:
@telezhnaya last time we had an issue with "too timeconsuming" migration I've come up with a script and left it for the history here https://github.com/near/near-indexer-for-explorer/pull/99#issuecomment-838871700 you might find it handy. Let me know if you need my help.
I just remembered that we may also need an index over the (changed_in_block_timestamp, index_in_block)
to speed up sorting.
Partially resolves #77 (this PR introduces an easy way to sort the account changing events:
ORDER BY changed_in_block_timestamp, index_in_block
)I am still waiting for the server updates, want to test it on the production-like env, that's why I marked it as a draft.
We should apply the migration by pieces, otherwise, it will take too long to update existing lines.