Joystream / status-endpoint-joystream

Status endpoint for the Joystream network
GNU General Public License v3.0
1 stars 4 forks source link

Exchanges logging and database updates #6

Closed Lezek123 closed 4 years ago

Lezek123 commented 4 years ago

This PR implements adding logs of burning transactions to the databse based on the chain subscription. Each transaction log consists of:

It also updates tokensBurned and sizeDollarPool after each processed block.

Important things to note:

Lezek123 commented 4 years ago

I refactored transfers.ts in order to make the server more failproof and be able to recover after critical errors, loss of power etc.

The flow should be like this:

  1. When recieving new block head from subscription, if there are some blocks missing between lastBlockProcessed and the new block - try to process missing blocks first (by querying api.rpc.chain.getBlockHash(blockNumber) and api.rpc.chain.getBlock(blockHash))
  2. There is a lock that prevents multiple blocks from beeing processed simountinously. Along with other implemented mechanisms it should prevent processing same block twice or processing blocks before their parent is processed. When the block that is requested to be processed is <= lastProcessedBlock - the processing is skipped.
  3. If proccessing a block takes too much time (which is specified by BLOCK_PROCESSING_TIMEOUT) and the lock cannot be released or some other critical error occurs during the processing - this event will be logged into the database, the lastProcessedBlock will be updated to the faulty block number (to prevent trying to proccess it again) and the process will exit, allowing safe restart (which can be implemented with tools like forever).

The most noticable change is that now the server can "break" at any point for any time and it should be able to recover from such situation and process all the remainig blocks between lastBlockProcessed and current block when the script is re-run. It should also be able to handle a situation when subscription is not working as expected and is not providing us all the headers or providing them in a wrong order or some of the blocks cannot be processed for some reason.

bwhm commented 4 years ago

7