search

mlabs-haskell / ogmios-datum-cache

6 stars 4 forks source link

Multiple fetching the same block #88

Open ryukzak opened 2 years ago

ryukzak commented 2 years ago

Problem: when the fetcher process fails, it doesn't correctly store the last fetched block, so after recovery, it re-fetched repeatedly.

Version: https://github.com/mlabs-haskell/ogmios-datum-cache/pull/86 , c69cea794786f9f1b1567c1e4cf3e81f5eb5a540

How to reproduce:

cabal run ogmios-datum-cache -- --db-connection 'host=localhost port=5432 user=aske dbname=ogmios-datum-cache' --server-port 9999 --server-api 'usr:pwd' --ogmios-address '127.0.0.1' --ogmios-port 1337 --origin | tee log

After some time:

parallels@vm ~/D/ogmios-datum-cache (t4/start-from-origin)> rg d0bf3cb095a377d5ca1a3f1a2c64bd711f6ff040c942a887930824e12ae53f04 log
298:[Info#fetchNextBlock] Fetched Byron block: (293,"d0bf3cb095a377d5ca1a3f1a2c64bd711f6ff040c942a887930824e12ae53f04")
1418:[Info#fetchNextBlock] Fetched Byron block: (293,"d0bf3cb095a377d5ca1a3f1a2c64bd711f6ff040c942a887930824e12ae53f04")
2472:[Info#fetchNextBlock] Fetched Byron block: (293,"d0bf3cb095a377d5ca1a3f1a2c64bd711f6ff040c942a887930824e12ae53f04")
4041:[Info#fetchNextBlock] Fetched Byron block: (293,"d0bf3cb095a377d5ca1a3f1a2c64bd711f6ff040c942a887930824e12ae53f04")
5552:[Info#fetchNextBlock] Fetched Byron block: (293,"d0bf3cb095a377d5ca1a3f1a2c64bd711f6ff040c942a887930824e12ae53f04")
7180:[Info#fetchNextBlock] Fetched Byron block: (293,"d0bf3cb095a377d5ca1a3f1a2c64bd711f6ff040c942a887930824e12ae53f04")

Tested on vasil test network.

Other unexpected effect:

  1. Run ODC with --origin
  2. After the fetcher process fail -- continue from the last block
t4ccer commented 2 years ago

I'll look into it, technically in database isn't stored last fetched block but last processed block (so unprocessed block will be refetched)