search

streamingfast / dstore

Blob stores abstractions. Supports AWS S3, Google Storage, Azure Blob File Storage, and local FS
https://dfuse.io
Apache License 2.0
11 stars 10 forks source link

Losing files when power off machine unexpected #4

Closed mowind closed 2 years ago

mowind commented 2 years ago

Last week the server power off which is running sf-ethereum, after restarted, we found firehose report many errors:

{"level":"info","ts":1643084612.7032914,"logger":"bstream","caller":"firehose/firehose.go:236","msg":"source shutting down","error":"file source failed: processing of file \"0007930800\" failed: unable to create block reader: unable to read file header: EOF"}

Check merged-blocks file 0007930800, its size is 0:

0 Jan 24 03:25 0007930800.dbin.zst

We should Sync file before Close: https://github.com/streamingfast/dstore/blob/6fa17808533b71069ad9a6f7bdfa92b2285e0957/localstore.go#L131