search

streamingfast / substreams

Powerful Blockchain streaming data engine, based on StreamingFast Firehose technology.
Apache License 2.0
159 stars 45 forks source link

substreams returns unclear "reading file existence: storage error" #215

Closed sduchesneau closed 1 year ago

sduchesneau commented 1 year ago

ex:

substreams encountered a retryable error {"error": "rpc error: code = Internal desc = error building pipeline: failed setup request: parallel processing run: scheduler run: process job result for target \"store_positions\": worker ended in error: receiving stream resp: rpc error: code = Internal desc = reading file existence: storage error"}

these failures may or may not be related to concurrent substreams running.

abourget commented 1 year ago

Bumping dstore will solve this particular issue, as it would retry. See commit: https://github.com/streamingfast/dstore/commit/fd4278f938b1e7a5033d20a7eff320b82e9d88fb which would affect the culprit here: https://github.com/streamingfast/dstore/blob/develop/gsstore.go#L170

(reading file existence happens here: https://github.com/streamingfast/bstream/blob/develop/filesource.go#L275 called by https://github.com/streamingfast/bstream/blob/develop/filesource.go#L578 )

sduchesneau commented 1 year ago

Duplicate of https://github.com/streamingfast/substreams/issues/222