filecoin-project / lily

capturing on-chain state for the filecoin network

Other

49 stars 44 forks source link

Improve lily task retry logic #1163

Open frrist opened 1 year ago

frrist commented 1 year ago

Description

When running lily as a distributed indexer and a job fails to complete all tasks successfully, lily reschedules all tasks to be retried. This is inefficient in some cases since some of tasks may have been completed successfully. Lily should instead reschedule only the failed tasks to be retried.

Acceptance criteria

[ ] Only reschedule failed tasks to rety

Where to begin

kasteph commented 1 year ago

closed from #1162

birdychang commented 1 year ago

1162 will stop all task retries when there's no fatal error. We still need a better solution to retry only fail tasks