Make batch message more clever

jennijuju commented 2 years ago

Checklist

[X] This is not a new feature or an enhancement to the Filecoin protocol. If it is, please open an FIP issue.
[X] This is not a new feature request. If it is, please file a feature request instead.
[X] This is not brainstorming ideas. If you have an idea you'd like to discuss, please open a new discussion on the lotus forum and select the category as Ideas.
[X] I have a specific, actionable, and well motivated improvement to propose.

Lotus component

[ ] lotus daemon - chain sync
[ ] lotus miner - mining and block production
[ ] lotus miner/worker - sealing
[ ] lotus miner - proving(WindowPoSt)
[ ] lotus miner/market - storage deal
[ ] lotus miner/market - retrieval deal
[ ] lotus miner/market - data transfer
[ ] lotus client
[ ] lotus JSON-RPC API
[ ] lotus message management (mpool)
[X] Other

Improvement Suggestion

as stated in PRs like https://github.com/filecoin-project/lotus/pull/8986

batch messages may hit block limit - lotus should be clever enough know when to chunk them and how.

jennijuju commented 2 years ago

if message runs out of the gas (which the error returned is ErrOutOfGas), lotus should handle the message more intelligently, i.e: do less batching and split things into multiple messages. Message we wanna handle:

P1: SubmitWindowedPost
P3: Sector Compaction

current state of art & related previous work: https://github.com/filecoin-project/lotus/releases/tag/v1.16.1

shrenujbansal commented 2 years ago

Splitting the batch for WindowedPost is considerably high effort for the below reasons:

Construction and submission of PoSTs happen in completely different pipelines so a simple error check on failure and repacking of batches is not feasible
An option is to use the FVM to simulate the PoST submissions and check if they satisfy the gas requirements. However, this requires a way to mock proofs while preserving gas usage which would need work on the FVM

In addition, https://github.com/filecoin-project/builtin-actors/pull/677 in NV17 significantly reduces the underlying problem this is trying to solve with improving gas usage for expiring sectors

Instead, we will implement a simpler change to have a user config which makes partitions with recovery sectors, the only partition in a single PoST message. This should further reduce the occurrence of the problem and is a significantly simpler change

arajasek commented 2 years ago

Descoping from nv17 milestone now that 9413 has landed -- it needs to be backported into release/v1.18.0 though.

filecoin-project / lotus