deal failed: (State=26) deal data verification failed: proposal CommP doesn't match calculated CommP

aueam commented 2 years ago

Checklist

[X] This is not a security-related bug/issue. If it is, please follow please follow the security policy.
[X] This is not a question or a support request. If you have any lotus related questions, please ask in the lotus forum.
[X] This is not a new feature request. If it is, please file a feature request instead.
[X] This is not an enhancement request. If it is, please file a improvement suggestion instead.
[X] I have searched on the issue tracker and the lotus forum, and there is no existing related issue or discussion.
[X] I am running the Latest release, or the most recent RC(release canadiate) for the upcoming release or the dev branch(master), or have an issue updating to any of these.
[ ] I did not make any code changes to lotus.

Lotus component

[ ] lotus daemon - chain sync
[ ] lotus miner - mining and block production
[ ] lotus miner/worker - sealing
[ ] lotus miner - proving(WindowPoSt)
[X] lotus miner/market - storage deal
[ ] lotus miner/market - retrieval deal
[ ] lotus miner/market - data transfer
[ ] lotus client
[ ] lotus JSON-RPC API
[ ] lotus message management (mpool)
[ ] Other

Lotus Version

lotus version:
Daemon:  1.13.2+calibnet+git.f9db71e.dirty+api1.4.0
Local: lotus version 1.13.2+calibnet+git.f9db71e.dirty

lotus-miner version:
Daemon:  1.13.2+calibnet+git.f9db71e.dirty+api1.2.0
Local: lotus-miner version 1.13.2+calibnet+git.f9db71e.dirty

Describe the Bug

Hi, I have problem with failed deal on the side of the miner. It failed every time after StorageDealTransferring. I found that it doesn't work if the translated dataCid (from lotus client import) entered in to lotus client deal contains identity instead of blake2b-256, you can look at an example: https://cid.ipfs.io/#bafyaa6qsgafcmalqudsaeigs5eamwvxo6ipa6nw3e375szaxj33hlmzhibu723ajpcg4asicjujaagelucbyabasgafcmalqudsaeib2byxqv5qccaof6fccfccfcfccfccfc

When I tried to make the same deal on another miner, it worked. (You cannot download the data because the deal is still in the StorageDealAwaitingPreCommit state.)

Miner where the deal failed: t01105 (version above) Miner where the deal worked: t028659(unknown version)

When I import a file that has 1GiB or less, the translated cid have blake2b-256 insted of identity, it is shorter and the deal on miner t01105 works.

Deal info from lotus client list-deals -v --show-failed:

Jan 30 13:43:53  bafyreihsasxcvozrewjsqxqn2hlbjthj5b5cjomm4owlyqghbte7k3epue  0       t01105    StorageDealError     N                 N                 baga6ea4seaqlcu5hmz2yid4jkwvp2eqplusoprvyhs2svwokr4dbwfunaexlomq  1.984 GiB  0.0015584754024 FIL   779004    12D3KooWMoahTDLLpdhWJCETqNZCGdtBCAfMnSMKZG3zAuMUrHMo-12D3KooWN4MX4hWHjsJUJi6bKCnmXJMN8jL6ne55nCCTaKV4HLSm-1643468399046691677  Completed       false     deal failed: (State=26) deal data verification failed: proposal CommP doesn't match calculated CommP

Logging Information

N/A

Repo Steps

Create file head -c 1610612736 /dev/urandom > /tmp/1.5GiB.txt
Import file lotus client import /tmp/1.5GiB.txt
Make deal lotus client deal <dataCid> t01105 0.00005 777600
See error lotus client list-deals -v --show-failed

aueam commented 2 years ago

I tried downgrading the miner to v1.13.1 and the problematic deal was successful. So I think the problem is in version v1.13.2.

Reiers commented 2 years ago

Hi @aueam

Are you still having this issue on the latest tag v1.15.0?

*Link to latest tags here.

github-actions[bot] commented 2 years ago

Oops, seems like we needed more information for this issue, please comment with more details or this issue will be closed in 24 hours.

github-actions[bot] commented 2 years ago

This issue was closed because it is missing author input.

margox commented 2 years ago

I am facing the same issue with lotus/lotus-miner v1.14.1, and as @aueam said, the failed deal always with a long data cid, like below:

bafyaa6qsgafcmalqudsaeiazs3ookqahnrckbzkwh2zjak7h5sqat23duzauz43zl4xt3dw2mijaagelucbyabasgafcmalqudsaeigifozhpcni5hngig7gongxjla3yjsxx5cvmuoy3ih4dnnirtnwdmjaaghgukn5waykcqeaegfpucmnwbzaqcaibaaeecx2bgg3am

by the way, this original file size of the failed deal is 2070286383, about 2GB, but I have other files successfully storaged with larger size (4GB) and smaller size (150MB) both.

margox commented 2 years ago

Oops, seems like we needed more information for this issue, please comment with more details or this issue will be closed in 24 hours.

I think @aueam has explained the problem in enough detail :(

aueam commented 2 years ago

Hi @margox and @Reiers! lotus version:

Daemon:  1.15.0+calibnet+git.0ac1bbc.dirty+api1.5.0
Local: lotus version 1.15.0+calibnet+git.0ac1bbc.dirty

The import still generates a long CID. (bafyaa6qsgafcmalqudsaeib3vflzvdia42fit5em2qafw3xcld6g3frdilb5ktt4zm6e7434tajaagelucbyabasgafcmalqudsaeifmrgyqfhwdxfxmelwanezhjbafw4qxtmcy65covvyxpcttbyrw54jaagel2cayaaqkcqeaegeaqcaiabraqcaibaaeecaibaeaai)

I'm still getting a message after starting a deal: Message: deal failed: (State=11) deal data verification failed: proposal CommP doesn't match calculated CommP

It's only a problem on calibnet on my side.

margox commented 2 years ago

Hi @margox and @Reiers! lotus version:
Daemon:  1.15.0+calibnet+git.0ac1bbc.dirty+api1.5.0
Local: lotus version 1.15.0+calibnet+git.0ac1bbc.dirty
The import still generates a long CID. (bafyaa6qsgafcmalqudsaeib3vflzvdia42fit5em2qafw3xcld6g3frdilb5ktt4zm6e7434tajaagelucbyabasgafcmalqudsaeifmrgyqfhwdxfxmelwanezhjbafw4qxtmcy65covvyxpcttbyrw54jaagel2cayaaqkcqeaegeaqcaiabraqcaibaaeecaibaeaai)

I'm still getting a message after starting a deal: Message: deal failed: (State=11) deal data verification failed: proposal CommP doesn't match calculated CommP

It's only a problem on calibnet on my side.

Maybe I should look into the lotus code and find out why long cid generated, I guess it might has something to do wtih the file size? two of my files with size 1934919433 and 2070286383 failed, both nearly 2GB, other files with larger or smaller size are working.

margox commented 2 years ago

Oh god it's hard for me to find out a little needle from the ocean, I give up ~ 😭, but I think it's because there's a little difference between the commp generating functions in lotus client side and market side ?

But I wondering why just us two (maybe three, I found someone else asking the same question in project boost LOL) facing the issue if this is a lotus bug, coz there's no other people storage files larger than 1GB all over the world?🤪

filecoin-project / lotus