Closed mezerotm closed 3 years ago
Reproduceable on v1.9.0 build. The miner actor was freshly generated so it has no storage power.
lotus daemon:
2021-05-19T03:16:44.937+0900 INFO chainstore store/store.go:582 New heaviest tipset! [bafy2bzacedykwbvipf6dv7pwhzbdloec4d3ksuweqjae6ynenvbltbt3g2n3k bafy2bzacec63yoyeuayncubo4uqsm44pysjcyr5urzrjkke7fj76vlpchnq3k bafy2bzacecqwbuctzbahahhwwqd3rir2kmfpki3jhsidu52d7fk6lzvtivcnu bafy2bzaceahdiadb4t27lbwgpmjalliokm24hwpycx4qrg2i2wizpa35gyjoy bafy2bzacecf7m57x7g7b5yko6ehjghe3uig5c7lmjx6bbka6c342meblens42 bafy2bzacebxxn6w6sbpkr62qqipsku3vchez7rwmiais2c7wssuuhvmkd37xo] (height=768513)
2021-05-19T03:16:44.938+0900 INFO chain chain/sync_manager.go:322 worker 0 done; took 4.91923124s
2021-05-19T03:16:47.194+0900 WARN vm vm/runtime.go:145 VM.Call failure: not enough gas: used=46679434, available=47071200, use=718940 (RetCode=7):
github.com/filecoin-project/lotus/chain/vm.(*Runtime).chargeGasInternal
/home/downloads/lotus/chain/vm/runtime.go:575
2021-05-19T03:16:47.194+0900 WARN vm vm/runtime.go:376 vmctx send failed: to: f04, method: 8: ret: [], err: not enough gas: used=46679434, available=47071200, use=718940 (RetCode=7)
2021-05-19T03:16:47.194+0900 WARN vm vm/runtime.go:332 Abortf: failed to submit proof for bulk verification
2021-05-19T03:16:47.194+0900 WARN vm vm/runtime.go:145 VM.Call failure: failed to submit proof for bulk verification (RetCode=7):
github.com/filecoin-project/specs-actors/v4/actors/builtin.RequireSuccess
/root/go/pkg/mod/github.com/filecoin-project/specs-actors/v4@v4.0.0/actors/builtin/shared.go:64
2021-05-19T03:16:47.194+0900 WARN vm vm/vm.go:532 Send actor error {"from": "f3wimvziifiymg34kukmxnwskzdiztbe5yg4ubjtv7bwepr6fprkvwcgfxj73nqvqlv43ctj3fx3dlkh55m7mq", "to": "f0744513", "nonce": 7447, "method": "7", "height": "768513", "error": "failed to submit proof for bulk verification (RetCode=7):\n github.com/filecoin-project/specs-actors/v4/actors/builtin.RequireSuccess\n /root/go/pkg/mod/github.com/filecoin-project/specs-actors/v4@v4.0.0/actors/builtin/shared.go:64"}
2021-05-19T03:18:11.766+0900 ERROR statetree state/statetree.go:253 failed to load state tree: failed to load hamt node: blockstore: block not found
2021-05-19T03:18:11.766+0900 ERROR statetree state/statetree.go:253 failed to load state tree: failed to load hamt node: blockstore: block not found
2021-05-19T03:18:11.766+0900 ERROR statetree state/statetree.go:253 failed to load state tree: failed to load hamt node: blockstore: block not found
2021-05-19T03:18:11.766+0900 ERROR statetree state/statetree.go:253 failed to load state tree: failed to load hamt node: blockstore: block not found
2021-05-19T03:18:33.791+0900 ERROR statetree state/statetree.go:253 failed to load state tree: failed to load hamt node: blockstore: block not found
2021-05-19T03:19:32.889+0900 ERROR statetree state/statetree.go:253 failed to load state tree: failed to load hamt node: blockstore: block not found
2021-05-19T03:20:03.338+0900 ERROR statetree state/statetree.go:253 failed to load state tree: failed to load hamt node: blockstore: block not found
2021-05-19T03:20:20.343+0900 ERROR hello hello/hello.go:113 failed to fetch tipset from peer during hello: doRequest failed for single peer 12D3KooWJCHP6B61BRtTpPVNDM4irARqadrwv4GVCeEsoeoA2jAd:
github.com/filecoin-project/lotus/chain/exchange.(*client).doRequest
/home/downloads/lotus/chain/exchange/client.go:142
2021-05-19T03:24:32.973+0900 ERROR statetree state/statetree.go:253 failed to load state tree: failed to load hamt node: blockstore: block not found
lotus-miner:
2021-05-19T03:24:06.015+0900 WARN miner miner/miner.go:437 mineOne: unexpectedly nil MiningBaseInfo for round 768529, off tipset 768528/{bafy2bzacea2wa4few4vkxjbjto63rizutmsgvsoqxfraj5su5lmzz5zesffcg,bafy2bzacedudiiqi5m475lqscw6glo2isfz3dq4we3ggtbhrrkr2petfnlwzo,bafy2bzaceci44g66l4cmamspfi6awkctgt7zpm3xpirvh77dmmpj6klisu7qk,bafy2bzacea7v6vs7bwaa3mioy66wgjwtjw7hzmk64j3skw26us6csd373dod6,bafy2bzacebgrxm7qdruriz73lxpmjysg6o5aol37h7dsm7pz2klkfpmiorny4}
2021-05-19T03:24:36.039+0900 WARN miner miner/miner.go:437 mineOne: unexpectedly nil MiningBaseInfo for round 768530, off tipset 768529/{bafy2bzaceb5h2xq4hzpfk7fjxquzywqwmpkcc6xmy5wcmbdmf6ren3tiie7iu,bafy2bzacecidlglc6pyyf4ij5a5rhlm2xtskk3xbyy4yqu76gbyp54k2xic3k,bafy2bzacebuoscmqf4u7cx2ukq2rcztpbstrarxjn33ywlml2eqr7drtjtego,bafy2bzacear52b5pujpbfuv4rrv3jrfewsmw3n5nxmr66ra2lzvmhjugn5gni}
2021-05-19T03:25:06.042+0900 WARN miner miner/miner.go:437 mineOne: unexpectedly nil MiningBaseInfo for round 768531, off tipset 768530/{bafy2bzacecoeshsruxblhx6po624tmuuedrhye4m4ewnqrk5rys4bvjvruh4i,bafy2bzacebrxqv6gtnrbkjwsxrlrzbjq6upeugao7e5ewuq72cfjisqm6pelq,bafy2bzacebxhlduemobjqemforj7szs5wp23kgw4zqufvkkl53z46pbmp4sba,bafy2bzaced2tugmgp3zuhxlvnmgvhnmwelc2ow2uhiwcavlsu7ylgmvix3t4i}
2021-05-19T03:25:36.220+0900 WARN miner miner/miner.go:437 mineOne: unexpectedly nil MiningBaseInfo for round 768532, off tipset 768531/{bafy2bzacecsrcxrljinkfm3hi27zne5lnxjcxi72lerv2aiizcsbvrpuka6ua,bafy2bzacedvag4meucew6ca6seeslj375vfje4zczokny2ya7bpwk4efwmxvg,bafy2bzacedzboeamc4ajb7agi5bb43dooxocjodzfhyvs2jhhdfkmi6ideors}
2021-05-19T03:26:06.519+0900 WARN miner miner/miner.go:437 mineOne: unexpectedly nil MiningBaseInfo for round 768533, off tipset 768532/{bafy2bzacecj7qw3tk7pm7dasjxuhnxa3mrwgqwcg6hk6ilurokkd7mlmm6dwi,bafy2bzacea53w6ij73ckhibghnzjjqrbadgss3yu3abc2fgeckrph4ksruqgo,bafy2bzaced7goehjn534q6q2lea5g5smeycdspsb2emo4ensswots2nlz7svy}
2021-05-19T03:26:36.027+0900 WARN miner miner/miner.go:437 mineOne: unexpectedly nil MiningBaseInfo for round 768534, off tipset 768533/{bafy2bzaceaxpybd5oy7vrcwkaubsuaq26ta3lj6i7rhtclzrxqv42fri57elo,bafy2bzacedqkxpa2a673v7gyh7p7l4e533eyobyxepu4iivk6yhawqbzcalgy,bafy2bzacedukzfawg75pvuv4i4w3saufbxy2u3ztr52titgwfsrdu6f76zcrw,bafy2bzacea3u5i24o7aspz3b3445a5siaxoe6vr7rbcbuaid2a7yahienqma2,bafy2bzacecisawu6mn63vbv6khrekbzvytxccpx5alswyzmynwc2jypnm46tw,bafy2bzaceai5xiifzaj72vpcsmkeacymaty5jbymzoscijocxxasmbulsl6lc,bafy2bzacecukchq4llc7vhnbatd45apjwyhzhdvb3aytohgdggicvbhvyzb66,bafy2bzacec4p6ba62bk7t3faxypv2hjipyvlkr3s2e4rkm5nieepvrao32isg,bafy2bzacedmsuw446g7fct4xeljjmyjjz7jddyny2usmszv4so3h2usbjhuca,bafy2bzacebgaspmpi4ha7jxhmmi4pleugpamjwy5kw4o324vt4yeanf5ckpe2}
2021-05-19T03:27:06.401+0900 WARN miner miner/miner.go:437 mineOne: unexpectedly nil MiningBaseInfo for round 768535, off tipset 768534/{bafy2bzacebndndtat54xzj5u6dj2simvqeihipo5loeqaajk4ammxtmzwtujm,bafy2bzacedmtcyrsfj2ke6eiwsjtzonqyi6if3h4hgrw63dm5dhqtxqonuft2,bafy2bzacebwcshivdo5zoqf6fwaxzlfql7imvigfsvvzoc6a3tfyifbxj3h6s,bafy2bzaceagrbnmla6m2jdjq5l2zcpno4d67vsdmqm77nqaikxowqyd5xiubc,bafy2bzacebplprxyn7iwjlktpi4ektlqllwextptuida5wu7yftnpua372nku,bafy2bzaceaiqzajrbtcuokccnabhsvmgkcgqnlevu3o4obawv66semjyzrrr6,bafy2bzacedvf3t4qncxcokydbexnd6dkkljgulwfjukoasyhq6om2wf2kpwxe}
2021-05-19T03:27:36.425+0900 WARN miner miner/miner.go:437 mineOne: unexpectedly nil MiningBaseInfo for round 768536, off tipset 768535/{bafy2bzaceclvrlg74awylet4cjth55wmwhyqbplua36u2a4nl6puplgf2lb2o,bafy2bzacebfc7fohe2xn6pdamgwggyyevfcwp4gwstlehfxz2qhegiyncoxne}
2021-05-19T03:28:06.088+0900 WARN miner miner/miner.go:437 mineOne: unexpectedly nil MiningBaseInfo for round 768537, off tipset 768536/{bafy2bzaceby6u42d2zth6erhqmd4xpt6nvfovzjkza3wl6u2bclwmkjmrp3tq,bafy2bzacebvlsvv7sktfyrqihxy4khbeod2c7r5qzg5je3swwvqfpnh7im3vm,bafy2bzacec2stuwysvfb7qvrgzi5lttnwtq3usivjzgngl4u3ax7oblokq3ek,bafy2bzaceck3crstbr53ana64mgqlhg3k2ibymsb37uou6xjyx3umqcwslsqc,bafy2bzacecart25brlvurlowexvuukrbb62wzqx7idd5inx2fburrv2vjhhuw,bafy2bzacedupde3uzcaqjwxhfrqbbmau4e2htwe2nkvegci75paxx5u7ynmhy,bafy2bzacec45kcorwa5fkrnrt7asz2ascpbcejnmjcagfqmznk4u62dgquxiw,bafy2bzacebemz5n5kc5mztxyffbgzd22qzdiirc5brn3uzcz2rw2up7hvwdx6}
2021-05-19T03:28:36.400+0900 WARN miner miner/miner.go:437 mineOne: unexpectedly nil MiningBaseInfo for round 768538, off tipset 768537/{bafy2bzaceabu6ibjokyervizg6vwaxdxxa22yv7gvmccleysgweirui46mjiq,bafy2bzacebq7duk5wqvmzgvpmseielxes535aijry4j7mxt2dexr7mjr7vr2g,bafy2bzacecmoytgeaneuamm3ob6jqzjiofocpetccffq6nas6b7ga3yqb7mxu,bafy2bzaceban4p7kj45qbj2izrnavw4iytgou4uypr76c2ae7s2a6rgtlcxei}
2021-05-19T03:29:06.137+0900 WARN miner miner/miner.go:437 mineOne: unexpectedly nil MiningBaseInfo for round 768539, off tipset 768538/{bafy2bzacedidlnuugmv2zm7yxxmmsfd4jur2yoi3ias26nntegckwkblq7way,bafy2bzaceczk53i6meohxq5rgg5pivy242z3gqiczl2ib72s7oulcfxtcxfmc,bafy2bzaceam4jdwk4dveplzavrlk5lvhjihrpwcchzgdwbnfzzu7dpiojv2fc,bafy2bzacecpjsbsrfxa36p6kxuoqjfnxh67ul557b3ata2h4rljlntmaewtig,bafy2bzaced4c6kyr43osgbct4ku6sbohw32rubpiq4uo74xjnujyl6u4vcxce,bafy2bzacecxt4smlim4n6klahqv7jyisow2s7qb4fvhztpvj3vgoj7iz5yuyo}
2021-05-19T03:29:36.191+0900 WARN miner miner/miner.go:437 mineOne: unexpectedly nil MiningBaseInfo for round 768540, off tipset 768539/{bafy2bzacecmfsasz5svm5m2r6sa7uecdy2dajapesvgfldozxfsiu6p2sgbym}
2021-05-19T03:30:06.013+0900 WARN miner miner/miner.go:437 mineOne: unexpectedly nil MiningBaseInfo for round 768541, off tipset 768540/{bafy2bzaceb7bclicuwex4ko24comjpbfsapoeemdlkgm3jqwkysxaprb5ogfu,bafy2bzaceao5symilmhoregobdc46airigannv6juour3pws426orevp22rn4,bafy2bzaceaniaedk6gv2kykg3x5yybmypvwi2ck7nkai55skwx7dril4f6dti,bafy2bzacedcdud2skwiq63ahssmwsx7dpxqi6hpk5bpkhfcxtxxqdce454n3w,bafy2bzacedr7eqdgmjvutojsav6nnura5i44mnooqf3r4lt3v4learot4c47s}
@mezerotm @hyunmoon this is my fault, I misunderstood the internal flow. There is no problem aside from the text of the warning being misleading - this is expected when you are penalized for an entire cycle. A PR is coming up to fix this, but please ignore for the time being.
I compiled & installed lotus with tag v1.9.0
, the nerpa testnet.
After creating a 512Mb sector for a day, my lotus-miner sector list
always shows:
ID State OnChain Active Expiration Deals 0 Committing NO NO n/a CC
And I saw following warning in lotus's log:
2021-06-14T12:15:36.678+0800 WARN miner miner/miner.go:429 mineOne: unexpectedly nil MiningBaseInfo for round 252970, off tipset 252969/{bafy2bzaceduumindh3htfg4owirchcchzo6xklvo42lqiphjabl7ad7leiogs,bafy2bzaceaus5qnfpm2rgpx6awxufx5zc66du4yvhnojh2zbwph47ozbrbmgi}
Are the 2 related ? Any suggestion on solve the issue?
Problem
Version
lotus version 1.11.0-dev+mainnet+git.bf10b051f lotus-miner version 1.11.0-dev+mainnet+git.bf10b051f lotus-worker version 1.11.0-dev+mainnet+git.bf10b051f
Setup
CPU: 16 Core AMD Epyc 7281 16c/32t RAM: 256 gb 2666 mhz DDR4 ECC Ram (8x32gb) Boot: 1tb Western Digital NVME SSD GPU: Nvidia GT 710 Motherboard: ASUS KNPA U-16 LAN: 10 gbps WAN: 1gbps down, 50 mbps up RAID1: 2x 8tb Seagate Exos 7e8
notes
lotus-miner actor control list
environment
lotus-miner info
lotus sync wait
lotus-miner sectors status --log 0
~/.lotus/config.toml
~/.lotusminer/config.toml
You miner and daemon setup, including what hardware do you use, your environment variable settings, how do you run your miner and worker, do you use GPU and etc.
Miner Logs
Daemon Logs