search

filecoin-project / lotus

Reference implementation of the Filecoin protocol, written in Go
https://lotus.filecoin.io/
Other
2.84k stars 1.26k forks source link

proof validation failed, sector not found in sector set after cron: sector not found #3355

Closed RobQuistNL closed 2 years ago

RobQuistNL commented 4 years ago

Describe the problem

Sector is stuck in a loop (have 2 of these)

Sectors status

SectorID:   16
Status:     CommitFailed
CIDcommD:   baga6ea4seaqnuflxnoeju5ilz7b4ry3anpavfm4lqxlinhx6d734wrzrq7dswnq
CIDcommR:   bagboea4b5abcakxgvcybzkocifp7vosysppkaltm75l2mo3n6gemecwnfci25sc7
Ticket:     ff8ef989be532ef30d70b495b7e9f1c96dc32356eb7b4db5b8e0bbe326ed73b4
TicketH:    3552
Seed:       3f2286632086d0638b482179a4f59df132a0423f740fa7b2beb15ae05fcbfdb2
SeedH:      5952
Proof:      a180cd8ae1f8cbd93790049aca35d39e84724b4b4d660495e4a569e134a6a4319a3948e8e278d99413754acbc3112bccb07279ba7ab3035f73d5dc4bf4fa4c90bbd3d05579345819594f3741d228b6f06eeb9c44b3bb5fb3a01a7f0d51dc258d0bf7425a719ecc6986878ff9cef637bd22343963338bac97ec10fe1eed8a574ecda565fbd5e37e9af262422cf96a9c3db2e32d12dfebc89836916f08422c61d401b97bbc8890f02da53a495bea18699a973f5179d726811d2e5eb01a0ce82e678dad41c292e0639ab2c2a73eaff6d079b89a2341fc3a7c3401223d32e9eb4357d7183e9c12255cdf98ff70d91a6733eb823e940e50bf9bd8641a4d1260b41b8f4790b92119ae1840e66b822c943b8654d48b913c165ab3803735c971ac8d0ae402bfa362f2d2be506078a53c3822e6072c9c7c2133aff2d320cc7013e78010c8a12f267880dc134c518de5dd498b530481e8180bf5c632e6f864e1a11e885334d034f105815bb7938ef42d9bd3540df74343b4aa1a342bf7d527f94d7d4aa5b68ac0f6d07fbf77cd7dbaf7ec7cc5033f8e56cc69f3a83f687af1e2e30010aff3288828f568da44c9e206c7bac0d908f3a7ab15ab896b25019c3f8c5027de90cfce8fe0eb23f8d6dd21755077ac98e011f89c68e020b3d06a761fd002c88102cd114bbb67cddf15c414b823c64c8c221ba8fa8e0d7c60e01d9349276c35fb293be4555aaacfef2be307cfeab3027682a78c57da5e5b7f385991b427053e9d96706c9874179e72a58bb9465d6667d66093a323a7a5adf2a73f3c102ca259d895ccb4f80520504e423d532d6cfb81ae3f47f151a88982ed5b7da61ed59d0a16c89131ce298d0c2d3c9de34527e5b93b5af58729ead0ed8472508b626c7fc4b5250291b9af8b240d71e78e755c463906579faf9b69f75cfad96331cebb2df385981711b4c5420073fc3e10a51707b1641c8ea89ecf5859edcb647c5f645580557c67c60125453ab9dfa97a1a561424ee09798ef880ac9529faac51460b908e282b047ef5cea15169f60172f12f071472cf49285d3a44694706e2332a694410667f4394b46c8422c4cd799d4c2537a41eea354b945edf301d862b4cf51199e377cd5c128abfba8eebd6aa4e8a7a9c42e8087c87e7116166ee8c27eed8692ad92df8d88d1c2ffef99590d0795b27658c9d2e7f000b903445c5bf4fdb77353c9bb4fb690244a6797e075d4afff411d264fb3005de5f775132523bc60b0bbed5fd17d2e755e9113c9a387846c7c4f8bbce274a049067e9c7641ff8b37f21451f7544b85fa5ac957370e8ecb4c976daedae735af9efe485aecefc251233007cdf2cd8aa4ba7ed5a16358de00abdab60a0780e1e503c18cbe1a529abedab1d6016c67478d0e492456cbfb7ce21c9383d3e65f9cdb8b34b37cad03687bd41cff14fb519a660ffea14b1e65750a752651e61535348d75d8e157108671f2fa0f0170a73b25aea13e20bbef2b9c0526c6a3d66c2c26b3dcdc3194be124f469e54c34d910f65ff0efe55f77a42aa0bbc75daac7313e629caaa7a8ed52825793f3dc095ae0b2509956ecb3d0807f0d3711bd2a0aca88a4b0e7e13e2b3117ae4301bd1c01b3d1f474a5ecb91da6802a6d8fff744e45a132d11589ba15ab51d39973a7fc6ed57dbdcd008e1d32705a2c89921b94790ed1cd23a37f3e9d0ca01605b0f8ca0a93478d9c3b251c6caa848948efbea0200b1fd8c6a827f7111d7830aa53b687689a53b43e171f1c72d307f85d4ac3fb3b38b07fc699dfb8ff35d04921af0b535c28df5971cfc067958b2ac2f306a309e441302640ad291273d47103c793f33180626d9ce409f369380b39c11ccc8e67637150d84830787d5db93e15b6168edc5ad9e8ba46934379a65da43dac93d8910c5e547f6c03b34e6ac4622678d2c0755e1a45fed3dd7825bb5737b56849e448f1b16eabbb827f9783612be0e6f5aad92162c102867378a18c3ad7c3e9afa1c4d2ee51728711c45a4126e0515b364002fba97471b500c8cb833cf29c2e7b77c45edc0c23b14ba2647663f33cd2a63490fdc072db7949bad21b8bd61f1702164ef7092a4e0493e117c75a2d697b93cd2a256ff870896aaa4c111db4043438b07ad74810deb98c44fb191327f78eed8970c555bc58a8b525dac7e824dccd3cf1685f2d28c762b679efbd5366062aa0541556c654111c77ecc6ea3b94416beea1c5007e6191b3ae463c4fa753d5200deb2a834b2d5a284df6ee4e8f5a4db596f7e105d50ea9a89678c8b2aa59ce5d16c6278334c67e7f1168af65e16e94be6655f32e9f2a97c09b5179bfed942e8e59b55264a164daab0b4fbf3e015729cc5d82667565b36c12863ace507cc7d82ee72d92a9c96f6626328f2d1e4b9287aefef6fe045c9c7fd027aab745b287cd5333a7743b1681912bb0494b30f554b516d4747de8ea8731a883ac915933d4b0eb16dfa2e224c39ff08e560e93f4240fdbc7925507260665ce97bc4b671437e1f5207b9af4b40a26a1f6cad10d339e8843caf6d981171c26f087abd964191b9593c3e5220efb9555780bebcbb7c47c9e093fdf569b1349b09fea2470c9092172953d8cfc29c83e646552807140128249a52b58152773d6d15caf7b2dd9802cb6b09e44086c3f99c29e0202b3af82cfeee8e54ecdba5eec198b50c9743cb6ff51503b60b499a17906c8
Deals:      [9459 0 0 0 0]
Retries:    312
--------
Event Log:
0.  2020-08-26 04:36:59 +0000 UTC:  [event;sealing.SectorStart] {"User":{"ID":16,"SectorType":3}}
1.  2020-08-26 05:36:59 +0000 UTC:  [event;sealing.SectorStartPacking]  {"User":{}}
2.  2020-08-26 08:22:34 +0000 UTC:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":2147483648,"PieceCID":{"/":"baga6ea4seaqifnolooacsrnd2x5tcmjd3nzcxd7mpy6ho2eddvy6wa6fdjg2uhq"}},"DealInfo":{"DealID":9459,"DealSchedule":{"StartEpoch":6532,"EndEpoch":709369},"KeepUnsealed":true}}}}
3.  2020-08-26 10:36:02 +0000 UTC:  [event;sealing.SectorStartPacking]  {"User":{}}
4.  2020-08-26 11:06:16 +0000 UTC:  [event;sealing.SectorPacked]    {"User":{"FillerPieces":[{"Size":2147483648,"PieceCID":{"/":"baga6ea4seaqh34u3nf3tdgpi6k2aw54rtucikcpo25uofrzjpmprinydj7b4mla"}},{"Size":4294967296,"PieceCID":{"/":"baga6ea4seaqgntqfunthkuwpixacxtcoqojjdg66vq254l7vmjyyjdu7pntvcby"}},{"Size":8589934592,"PieceCID":{"/":"baga6ea4seaqnqyicdbbfvnpjlmokmi45fgroiigxa2uw6nz6f6ojveoxlhizwai"}},{"Size":17179869184,"PieceCID":{"/":"baga6ea4seaqg2nsld34emra2ljfgrbrdcswmbjdpaftrpzjuipudt3w7qpbikpa"}}]}}
5.  2020-08-26 15:11:04 +0000 UTC:  [event;sealing.SectorPreCommit1]    {"User":{"PreCommit1Out":"eyJyZWdpc3RlcmVkX3Byb29mIjoiU3RhY2tlZERyZzMyR2lCVjEiLCJsYWJlbHMiOnsiU3RhY2tlZERyZzMyR2lCVjEiOnsibGFiZWxzIjpbeyJwYXRoIjoiL21udC9udm1lXzJ0Yl9zbG93L2xvdHVzLWNhbGlicmF0aW9uL2NhY2hlL3MtdDAyMzg4LTE2IiwiaWQiOiJsYXllci0xIiwic2l6ZSI6MTA3Mzc0MTgyNCwicm93c190b19kaXNjYXJkIjo3fSx7InBhdGgiOiIvbW50L252bWVfMnRiX3Nsb3cvbG90dXMtY2FsaWJyYXRpb24vY2FjaGUvcy10MDIzODgtMTYiLCJpZCI6ImxheWVyLTIiLCJzaXplIjoxMDczNzQxODI0LCJyb3dzX3RvX2Rpc2NhcmQiOjd9LHsicGF0aCI6Ii9tbnQvbnZtZV8ydGJfc2xvdy9sb3R1cy1jYWxpYnJhdGlvbi9jYWNoZS9zLXQwMjM4OC0xNiIsImlkIjoibGF5ZXItMyIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lXzJ0Yl9zbG93L2xvdHVzLWNhbGlicmF0aW9uL2NhY2hlL3MtdDAyMzg4LTE2IiwiaWQiOiJsYXllci00Iiwic2l6ZSI6MTA3Mzc0MTgyNCwicm93c190b19kaXNjYXJkIjo3fSx7InBhdGgiOiIvbW50L252bWVfMnRiX3Nsb3cvbG90dXMtY2FsaWJyYXRpb24vY2FjaGUvcy10MDIzODgtMTYiLCJpZCI6ImxheWVyLTUiLCJzaXplIjoxMDczNzQxODI0LCJyb3dzX3RvX2Rpc2NhcmQiOjd9LHsicGF0aCI6Ii9tbnQvbnZtZV8ydGJfc2xvdy9sb3R1cy1jYWxpYnJhdGlvbi9jYWNoZS9zLXQwMjM4OC0xNiIsImlkIjoibGF5ZXItNiIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lXzJ0Yl9zbG93L2xvdHVzLWNhbGlicmF0aW9uL2NhY2hlL3MtdDAyMzg4LTE2IiwiaWQiOiJsYXllci03Iiwic2l6ZSI6MTA3Mzc0MTgyNCwicm93c190b19kaXNjYXJkIjo3fSx7InBhdGgiOiIvbW50L252bWVfMnRiX3Nsb3cvbG90dXMtY2FsaWJyYXRpb24vY2FjaGUvcy10MDIzODgtMTYiLCJpZCI6ImxheWVyLTgiLCJzaXplIjoxMDczNzQxODI0LCJyb3dzX3RvX2Rpc2NhcmQiOjd9LHsicGF0aCI6Ii9tbnQvbnZtZV8ydGJfc2xvdy9sb3R1cy1jYWxpYnJhdGlvbi9jYWNoZS9zLXQwMjM4OC0xNiIsImlkIjoibGF5ZXItOSIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lXzJ0Yl9zbG93L2xvdHVzLWNhbGlicmF0aW9uL2NhY2hlL3MtdDAyMzg4LTE2IiwiaWQiOiJsYXllci0xMCIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lXzJ0Yl9zbG93L2xvdHVzLWNhbGlicmF0aW9uL2NhY2hlL3MtdDAyMzg4LTE2IiwiaWQiOiJsYXllci0xMSIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N31dLCJfaCI6bnVsbH19LCJjb25maWciOnsicGF0aCI6Ii9tbnQvbnZtZV8ydGJfc2xvdy9sb3R1cy1jYWxpYnJhdGlvbi9jYWNoZS9zLXQwMjM4OC0xNiIsImlkIjoidHJlZS1kIiwic2l6ZSI6MjE0NzQ4MzY0Nywicm93c190b19kaXNjYXJkIjo3fSwiY29tbV9kIjpbMjE4LDIxLDExOSwxMDcsMTM2LDE1NCwxMTcsMTEsMjA3LDE5NSwyMDAsMjI3LDk2LDEwNywxOTMsODIsMTc5LDEzOSwxMzMsMjE0LDEzNCwxNTgsMjU0LDMxLDI0NywyMDMsNzEsNDksMTM1LDE5OSw0Myw1NF19","TicketValue":"/475ib5TLvMNcLSVt+nxyW3DI1bre021uOC74ybtc7Q=","TicketEpoch":3552}}
6.  2020-08-26 22:20:34 +0000 UTC:  [event;sealing.SectorPreCommit2]    {"User":{"Sealed":{"/":"bagboea4b5abcakxgvcybzkocifp7vosysppkaltm75l2mo3n6gemecwnfci25sc7"},"Unsealed":{"/":"baga6ea4seaqnuflxnoeju5ilz7b4ry3anpavfm4lqxlinhx6d734wrzrq7dswnq"}}}
7.  2020-08-26 22:20:41 +0000 UTC:  [event;sealing.SectorPreCommitted]  {"User":{"Message":{"/":"bafy2bzacecfd6jgqno3otvronyg5vsfgjuatc4aokkvrycxexrpmnfolxwura"},"PreCommitDeposit":"1085841941932828268","PreCommitInfo":{"SealProof":3,"SectorNumber":16,"SealedCID":{"/":"bagboea4b5abcakxgvcybzkocifp7vosysppkaltm75l2mo3n6gemecwnfci25sc7"},"SealRandEpoch":3552,"DealIDs":[9459],"Expiration":712249,"ReplaceCapacity":false,"ReplaceSectorDeadline":0,"ReplaceSectorPartition":0,"ReplaceSectorNumber":0}}}
8.  2020-08-26 22:24:00 +0000 UTC:  [event;sealing.SectorPreCommitLanded]   {"User":{"TipSet":"AXGg5AIgIS0dL6bQD5SrcrIjKhYpggnmbqmOkiLvLEOqs1q/yzoBcaDkAiBuHw2b0j6EzpnoquSzNvVgwNFe95D7QNAG2YikZG2ZZw=="}}
9.  2020-08-26 23:24:26 +0000 UTC:  [event;sealing.SectorRestart]   {"User":{}}
10. 2020-08-26 23:48:28 +0000 UTC:  [event;sealing.SectorRestart]   {"User":{}}
11. 2020-08-26 23:48:29 +0000 UTC:  [event;sealing.SectorSeedReady] {"User":{"SeedValue":"PyKGYyCG0GOLSCF5pPWd8TKgQj90D6eyvrFa4F/L/bI=","SeedEpoch":5952}}
12. 2020-08-27 07:03:08 +0000 UTC:  [event;sealing.SectorCommitted] {"User":{"Message":{"/":"bafy2bzacecjkir7xzxn77pwzpmj376wzzprdtxlnbtv4zkapizjsa4phgunza"},"Proof":"oYDNiuH4y9k3kASayjXTnoRyS0tNZgSV5KVp4TSmpDGaOUjo4njZlBN1SsvDESvMsHJ5unqzA19z1dxL9PpMkLvT0FV5NFgZWU83QdIotvBu65xEs7tfs6Aafw1R3CWNC/dCWnGezGmGh4/5zvY3vSI0OWMzi6yX7BD+Hu2KV07NpWX71eN+mvJiQiz5apw9suMtEt/ryJg2kW8IQixh1AG5e7yIkPAtpTpJW+oYaZqXP1F51yaBHS5esBoM6C5nja1BwpLgY5qywqc+r/bQebiaI0H8Onw0ASI9MunrQ1fXGD6cEiVc35j/cNkaZzPrgj6UDlC/m9hkGk0SYLQbj0eQuSEZrhhA5muCLJQ7hlTUi5E8FlqzgDc1yXGsjQrkAr+jYvLSvlBgeKU8OCLmByycfCEzr/LTIMxwE+eAEMihLyZ4gNwTTFGN5d1Ji1MEgegYC/XGMub4ZOGhHohTNNA08QWBW7eTjvQtm9NUDfdDQ7SqGjQr99Un+U19SqW2isD20H+/d819uvfsfMUDP45WzGnzqD9oevHi4wAQr/MoiCj1aNpEyeIGx7rA2Qjzp6sVq4lrJQGcP4xQJ96Qz86P4Osj+NbdIXVQd6yY4BH4nGjgILPQanYf0ALIgQLNEUu7Z83fFcQUuCPGTIwiG6j6jg18YOAdk0knbDX7KTvkVVqqz+8r4wfP6rMCdoKnjFfaXlt/OFmRtCcFPp2WcGyYdBeecqWLuUZdZmfWYJOjI6elrfKnPzwQLKJZ2JXMtPgFIFBOQj1TLWz7ga4/R/FRqImC7Vt9ph7VnQoWyJExzimNDC08neNFJ+W5O1r1hynq0O2EclCLYmx/xLUlApG5r4skDXHnjnVcRjkGV5+vm2n3XPrZYzHOuy3zhZgXEbTFQgBz/D4QpRcHsWQcjqiez1hZ7ctkfF9kVYBVfGfGASVFOrnfqXoaVhQk7gl5jviArJUp+qxRRguQjigrBH71zqFRafYBcvEvBxRyz0koXTpEaUcG4jMqaUQQZn9DlLRshCLEzXmdTCU3pB7qNUuUXt8wHYYrTPURmeN3zVwSir+6juvWqk6KepxC6Ah8h+cRYWbujCfu2Gkq2S342I0cL/75lZDQeVsnZYydLn8AC5A0RcW/T9t3NTybtPtpAkSmeX4HXUr/9BHSZPswBd5fd1EyUjvGCwu+1f0X0udV6RE8mjh4RsfE+LvOJ0oEkGfpx2Qf+LN/IUUfdUS4X6WslXNw6Oy0yXba7a5zWvnv5IWuzvwlEjMAfN8s2KpLp+1aFjWN4Aq9q2CgeA4eUDwYy+GlKavtqx1gFsZ0eNDkkkVsv7fOIck4PT5l+c24s0s3ytA2h71Bz/FPtRmmYP/qFLHmV1CnUmUeYVNTSNddjhVxCGcfL6DwFwpzslrqE+ILvvK5wFJsaj1mwsJrPc3DGUvhJPRp5Uw02RD2X/Dv5V93pCqgu8ddqscxPmKcqqeo7VKCV5Pz3Ala4LJQmVbss9CAfw03Eb0qCsqIpLDn4T4rMReuQwG9HAGz0fR0pey5HaaAKm2P/3RORaEy0RWJuhWrUdOZc6f8btV9vc0Ajh0ycFosiZIblHkO0c0jo38+nQygFgWw+MoKk0eNnDslHGyqhIlI776gIAsf2MaoJ/cRHXgwqlO2h2iaU7Q+Fx8cctMH+F1Kw/s7OLB/xpnfuP810EkhrwtTXCjfWXHPwGeViyrC8wajCeRBMCZArSkSc9RxA8eT8zGAYm2c5AnzaTgLOcEczI5nY3FQ2EgweH1duT4VthaO3FrZ6LpGk0N5pl2kPayT2JEMXlR/bAOzTmrEYiZ40sB1XhpF/tPdeCW7Vze1aEnkSPGxbqu7gn+Xg2Er4Ob1qtkhYsEChnN4oYw618Ppr6HE0u5RcocRxFpBJuBRWzZAAvupdHG1AMjLgzzynC57d8Re3AwjsUuiZHZj8zzSpjSQ/cBy23lJutIbi9YfFwIWTvcJKk4Ek+EXx1otaXuTzSolb/hwiWqqTBEdtAQ0OLB610gQ3rmMRPsZEyf3ju2JcMVVvFiotSXax+gk3M088WhfLSjHYrZ5771TZgYqoFQVVsZUERx37MbqO5RBa+6hxQB+YZGzrkY8T6dT1SAN6yqDSy1aKE327k6PWk21lvfhBdUOqaiWeMiyqlnOXRbGJ4M0xn5/EWivZeFulL5mVfMunyqXwJtReb/tlC6OWbVSZKFk2qsLT78+AVcpzF2CZnVls2wShjrOUHzH2C7nLZKpyW9mJjKPLR5Lkoeu/vb+BFycf9AnqrdFsofNUzOndDsWgZErsElLMPVUtRbUdH3o6ocxqIOskVkz1LDrFt+i4iTDn/COVg6T9CQP28eSVQcmBmXOl7xLZxQ34fUge5r0tAomofbK0Q0znohDyvbZgRccJvCHq9lkGRuVk8PlIg77lVV4C+vLt8R8ngk/31abE0mwn+okcMkJIXKVPYz8Kcg+ZGVSgHFAEoJJpStYFSdz1tFcr3st2YAstrCeRAhsP5nCngICs6+Cz+7o5U7Nul7sGYtQyXQ8tv9RUDtgtJmheQbI"}}
13. 2020-08-27 07:06:29 +0000 UTC:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron: sector not found
14. 2020-08-27 07:07:29 +0000 UTC:  [event;sealing.SectorRetryComputeProof] {"User":{}}
15. 2020-08-27 07:07:29 +0000 UTC:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
16. 2020-08-27 07:07:29 +0000 UTC:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron: sector not found
17. 2020-08-27 07:08:29 +0000 UTC:  [event;sealing.SectorRetryComputeProof] {"User":{}}
18. 2020-08-27 07:08:29 +0000 UTC:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
19. 2020-08-27 07:08:29 +0000 UTC:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron: sector not found
20. 2020-08-27 07:09:29 +0000 UTC:  [event;sealing.SectorRetryComputeProof] {"User":{}}
21. 2020-08-27 07:09:29 +0000 UTC:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
22. 2020-08-27 07:09:29 +0000 UTC:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron: sector not found
(.... more of the same ....)
    proof validation failed, sector not found in sector set after cron: sector not found
941.    2020-08-27 13:22:03 +0000 UTC:  [event;sealing.SectorRetryComputeProof] {"User":{}}
942.    2020-08-27 13:22:17 +0000 UTC:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
943.    2020-08-27 13:22:30 +0000 UTC:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron: sector not found
944.    2020-08-27 13:23:30 +0000 UTC:  [event;sealing.SectorRetryComputeProof] {"User":{}}
945.    2020-08-27 13:23:44 +0000 UTC:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
946.    2020-08-27 13:23:58 +0000 UTC:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron: sector not found
947.    2020-08-27 13:24:58 +0000 UTC:  [event;sealing.SectorRetryComputeProof] {"User":{}}
948.    2020-08-27 13:25:13 +0000 UTC:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
949.    2020-08-27 13:25:28 +0000 UTC:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron: sector not found

Lotus miner logs

2020-08-27T13:26:55.074Z    WARN    sectors storage-sealing/fsm.go:403  sector 16 got error event sealing.SectorCommitFailed: proof validation failed, sector not found in sector set after cron: sector not found
2020-08-27T13:26:55.088 INFO filcrypto::proofs::api > verify_seal: start
2020-08-27T13:26:55.088 INFO filecoin_proofs::api::seal > verify_seal:start
2020-08-27T13:26:55.088 INFO filecoin_proofs::caches > trying parameters memory cache for: STACKED[34359738368]-verifying-key
2020-08-27T13:26:55.088 INFO filecoin_proofs::caches > found params in memory cache for STACKED[34359738368]-verifying-key
2020-08-27T13:26:55.088 INFO filecoin_proofs::api::seal > got verifying key (34359738368) while verifying seal
2020-08-27T13:26:55.121 INFO filecoin_proofs::api::seal > verify_seal:finish
2020-08-27T13:26:55.121 INFO filcrypto::proofs::api > verify_seal: finish
2020-08-27T13:26:55.121Z    INFO    sectors storage-sealing/states_failed.go:19 CommitFailed(16), waiting 59.878920207s before retrying

Lotus miner diagnostic info

Please collect the following diagnostic information, and share a link here

Code modifications

None

Version

lotus version 0.5.4+git.d4fef1b5

Meatball13 commented 4 years ago

I just got the same thing in one of mine after a pretty long/drawn out sector cycle: 

lotus-miner sectors status --log 3
SectorID:   3
Status:     CommitFailed
CIDcommD:   baga6ea4seaql357mkwihdscxyjr7oaaj5flcuw5kpznfspiv4z3mcdw6vntygna
CIDcommR:   bagboea4b5abcalwlsn7jr3ry7izlsvbuthz5ssfaql3xd74cd465tn2w4rnlxuix
Ticket:     63d55112f71ea2af8ca108082232a131d7d4401522c0c1e9fe5678d530554550
TicketH:    5949
Seed:       42d3e95af2b34e8a01c77d4b69466348b044ce25cfe6e588199b6f1e6d8ecd93
SeedH:      8087
Proof:      b71f889443168883ee2ff00de84689621d4ddc465bfa95e6c4777ec8eef4f6e757b264c6c5bc94349994a8581efbc3978e22d6014d9f721e584920b5fca0511395bddbf8461e30ac82c19e14677b37303ccbfd581de594e17ec849594d66222507ee0cc0fa09f978698eca4f029cf878b50a0e68fc4ba5a6d5a6baa419e3702effc3f49118c02fc8cf51407f5d7bef6387c5cc68fcb0ba990580cbc2463b446c1069088c99416afede1774bc9f0fc86de843825d3c92c413ee0f89e833597d5db58e77eec0a38b9bd9cfeddabd889e93c48de447724ef7f2aa068992c2e5dc8e025a17638ebcefeb405249300803aebbb33f4651024d6d035196f27d0a2b08cfd857c01026d0e5fc9105f5afdbc3a7735ec5abc2d0f53f601849d394e487487d0a44334d691fd99eeb94727485d01c934ba0508351d98d51a50b5857a526e1595cb25c8cb37c40fcfaf0ca3b39d7bf228b44d4a526e1e530fb92048d3cdaadd72708b49d1aa2237517bc796c6995a97c79d5b237ef0a95a42b18bff32b7bb4f6824f75110831092d707514151698062ca5be5a69b46b06522388bae44571e6b0d659cdc50b17ae4492f760b4ab59d2dbad8b332e532add2412959fdc4050dd36b1e9b373d69e7671a1ee5ba6bd6324710447e7bca388022cdca42f75e5bf8aa211c1a6609ba3eebcb6ddb08341c9f528640b50e64eede9bc1cc6d00eb375998b4f55cf42604f83c3da3b3835cdb06b88b7b4401e2e997c451917f795383425a5aba3e2f88be2050382dbd2430c34a5c1c0e4edda3fd07a60b0acbc5bbcba62c285a9f3c270a2156a7be7e48bed496ab479fc86d0b43cabad270d97a2cb34d1231343ebf801cc9994bfe30ebe2e4fc8b58b8d2899acf9dcd3b01bbeca73273c7b5b0940905521e74364329e786a47818330c9d4961220ea3ebf5805dd0f1d524b072632064f7354628d866709b3a8c8ef1156e57f81dca031fd451bc7d2caf90fbc909befcd8d5264107eb98148fdf123ad7bc60b903fb7a777981f5210d7688edbb151b1aab6772c87eb2fec3483604ac29c6810109007fe341b0bcd6b1a8d3589672062f2d5c9f7592e37893dc65c86d00f8e8b9c8acbf7a9654e804ce2e283ece833713696fa691a17aecec677caeca189182867bd3688a4c7d13d7113b77a182c7eb71029ef4c166243fed0bccd90a485f2c74cda9046bbdfd1051ff76ee60beea293fe9f17dc65655796834f2d2852254c754e5dc39b3f6c27528f27864f33fa73689433509a89795051a410cb799576fe8502b8eb965d8ca3dd25b9613c9dfd402db47debbc9b4f4c9be6675681c2b3655d9f7471e1277f141b5d08c0d6a4e791a77fa3090d63e0dc71f8c47f35a7622b29653e47799988c1a6b6442daf39fd970d31159ee9633f38e8a42e2baca661b812b3ce8b883d2182fedf7d1a7fb6dcc17bdecc3d709ddd95ca0dea292abe9d9e3c373d4fc847ca132953bf8669136925b75189d37266e25b2c1d5b1299fdb457726989a3edacff9e533abb8875bfc943dba89e5c6547d54d9c8179a672b670e96dbcca995c54f2e9ba326aee70c11fca17c99fa757b181af2de6378ed85e3beb0f34f3dbd0e89c28dde2c4b5f1b5997a37fd129e220ffadc6658ffeb6882eb6a68e04088eb985f65fceea6c88a8ff1bc0749cee0dd0088533033aecdb5add960f0219f1eaac2addd6bd705e7e4cfa44cfe9234effe624b28fb82428b76de453fc4f4938166d3492d26baca5ee813c462797f59122a74d4541a920883a5d8a8f7d7895ad2b264b227e96ae080cc2627f253629358d3d9d1e30cb90c1bed9792d98da1ecbe8ed3101f80b28196b121c69b43f2556aefebc48acfe60e6cbe9bea522a803934ad7945bf94931c72fd83834a225f6372c342db950ba7cfb9e8fb8afb9f0392b98019d1ee8a71f81cd7059cd0edbc439397a22ddf8121a19c4782e30a2dcdcce46c02ffa9e32530d1d0fc40422768a6d2bcbdc62de1cc319a16155b2b4418689152d71016d8f8d2c72919a936fe6f3deace0a92d3e3292cacabcfa7bef4e8fad69e3915e5762af07f52d0194d069dcff86bf3228e6179afee1987a2757850e39f84a1ddeed3094959f9e09e8734fa1382b2d37a617496a69114d8cbcaa046f7ee277b270d215a2b5bc890bb1af78360540c43ca1c253d35a017bf89447eb6b88ef64cf8ef5ff2e1dcc3e225d3b18ad7dcf712ad8d67fe98a28ba2c7addd158cf93d206ce997aa05a1d499259e4c7ef173bc1e7f1466a3464f277938074526122239817d3ea8ace37b5b154442e7656b181407ff42d58269b7b1c028dd4bc14023947b609fd7d320413a3b12f3928643e2fc29047d913bf60d088411321aa655b9223e6d9672d3b43f49588cb51e669e4c72d8f270328fc4f474cdceb37f922ddcbd6eadc1240af312ec802c73c9a1a649049ff5bdfad3acb00c1cacb11a69c4e03ece517886db6b8c1de1d89aa2da391ea0b84882fd695f0c10b1f82c46fba47c78d53b47f4876052aed7a851e0ccdd250f9fdd2af83982ab7a86c847457e0741ad5c9669d1e73d31c1169980fe5565d961c01de6fa881391be4314e40181ccc41d7f8d234787086f4d280946a953ca1a151774de50a5decc70a3f9010c104e2624f7b4c4cb5571d1f4860a0f191cbacd4da359defb3df05e8998304a143f5223577b46f70fd8478a6c
Deals:      [12329 0 0 0 0 0 0 0 0 0 0 0 0 0 0 12590 12939 13365 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
Retries:    1
--------
Event Log:
0.  2020-08-26 12:44:00 -0400 EDT:  [event;sealing.SectorStart] {"User":{"ID":3,"SectorType":3}}
1.  2020-08-26 12:44:00 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":131072,"PieceCID":{"/":"baga6ea4seaqaccyfydszdxzmtcda5wirzycykkesjrfwqigedg4j2ztmcje4eja"}},"DealInfo":{"DealID":12329,"DealSchedule":{"StartEpoch":8000,"EndEpoch":709606},"KeepUnsealed":true}}}}
2.  2020-08-26 14:00:59 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":131072,"PieceCID":{"/":"baga6ea4seaqi5hreap5iqthwen7wbxzf7a7oidokt3mht23pmnjncuee6wwq2py"}},"DealInfo":null}}}
3.  2020-08-26 14:00:59 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":262144,"PieceCID":{"/":"baga6ea4seaqhklmwsp5bm5jehfkhnyyxvgcyb4aji6x3piyfidlclkjjdtasuby"}},"DealInfo":null}}}
4.  2020-08-26 14:00:59 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":524288,"PieceCID":{"/":"baga6ea4seaqhaixwb57pnlp2c4ixuutbtyym5kbmnadvvxy4mz3yn3cqn3xs2gi"}},"DealInfo":null}}}
5.  2020-08-26 14:00:59 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":1048576,"PieceCID":{"/":"baga6ea4seaqntgehxfzvoouw4ejzgzcsg3axwh2moa2noi6hvgpxbg5u3jqrmky"}},"DealInfo":null}}}
6.  2020-08-26 14:00:59 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":2097152,"PieceCID":{"/":"baga6ea4seaqnbnjq3oylj4s4luxsukg752aiwu2bfibjghyyysm7lisubbvrgjq"}},"DealInfo":null}}}
7.  2020-08-26 14:01:00 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":4194304,"PieceCID":{"/":"baga6ea4seaqijqccdoqgqwqbx54vui2eazh6ijf5kku5eq3xwokp6tclivuoqei"}},"DealInfo":null}}}
8.  2020-08-26 14:01:00 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":8388608,"PieceCID":{"/":"baga6ea4seaqgl4u6lwmnerwdrm4iz7ag3mpwwaqtapc2fciabpooqmvjypweeha"}},"DealInfo":null}}}
9.  2020-08-26 14:01:00 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":16777216,"PieceCID":{"/":"baga6ea4seaqkejdvbaufquewln7dgszre6ymaqvr2bdnyvcaee3we7gypgoocoq"}},"DealInfo":null}}}
10. 2020-08-26 14:01:01 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":33554432,"PieceCID":{"/":"baga6ea4seaqnv7nlnwutmrctyjwtg4tlt7x6gq56r6awjhwabgvnh6x7kbqxkca"}},"DealInfo":null}}}
11. 2020-08-26 14:01:02 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":67108864,"PieceCID":{"/":"baga6ea4seaqnsqov4dldcsuzlqz77pkpxzurddlt2ts72lgtd4hxzbxl3ukoobq"}},"DealInfo":null}}}
12. 2020-08-26 14:01:04 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":134217728,"PieceCID":{"/":"baga6ea4seaqfctcdlq6qju2juu3f7pkz77drgyurcf4fteobupctv4rapf2buly"}},"DealInfo":null}}}
13. 2020-08-26 14:01:07 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":268435456,"PieceCID":{"/":"baga6ea4seaqk2bufhfu5g7ju74eobh2wsmfevum2rhppmdf75z7b2m4byhtryny"}},"DealInfo":null}}}
14. 2020-08-26 14:01:15 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":536870912,"PieceCID":{"/":"baga6ea4seaqdsvqopmj2soyhujb72jza76t4wpq5fzifvm3ctz47iyytkewnubq"}},"DealInfo":null}}}
15. 2020-08-26 14:01:32 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":1073741824,"PieceCID":{"/":"baga6ea4seaqmzq6acl23axubdiv37xipnaz3qqtvwr57ekoaauviescphqnfwpi"}},"DealInfo":null}}}
16. 2020-08-26 14:02:16 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":2147483648,"PieceCID":{"/":"baga6ea4seaql277a6jgfbtlo6i7tlupr7ylncrvrnw7di2gquhnwudml7ml62bi"}},"DealInfo":{"DealID":12590,"DealSchedule":{"StartEpoch":8150,"EndEpoch":709606},"KeepUnsealed":true}}}}
17. 2020-08-26 16:55:27 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":134217728,"PieceCID":{"/":"baga6ea4seaqbgvql2eqbcavfeifmz7r3a6vmihmoy2435sk2j5h5k7y3glo7kby"}},"DealInfo":{"DealID":12939,"DealSchedule":{"StartEpoch":8341,"EndEpoch":709606},"KeepUnsealed":true}}}}
18. 2020-08-26 17:00:00 -0400 EDT:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":131072,"PieceCID":{"/":"baga6ea4seaqdntccoxi5hksmlv4lqssyt6z54gvp3nojryg67woyctpufng2kha"}},"DealInfo":{"DealID":13365,"DealSchedule":{"StartEpoch":8512,"EndEpoch":712486},"KeepUnsealed":true}}}}
19. 2020-08-26 17:05:53 -0400 EDT:  [event;sealing.SectorStartPacking]  {"User":{}}
20. 2020-08-26 17:14:35 -0400 EDT:  [event;sealing.SectorPacked]    {"User":{"FillerPieces":[{"Size":131072,"PieceCID":{"/":"baga6ea4seaqi5hreap5iqthwen7wbxzf7a7oidokt3mht23pmnjncuee6wwq2py"}},{"Size":262144,"PieceCID":{"/":"baga6ea4seaqhklmwsp5bm5jehfkhnyyxvgcyb4aji6x3piyfidlclkjjdtasuby"}},{"Size":524288,"PieceCID":{"/":"baga6ea4seaqhaixwb57pnlp2c4ixuutbtyym5kbmnadvvxy4mz3yn3cqn3xs2gi"}},{"Size":1048576,"PieceCID":{"/":"baga6ea4seaqntgehxfzvoouw4ejzgzcsg3axwh2moa2noi6hvgpxbg5u3jqrmky"}},{"Size":2097152,"PieceCID":{"/":"baga6ea4seaqnbnjq3oylj4s4luxsukg752aiwu2bfibjghyyysm7lisubbvrgjq"}},{"Size":4194304,"PieceCID":{"/":"baga6ea4seaqijqccdoqgqwqbx54vui2eazh6ijf5kku5eq3xwokp6tclivuoqei"}},{"Size":8388608,"PieceCID":{"/":"baga6ea4seaqgl4u6lwmnerwdrm4iz7ag3mpwwaqtapc2fciabpooqmvjypweeha"}},{"Size":16777216,"PieceCID":{"/":"baga6ea4seaqkejdvbaufquewln7dgszre6ymaqvr2bdnyvcaee3we7gypgoocoq"}},{"Size":33554432,"PieceCID":{"/":"baga6ea4seaqnv7nlnwutmrctyjwtg4tlt7x6gq56r6awjhwabgvnh6x7kbqxkca"}},{"Size":67108864,"PieceCID":{"/":"baga6ea4seaqnsqov4dldcsuzlqz77pkpxzurddlt2ts72lgtd4hxzbxl3ukoobq"}},{"Size":268435456,"PieceCID":{"/":"baga6ea4seaqk2bufhfu5g7ju74eobh2wsmfevum2rhppmdf75z7b2m4byhtryny"}},{"Size":536870912,"PieceCID":{"/":"baga6ea4seaqdsvqopmj2soyhujb72jza76t4wpq5fzifvm3ctz47iyytkewnubq"}},{"Size":1073741824,"PieceCID":{"/":"baga6ea4seaqmzq6acl23axubdiv37xipnaz3qqtvwr57ekoaauviescphqnfwpi"}},{"Size":2147483648,"PieceCID":{"/":"baga6ea4seaqh34u3nf3tdgpi6k2aw54rtucikcpo25uofrzjpmprinydj7b4mla"}},{"Size":8589934592,"PieceCID":{"/":"baga6ea4seaqnqyicdbbfvnpjlmokmi45fgroiigxa2uw6nz6f6ojveoxlhizwai"}},{"Size":17179869184,"PieceCID":{"/":"baga6ea4seaqg2nsld34emra2ljfgrbrdcswmbjdpaftrpzjuipudt3w7qpbikpa"}}]}}
21. 2020-08-27 02:24:04 -0400 EDT:  [event;sealing.SectorRestart]   {"User":{}}
22. 2020-08-27 02:44:04 -0400 EDT:  [event;sealing.SectorRestart]   {"User":{}}
23. 2020-08-27 02:45:10 -0400 EDT:  [event;sealing.SectorSealPreCommit1Failed]  {"User":{}}
    seal pre commit(1) failed: failed to acquire sector {8568 3} from remote(1): sector not found
24. 2020-08-27 02:48:04 -0400 EDT:  [event;sealing.SectorRestart]   {"User":{}}
25. 2020-08-27 02:49:04 -0400 EDT:  [event;sealing.SectorRetrySealPreCommit1]   {"User":{}}
26. 2020-08-27 02:49:10 -0400 EDT:  [event;sealing.SectorSealPreCommit1Failed]  {"User":{}}
    seal pre commit(1) failed: failed to acquire sector {8568 3} from remote(1): sector not found
27. 2020-08-27 02:50:10 -0400 EDT:  [event;sealing.SectorRetrySealPreCommit1]   {"User":{}}
28. 2020-08-27 03:01:56 -0400 EDT:  [event;sealing.SectorSealPreCommit1Failed]  {"User":{}}
    seal pre commit(1) failed: handler: websocket connection closed
29. 2020-08-27 03:02:56 -0400 EDT:  [event;sealing.SectorRetrySealPreCommit1]   {"User":{}}
30. 2020-08-27 03:02:56 -0400 EDT:  [event;sealing.SectorSealPreCommit1Failed]  {"User":{}}
    seal pre commit(1) failed: failed to acquire sector {8568 3} from remote (tried [{f440f65b-7f69-40cd-85f7-70140f47496f [http://172.16.0.2:3456/remote/unsealed/s-t08568-3] 10 true false false}]): 1 error occurred:
    * fetch error http://172.16.0.2:3456/remote/unsealed/s-t08568-3 (storage f440f65b-7f69-40cd-85f7-70140f47496f) -> /home/tbtminer/worker/unsealed/fetching/s-t08568-3: do request: Get "http://172.16.0.2:3456/remote/unsealed/s-t08568-3": dial tcp 172.16.0.2:3456: connect: connection refused

31. 2020-08-27 03:03:56 -0400 EDT:  [event;sealing.SectorRetrySealPreCommit1]   {"User":{}}
32. 2020-08-27 03:03:56 -0400 EDT:  [event;sealing.SectorSealPreCommit1Failed]  {"User":{}}
    seal pre commit(1) failed: failed to acquire sector {8568 3} from remote (tried [{f440f65b-7f69-40cd-85f7-70140f47496f [http://172.16.0.2:3456/remote/unsealed/s-t08568-3] 10 true false false}]): 1 error occurred:
    * fetch error http://172.16.0.2:3456/remote/unsealed/s-t08568-3 (storage f440f65b-7f69-40cd-85f7-70140f47496f) -> /home/tbtminer/worker/unsealed/fetching/s-t08568-3: do request: Get "http://172.16.0.2:3456/remote/unsealed/s-t08568-3": dial tcp 172.16.0.2:3456: connect: connection refused

33. 2020-08-27 03:04:56 -0400 EDT:  [event;sealing.SectorRetrySealPreCommit1]   {"User":{}}
34. 2020-08-27 07:34:08 -0400 EDT:  [event;sealing.SectorPreCommit1]    {"User":{"PreCommit1Out":"eyJyZWdpc3RlcmVkX3Byb29mIjoiU3RhY2tlZERyZzMyR2lCVjEiLCJsYWJlbHMiOnsiU3RhY2tlZERyZzMyR2lCVjEiOnsibGFiZWxzIjpbeyJwYXRoIjoiL2hvbWUvdGJ0bWluZXIvd29ya2VyL2NhY2hlL3MtdDA4NTY4LTMiLCJpZCI6ImxheWVyLTEiLCJzaXplIjoxMDczNzQxODI0LCJyb3dzX3RvX2Rpc2NhcmQiOjd9LHsicGF0aCI6Ii9ob21lL3RidG1pbmVyL3dvcmtlci9jYWNoZS9zLXQwODU2OC0zIiwiaWQiOiJsYXllci0yIiwic2l6ZSI6MTA3Mzc0MTgyNCwicm93c190b19kaXNjYXJkIjo3fSx7InBhdGgiOiIvaG9tZS90YnRtaW5lci93b3JrZXIvY2FjaGUvcy10MDg1NjgtMyIsImlkIjoibGF5ZXItMyIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL2hvbWUvdGJ0bWluZXIvd29ya2VyL2NhY2hlL3MtdDA4NTY4LTMiLCJpZCI6ImxheWVyLTQiLCJzaXplIjoxMDczNzQxODI0LCJyb3dzX3RvX2Rpc2NhcmQiOjd9LHsicGF0aCI6Ii9ob21lL3RidG1pbmVyL3dvcmtlci9jYWNoZS9zLXQwODU2OC0zIiwiaWQiOiJsYXllci01Iiwic2l6ZSI6MTA3Mzc0MTgyNCwicm93c190b19kaXNjYXJkIjo3fSx7InBhdGgiOiIvaG9tZS90YnRtaW5lci93b3JrZXIvY2FjaGUvcy10MDg1NjgtMyIsImlkIjoibGF5ZXItNiIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL2hvbWUvdGJ0bWluZXIvd29ya2VyL2NhY2hlL3MtdDA4NTY4LTMiLCJpZCI6ImxheWVyLTciLCJzaXplIjoxMDczNzQxODI0LCJyb3dzX3RvX2Rpc2NhcmQiOjd9LHsicGF0aCI6Ii9ob21lL3RidG1pbmVyL3dvcmtlci9jYWNoZS9zLXQwODU2OC0zIiwiaWQiOiJsYXllci04Iiwic2l6ZSI6MTA3Mzc0MTgyNCwicm93c190b19kaXNjYXJkIjo3fSx7InBhdGgiOiIvaG9tZS90YnRtaW5lci93b3JrZXIvY2FjaGUvcy10MDg1NjgtMyIsImlkIjoibGF5ZXItOSIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL2hvbWUvdGJ0bWluZXIvd29ya2VyL2NhY2hlL3MtdDA4NTY4LTMiLCJpZCI6ImxheWVyLTEwIiwic2l6ZSI6MTA3Mzc0MTgyNCwicm93c190b19kaXNjYXJkIjo3fSx7InBhdGgiOiIvaG9tZS90YnRtaW5lci93b3JrZXIvY2FjaGUvcy10MDg1NjgtMyIsImlkIjoibGF5ZXItMTEiLCJzaXplIjoxMDczNzQxODI0LCJyb3dzX3RvX2Rpc2NhcmQiOjd9XSwiX2giOm51bGx9fSwiY29uZmlnIjp7InBhdGgiOiIvaG9tZS90YnRtaW5lci93b3JrZXIvY2FjaGUvcy10MDg1NjgtMyIsImlkIjoidHJlZS1kIiwic2l6ZSI6MjE0NzQ4MzY0Nywicm93c190b19kaXNjYXJkIjo3fSwiY29tbV9kIjpbMTg5LDI0NywyMzYsODUsMTQ0LDExMywyMDAsODcsMTk0LDk5LDI0NywwLDksMjMzLDg2LDQyLDkxLDE3MCwxMjYsOTAsODksNjEsMjEsMjMwLDExOCwxOTMsMTQsMjIyLDE3MSwxMDMsMTMxLDUyXX0=","TicketValue":"Y9VREvceoq+MoQgIIjKhMdfUQBUiwMHp/lZ41TBVRVA=","TicketEpoch":5949}}
35. 2020-08-27 12:08:10 -0400 EDT:  [event;sealing.SectorPreCommit2]    {"User":{"Sealed":{"/":"bagboea4b5abcalwlsn7jr3ry7izlsvbuthz5ssfaql3xd74cd465tn2w4rnlxuix"},"Unsealed":{"/":"baga6ea4seaql357mkwihdscxyjr7oaaj5flcuw5kpznfspiv4z3mcdw6vntygna"}}}
36. 2020-08-27 12:08:10 -0400 EDT:  [event;sealing.SectorPreCommitted]  {"User":{"Message":{"/":"bafy2bzacebfxokwwhuxv5dwhgzzcajm6q7vv5hncvosnw25ptfzytkrrn5j4u"},"PreCommitDeposit":"941964936757896146","PreCommitInfo":{"SealProof":3,"SectorNumber":3,"SealedCID":{"/":"bagboea4b5abcalwlsn7jr3ry7izlsvbuthz5ssfaql3xd74cd465tn2w4rnlxuix"},"SealRandEpoch":5949,"DealIDs":[12329,12590,12939,13365],"Expiration":715366,"ReplaceCapacity":false,"ReplaceSectorDeadline":0,"ReplaceSectorPartition":0,"ReplaceSectorNumber":0}}}
37. 2020-08-27 12:11:29 -0400 EDT:  [event;sealing.SectorPreCommitLanded]   {"User":{"TipSet":"AXGg5AIgzWMPjcQjsCA1j1mTvZoC3h3XaiQS9MtlQDswkLEDSakBcaDkAiBQC/ORpA6DTqOCzdo+9ACO/2S/4iKneeU9t1zn3BIlbA=="}}
38. 2020-08-27 13:26:30 -0400 EDT:  [event;sealing.SectorSeedReady] {"User":{"SeedValue":"QtPpWvKzTooBx31LaUZjSLBEziXP5uWIGZtvHm2OzZM=","SeedEpoch":8087}}
39. 2020-08-27 14:53:41 -0400 EDT:  [event;sealing.SectorRestart]   {"User":{}}
40. 2020-08-27 16:46:03 -0400 EDT:  [event;sealing.SectorCommitted] {"User":{"Message":{"/":"bafy2bzaceb67yu4cjrtb4i2fwkaep7va7haynx6qc5cuvzl4jgb5it2gtcaxm"},"Proof":"tx+IlEMWiIPuL/AN6EaJYh1N3EZb+pXmxHd+yO709udXsmTGxbyUNJmUqFge+8OXjiLWAU2fch5YSSC1/KBRE5W92/hGHjCsgsGeFGd7NzA8y/1YHeWU4X7ISVlNZiIlB+4MwPoJ+XhpjspPApz4eLUKDmj8S6Wm1aa6pBnjcC7/w/SRGMAvyM9RQH9de+9jh8XMaPywupkFgMvCRjtEbBBpCIyZQWr+3hd0vJ8PyG3oQ4JdPJLEE+4PiegzWX1dtY537sCji5vZz+3avYiek8SN5EdyTvfyqgaJksLl3I4CWhdjjrzv60BSSTAIA667sz9GUQJNbQNRlvJ9CisIz9hXwBAm0OX8kQX1r9vDp3NexavC0PU/YBhJ05Tkh0h9CkQzTWkf2Z7rlHJ0hdAck0ugUINR2Y1RpQtYV6Um4VlcslyMs3xA/Prwyjs5178ii0TUpSbh5TD7kgSNPNqt1ycItJ0aoiN1F7x5bGmVqXx51bI37wqVpCsYv/Mre7T2gk91EQgxCS1wdRQVFpgGLKW+Wmm0awZSI4i65EVx5rDWWc3FCxeuRJL3YLSrWdLbrYszLlMq3SQSlZ/cQFDdNrHps3PWnnZxoe5bpr1jJHEER+e8o4gCLNykL3Xlv4qiEcGmYJuj7ry23bCDQcn1KGQLUOZO7em8HMbQDrN1mYtPVc9CYE+Dw9o7ODXNsGuIt7RAHi6ZfEUZF/eVODQlpauj4viL4gUDgtvSQww0pcHA5O3aP9B6YLCsvFu8umLChanzwnCiFWp75+SL7UlqtHn8htC0PKutJw2Xoss00SMTQ+v4AcyZlL/jDr4uT8i1i40omaz53NOwG77Kcyc8e1sJQJBVIedDZDKeeGpHgYMwydSWEiDqPr9YBd0PHVJLByYyBk9zVGKNhmcJs6jI7xFW5X+B3KAx/UUbx9LK+Q+8kJvvzY1SZBB+uYFI/fEjrXvGC5A/t6d3mB9SENdojtuxUbGqtncsh+sv7DSDYErCnGgQEJAH/jQbC81rGo01iWcgYvLVyfdZLjeJPcZchtAPjoucisv3qWVOgEzi4oPs6DNxNpb6aRoXrs7Gd8rsoYkYKGe9Noikx9E9cRO3ehgsfrcQKe9MFmJD/tC8zZCkhfLHTNqQRrvf0QUf927mC+6ik/6fF9xlZVeWg08tKFIlTHVOXcObP2wnUo8nhk8z+nNolDNQmol5UFGkEMt5lXb+hQK465ZdjKPdJblhPJ39QC20feu8m09Mm+ZnVoHCs2Vdn3Rx4Sd/FBtdCMDWpOeRp3+jCQ1j4Nxx+MR/NadiKyllPkd5mYjBprZELa85/ZcNMRWe6WM/OOikLiuspmG4ErPOi4g9IYL+330af7bcwXvezD1wnd2Vyg3qKSq+nZ48Nz1PyEfKEylTv4ZpE2klt1GJ03Jm4lssHVsSmf20V3JpiaPtrP+eUzq7iHW/yUPbqJ5cZUfVTZyBeaZytnDpbbzKmVxU8um6MmrucMEfyhfJn6dXsYGvLeY3jtheO+sPNPPb0OicKN3ixLXxtZl6N/0SniIP+txmWP/raILramjgQIjrmF9l/O6myIqP8bwHSc7g3QCIUzAzrs21rdlg8CGfHqrCrd1r1wXn5M+kTP6SNO/+Ykso+4JCi3beRT/E9JOBZtNJLSa6yl7oE8RieX9ZEip01FQakgiDpdio99eJWtKyZLIn6WrggMwmJ/JTYpNY09nR4wy5DBvtl5LZjaHsvo7TEB+AsoGWsSHGm0PyVWrv68SKz+YObL6b6lIqgDk0rXlFv5STHHL9g4NKIl9jcsNC25ULp8+56PuK+58DkrmAGdHuinH4HNcFnNDtvEOTl6It34EhoZxHguMKLc3M5GwC/6njJTDR0PxAQidoptK8vcYt4cwxmhYVWytEGGiRUtcQFtj40scpGak2/m896s4KktPjKSysq8+nvvTo+taeORXldirwf1LQGU0Gnc/4a/MijmF5r+4Zh6J1eFDjn4Sh3e7TCUlZ+eCehzT6E4Ky03phdJamkRTYy8qgRvfuJ3snDSFaK1vIkLsa94NgVAxDyhwlPTWgF7+JRH62uI72TPjvX/Lh3MPiJdOxitfc9xKtjWf+mKKLoset3RWM+T0gbOmXqgWh1Jklnkx+8XO8Hn8UZqNGTyd5OAdFJhIiOYF9Pqis43tbFURC52VrGBQH/0LVgmm3scAo3UvBQCOUe2Cf19MgQTo7EvOShkPi/CkEfZE79g0IhBEyGqZVuSI+bZZy07Q/SViMtR5mnkxy2PJwMo/E9HTNzrN/ki3cvW6twSQK8xLsgCxzyaGmSQSf9b3606ywDByssRppxOA+zlF4httrjB3h2Jqi2jkeoLhIgv1pXwwQsfgsRvukfHjVO0f0h2BSrteoUeDM3SUPn90q+DmCq3qGyEdFfgdBrVyWadHnPTHBFpmA/lVl2WHAHeb6iBORvkMU5AGBzMQdf40jR4cIb00oCUapU8oaFRd03lCl3sxwo/kBDBBOJiT3tMTLVXHR9IYKDxkcus1No1ne+z3wXomYMEoUP1IjV3tG9w/YR4ps"}}
41. 2020-08-27 16:49:29 -0400 EDT:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron: sector not found
42. 2020-08-27 16:50:29 -0400 EDT:  [event;sealing.SectorRetryComputeProof] {"User":{}}
43. 2020-08-27 16:50:29 -0400 EDT:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
44. 2020-08-27 16:50:29 -0400 EDT:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron: sector not found
flyworker commented 4 years ago

Same here:

VERSION: 0.5.2+git.4c71182c

2020-08-27T21:16:26.030Z WARN sectors storage-sealing/fsm.go:403 sector 18 got error event sealing.SectorChainPreCommitFailed: pushing message to mpool: GasEstimateMessageGas error: estimating gas used: message execution failed: exit 16, reason: failed to verify deals and get deal weight (RetCode=16) 2020-08-27T21:16:26.048Z INFO sectors storage-sealing/states_failed.go:19 PreCommitFailed(18), waiting 59.95155711s before retrying 2020-08-27T21:17:26.025Z INFO sectors storage-sealing/states_sealing.go:205 submitting precommit for sector 18 (deposit: 907611359740608812): 2020-08-27T21:17:26.030Z WARN sectors storage-sealing/fsm.go:403 sector 18 got error event sealing.SectorChainPreCommitFailed: pushing message to mpool: GasEstimateMessageGas error: estimating gas used: message execution failed: exit 16, reason: failed to verify deals and get deal weight (RetCode=16) 2020-08-27T21:17:26.047Z INFO sectors storage-sealing/states_failed.go:19 PreCommitFailed(18), waiting 59.95300321s before retrying 2020-08-27T21:18:03.288Z INFO rpc go-jsonrpc@v0.1.2-0.20200817153016-2ea5cbaf5ec0/client.go:213 rpc output message buffer {"n": 2} 2020-08-27T21:18:05.086Z INFO rpc go-jsonrpc@v0.1.2-0.20200817153016-2ea5cbaf5ec0/client.go:213 rpc output message buffer {"n": 2} 2020-08-27T21:18:06.839Z INFO rpc go-jsonrpc@v0.1.2-0.20200817153016-2ea5cbaf5ec0/client.go:213 rpc output message buffer {"n": 2} 2020-08-27T21:18:06.877Z INFO rpc go-jsonrpc@v0.1.2-0.20200817153016-2ea5cbaf5ec0/client.go:213 rpc output message buffer {"n": 2} 2020-08-27T21:18:26.028Z INFO sectors storage-sealing/states_sealing.go:205 submitting precommit for sector 18 (deposit: 907458829645106889): 2020-08-27T21:18:26.033Z WARN sectors storage-sealing/fsm.go:403 sector 18 got error event sealing.SectorChainPreCommitFailed: pushing message to mpool: GasEstimateMessageGas error: estimating gas used: message execution failed: exit 16, reason: failed to verify deals and get deal weight (RetCode=16) 2020-08-27T21:18:26.051Z INFO sectors storage-sealing/states_failed.go:19 PreCommitFailed(18), waiting 59.948073999s before retrying 2020-08-27T21:19:26.026Z INFO sectors storage-sealing/states_sealing.go:205 submitting precommit for sector 18 (deposit: 907397246332685588): 2020-08-27T21:19:26.030Z WARN sectors storage-sealing/fsm.go:403 sector 18 got error event sealing.SectorChainPreCommitFailed: pushing message to mpool: GasEstimateMessageGas error: estimating gas used: message execution failed: exit 16, reason: failed to verify deals and get deal weight (RetCode=16) 2020-08-27T21:19:26.046Z INFO sectors storage-sealing/states_failed.go:19 PreCommitFailed(18), waiting 59.953340157s before retrying

jennijuju commented 4 years ago

@RobQuistNL do you still have this issue with recent lotus?

WhoAmI233 commented 4 years ago

I ran a debug version and ran it all night. More than 400 of them were fine, and then I reported this error later. The sector log is as follows lotus version

Daemon: 0.7.1+debug+git.6b585734.dirty+api0.15.0 Local: lotus version 0.7.1+debug+git.6b585734.dirty

lotus-miner info

Miner: t01000 Sector Size: 2 KiB Byte Power: 808 KiB / 808 KiB (100.0000%) Actual Power: 844 Ki / 844 Ki (100.0000%) Committed: 808 KiB Proving: 808 KiB Expected block win rate: 21600.0000/day (every 4s)

Deals: 0, 0 B Active: 0, 0 B (Verified: 0, 0 B)

Miner Balance: 428031.97466098107070389 FIL PreCommit: 0 FIL Pledge: 0.00002515315970048 FIL Locked: 424670.892292775003981079 FIL Available: 3361.082343052907022331 FIL Worker Balance: 49999999.991293904351983978 FIL Market (Escrow): 0 FIL Market (Locked): 0 FIL

Expected Seal Duration: 12h0m0s

Sectors: Total: 416 Proving: 404 CommitFailed: 12

lotus-miner sectors status --log 415

SectorID: 415 Status: CommitFailed CIDcommD: baga6ea4seaqpy7usqklokfx2vxuynmupslkeutzexe2uqurdg5vhtebhxqmpqmy CIDcommR: bagboea4b5abca23yrlqpou6hcyjckytmjjpjynhyf24gbe77p3zjfmb67uvjkvqh Ticket: e38bbf35272cf9bcb603340c207631bc18ac3fd1c80517ab7da6c7ed0880ef36 TicketH: 8970 Seed: 1bb07c701868601d91d2ec6b36d130afd153d24e3aafb9b7c5ab21a974ebd126 SeedH: 10022 Precommit: bafy2bzacea6whj3vvxogd4aehwfafbqalsp3t6mvquepd2dftgkyrntbdk2g6 Commit: bafy2bzacebs2iat4r5qde4c3nnkyxgp5ajiqkdczznnaicop4nacs6gh7ajaq Proof: a8fd91b1a736eb36d4a88f7da1660a938752b066d33f7e3774997be971fcb20d30bc707031582260e861bff559e14f9499d5528e41bb2255e26a82f2878920cf7efb4df0f672f00d8472ec6430a840c65b7aadcc04baac39d20f6cdb23d221930170b5f1e3a2b807736dc9b1cbfecb65e4070e65888cb31a9e8441230b7c38c9d6501ca15ee31a784c9eea9393472c68ae9782dd34e6e024840b49f2b03cb9a48d4233b54cea039a09a536fdbdb88bf732cbafc279234b6d751ba0572825b5ef Deals: [0] Retries: 104 Event Log:

  1. 2020-09-25 07:30:01 +0800 CST: [event;sealing.SectorStartCC] {"User":{"ID":415,"SectorType":0,"Pieces":[{"Piece":{"Size":2048,"PieceCID":{"/":"baga6ea4seaqpy7usqklokfx2vxuynmupslkeutzexe2uqurdg5vhtebhxqmpqmy"}},"DealInfo":null}]}}
  2. 2020-09-25 07:30:01 +0800 CST: [event;sealing.SectorPacked] {"User":{"FillerPieces":null}}
  3. 2020-09-25 07:30:01 +0800 CST: [event;sealing.SectorPreCommit1WaitResponse] {"User":{"TicketValue":"44u/NScs+by2AzQMIHYxvBisP9HIBRerfabH7QiA7zY=","TicketEpoch":8970}}
  4. 2020-09-25 07:30:01 +0800 CST: [event;sealing.SectorPreCommit1] {"User":{"PreCommit1Out":"eyJyZWdpc3RlcmVkX3Byb29mIjoiU3RhY2tlZERyZzJLaUJWMSIsImxhYmVscyI6eyJTdGFja2VkRHJnMktpQlYxIjp7ImxhYmVscyI6W3sicGF0aCI6Ii9yb290L2xvdHVzd29ya2VyX2FtZEFsbC9jYWNoZS9zLXQwMTAwMC00MTUiLCJpZCI6ImxheWVyLTEiLCJzaXplIjo2NCwicm93c190b19kaXNjYXJkIjo1fSx7InBhdGgiOiIvcm9vdC9sb3R1c3dvcmtlcl9hbWRBbGwvY2FjaGUvcy10MDEwMDAtNDE1IiwiaWQiOiJsYXllci0yIiwic2l6ZSI6NjQsInJvd3NfdG9fZGlzY2FyZCI6NX1dLCJfaCI6bnVsbH19LCJjb25maWciOnsicGF0aCI6Ii9yb290L2xvdHVzd29ya2VyX2FtZEFsbC9jYWNoZS9zLXQwMTAwMC00MTUiLCJpZCI6InRyZWUtZCIsInNpemUiOjEyNywicm93c190b19kaXNjYXJkIjo1fSwiY29tbV9kIjpbMjUyLDEyNiwxNDYsMTMwLDE1MCwyMjksMjIsMjUwLDE3MywyMzMsMTM0LDE3OCwxNDMsMTQ2LDIxMiw3NCw3OSwzNiwxODUsNTMsNzIsODIsMzUsNTUsMTA2LDEyMSwxNDQsMzksMTg4LDI0LDI0OCw1MV19"}}
  5. 2020-09-25 07:30:01 +0800 CST: [event;sealing.SectorPreCommit2WaitResponse] {"User":{}}
  6. 2020-09-25 07:30:01 +0800 CST: [event;sealing.SectorPreCommit2] {"User":{"Sealed":{"/":"bagboea4b5abca23yrlqpou6hcyjckytmjjpjynhyf24gbe77p3zjfmb67uvjkvqh"},"Unsealed":{"/":"baga6ea4seaqpy7usqklokfx2vxuynmupslkeutzexe2uqurdg5vhtebhxqmpqmy"}}}
  7. 2020-09-25 07:30:01 +0800 CST: [event;sealing.SectorPreCommitted] {"User":{"Message":{"/":"bafy2bzacea6whj3vvxogd4aehwfafbqalsp3t6mvquepd2dftgkyrntbdk2g6"},"PreCommitDeposit":"0","PreCommitInfo":{"SealProof":0,"SectorNumber":415,"SealedCID":{"/":"bagboea4b5abca23yrlqpou6hcyjckytmjjpjynhyf24gbe77p3zjfmb67uvjkvqh"},"SealRandEpoch":8970,"DealIDs":[],"Expiration":1561617,"ReplaceCapacity":false,"ReplaceSectorDeadline":0,"ReplaceSectorPartition":0,"ReplaceSectorNumber":0}}}
  8. 2020-09-25 07:30:32 +0800 CST: [event;sealing.SectorPreCommitLanded] {"User":{"TipSet":"AXGg5AIg6+pkUHcmXuNsqf2DDv1qIGEA89aMQRLF18b6AzT/FLw="}}
  9. 2020-09-25 07:40:32 +0800 CST: [event;sealing.SectorSeedReady] {"User":{"SeedValue":"G7B8cBhoYB2R0uxrNtEwr9FT0k46r7m3xashqXTr0SY=","SeedEpoch":10022}}
  10. 2020-09-25 07:40:32 +0800 CST: [event;sealing.SectorCommitting1WaitResponse] {"User":{}}
  11. 2020-09-25 07:40:32 +0800 CST: [event;sealing.SectorCommitting1] {"User":{"Commit1Out":null}}
  12. 2020-09-25 07:40:32 +0800 CST: [event;sealing.SectorCommitting2WaitResponse] {"User":{}}
  13. 2020-09-25 07:42:01 +0800 CST: [event;sealing.SectorCommitting2] {"User":{"Proof":"qP2Rsac26zbUqI99oWYKk4dSsGbTP343dJl76XH8sg0wvHBwMVgiYOhhv/VZ4U+UmdVSjkG7IlXiaoLyh4kgz377TfD2cvANhHLsZDCoQMZbeq3MBLqsOdIPbNsj0iGTAXC18eOiuAdzbcmxy/7LZeQHDmWIjLManoRBIwt8OMnWUByhXuMaeEye6pOTRyxorpeC3TTm4CSEC0nysDy5pI1CM7VM6gOaCaU2/b24i/cyy6/CeSNLbXUboFcoJbXv"}}
  14. 2020-09-25 07:42:01 +0800 CST: [event;sealing.SectorCommitted] {"User":{"Proof":"qP2Rsac26zbUqI99oWYKk4dSsGbTP343dJl76XH8sg0wvHBwMVgiYOhhv/VZ4U+UmdVSjkG7IlXiaoLyh4kgz377TfD2cvANhHLsZDCoQMZbeq3MBLqsOdIPbNsj0iGTAXC18eOiuAdzbcmxy/7LZeQHDmWIjLManoRBIwt8OMnWUByhXuMaeEye6pOTRyxorpeC3TTm4CSEC0nysDy5pI1CM7VM6gOaCaU2/b24i/cyy6/CeSNLbXUboFcoJbXv"}}
  15. 2020-09-25 07:42:01 +0800 CST: [event;sealing.SectorCommitSubmitted] {"User":{"Message":{"/":"bafy2bzacebs2iat4r5qde4c3nnkyxgp5ajiqkdczznnaicop4nacs6gh7ajaq"}}}
  16. 2020-09-25 07:42:32 +0800 CST: [event;sealing.SectorCommitFailed] {"User":{}} proof validation failed, sector not found in sector set after cron
  17. 2020-09-25 07:43:32 +0800 CST: [event;sealing.SectorRetryComputeProof] {"User":{}}
  18. 2020-09-25 07:43:32 +0800 CST: [event;sealing.SectorRetryCommitWait] {"User":{}}
  19. 2020-09-25 07:43:32 +0800 CST: [event;sealing.SectorCommitFailed] {"User":{}} proof validation failed, sector not found in sector set after cron
  20. 2020-09-25 07:44:32 +0800 CST: [event;sealing.SectorRetryComputeProof] {"User":{}}
  21. 2020-09-25 07:44:32 +0800 CST: [event;sealing.SectorRetryCommitWait] {"User":{}}
  22. 2020-09-25 07:44:32 +0800 CST: [event;sealing.SectorCommitFailed] {"User":{}} proof validation failed, sector not found in sector set after cron
  23. 2020-09-25 07:45:32 +0800 CST: [event;sealing.SectorRetryComputeProof] {"User":{}}
  24. 2020-09-25 07:45:32 +0800 CST: [event;sealing.SectorRetryCommitWait] {"User":{}}
  25. 2020-09-25 07:45:32 +0800 CST: [event;sealing.SectorCommitFailed] {"User":{}} proof validation failed, sector not found in sector set after cron
RobQuistNL commented 4 years ago

@jennijuju no, but I also haven't had any deals yet with the recent lotus :)

TippyFlitsUK commented 4 years ago

Same issue for first time here.

lotus version 0.8.1+git.1ebad94d

t08403

SectorID:   129
Status:     Removing
CIDcommD:   baga6ea4seaqe4t6w3bjjbpzcwjo5s23eht6p3zvtv53gwd6dh42nfhtj5ghbuly
CIDcommR:   bagboea4b5abcbznott2nyuwn5yn23ylmxjrko5ts2dlm3jtxm5ydtbedzygkddlj
Ticket:     42726af3601ee7eae7bdb722a5afc095ed8690bc4f62faeb8ce374f08a635ba6
TicketH:    108717
Seed:       be2fb1fedad93909e9df32172296c41a06889ea3804d15b3d3d10e8f9c7b6380
SeedH:      110467
Precommit:  bafy2bzacecbshuledex32aqh33kgfszottcd7xsqv5ci2bfwis737hbxvliqc
Commit:     bafy2bzaceb6x4m3k7n5zmnsg3p2o43vsek5p4szg4rwzemxwlya5cvklslj34
Proof:      ad50cf2f7490b7e679d6f1db0bed4cc01a1df26643e8ceea7ae4e587853a37dfaa12ea7c5d8fa771c809afae64e05a368cffb5f9a1f66227704072f21184bfb4b5c582100bba2174b3443c7b68fe5b60d30374a0f684a03922fe5217ec70c0700ac94ab7a06236e3dbb880d95733a308a5f644ce1de24b39fbfdbdbfc2ed526001110b479a7d33dc213cbd50f21df6a7afa99f0907dcdfcc0094a915160af8034410b65f9c61cbf6fa61460d776ae123ab97d0b6ee1a3663b87d6c21020e8a3e8d145398034643fb7472a1b89ebe3951f87c354077d2d1ba74361c1c1c4a2c5178a7ea59bd1ea15a2853bf0d645ad662a83bae2d11bea3a288b28b646d0de2300c4ecb0839e7cfe455a46aa6d56df7db66a15028642773ef5411a20d039caf491492e4548ce3652f9bb1d35c819008420d21576e7bacf4529daeb19e3c245d0723a0abd5ced071038aba1ba61a9de990b032f0fb83af89c98206fe526ad1cbfc785b5d220916d6e4fd0b9a1e62fd4762bcac88320fd7569f07456006b761fb92a82e8ab63ed6e393cb36bce255e3130cae867287990326be684cae1a910c29ec45fff7e4fd566c59f0470fd9de235ec28b053b29a0bd2ceb35e9231a1e987658e3fbfb2af2222eb2097374693e47b74ae728461c8f4592b7eb2b9106cd3e52e301da2fbb686c49a5d002f7f677723fe4a445f78caa377d7bdd618d791914805a6ceba8f35d057204c504a0a4ef6c55eca0ca016442333e646974eabc5b38ce623d3d2c6d4a088f2c04c903fc2de2e1ff8ddf53c2ee7368890e57c887075b82dbb9f979835c83b6f1b3df1453e017687c9acf9e8048ae9f9301ce2086383c025b01c4feba2a6c97aaf6aaf7728925c5a6b432a9b83cfec9af250bec7d3f552dca52acbacafcb9ed8d19bbdc9f3b3d8bfa5872ea5c5eed869121216405998fffff0b4d3a953344d7ddd63675b2858ab1436ba4c0c8a0d29bfd140c03ab95d8fbc3a30af06dbf1f2ac8af8bf09e70d0e27d969a4ba62bd75a012d59a285df7a41e6eea6f3c0dcd60f685d7fa560eef408355720f195b6f32d3d28b1c55f4522e976a22b634d8320fee7d2912626e1b606063ba7bf0aa405a32e90ec7c17a00569f9ad0366d64cca68e49dbc0b1f49e12b34a5c3de280550b40337a52bd9ecdf691461508c18449b29e3dc966b445b0b901af65be77c94cbb8407c4dfe3418b6606a0eac5914b5eaf1b8171b3bf5275ccc5ff9ec16fc6baab86de093022b0596e95a5d96c756e6c5a1e1e60747a29ba36ccda1110cddf6b99eae5d32d3bb3422f1993967e6d3e9d981ce072dfcf2e633d2b92eb1582de162680b1394d571ea3efbeda47c4ef9895f76c1fffb815635a38555226abba20fee790724f1250f550f14ff2e26ae6773cb7c6839ea1ad81e8e48ef988724a19d16803707257bf416e9a2b79e52fb325248f4d06db0ce42bd3b29b45a1a7a43d5ac68a4067eae64aa7e9a290a770122ddaca1e0153d1d8d432dd6fc41bdc3a9c823d17ea1f133333cd19744f7d4979eef83a7442e36b135a6962b2faf5ed6fafabeebf028082412b47007291b4c70451e1fc54e4678049788cdb50180e5475a331a35aa41c71e31146c4666a83615d197bb341ea876ac15edea031e129fe9dcd79806b950ab9d04faad291bbaf05af0d17ab77671724c1f986b502ab9580e150a22bd32a522cc0569080682a965b9ff65f83b5ed0c5015d48e1e132cea8fc1c8aca4e0d3bec3c18e6be2b130b4030a5a9a6f0635481a8c0827e7172866b27787014aec400d13fd52fe472cc94d5fbf74a3f8c48809c3d77cb8fb7208419611def2cc18c6dc366496cf93ae0d6faa94a96480d199977e8430a9bdcd23a1a7ede27f4e212a6dacee7a946df28acc72fbe502076ca7837e63df63791580f756fffb4ae3c829360203ef17e65d062861990525d54ccf2ec071eb6674a3da10b72157fd2b3f6e7f2ad4b0e92ad9a8d6438d9cb6f0244f96e9399a665cc791afb6f44db946830cf33c476e3c944e31617b09066951b7e16bb163bd67bd2f962eb54a8459e1dd613344df90874eb9e152323d77abf28fd48b83bfa0614ff73b1944bc0d768e90afb4d78cf6ade91106321b776b1dbdc5728d31a1389cbe70a346e3f576678bfafef076889e90f19d093fc2e5564ce14bf1555bd218de372f80b7524735525b1fe52af387af8294cb903b5ce31905689a06de53163a9a18d66aad2ea7d71ecd7bfbf2e97ff6c0a69e292475af5472abe38e74be02eb88279cf56f5f34e227dd89d2315a8da0819b05e0d5caf2ca0bc5b535d50a41af05ea970382cf7fc3e405360ff1ccf13cf210a915d6ef49878e2c73fcb02cd04caef54a583dffc6d0b9a49615001ee28d07e91257e0eb75842242605f434b9782fbf3ad4010b8c39943515457b6d278e3311f930a2f8d065ab750f709483a7d74ddbdd1b3d766be336460a5ff2565bd17b3b0e7d005baef6086e8ff909d183c495da5fbfa94658c414bff0ff3bd44c7265cf63a86c988e00a9bdf6e4719d83686b034198cd9c8506531bcdc96843ad08ed45c77f0685ca9b27494e30d46018dab503802e59dc8b10de6160dba438904a2047e83064543c3799c0792da4da007afb218999b09f3f57fda742e564acd7f9ac25526348bc23dcf93a9ce181324df479083ad173c22ca465de1b53aaffccf1f375138a
Deals:      [355473 356644 358468 0 0 0 0 0 0 360851 0 364628 364867 364948 0 364993 365420 365542 0 365811 0 366176 366425 366437 0 0 0 0 366646 369038 0 0 369641 0 0 0 0]
Retries:    239
--------
Event Log:
0.  2020-10-01 17:09:30 +0100 BST:  [event;sealing.SectorStart] {"User":{"ID":129,"SectorType":3}}
1.  2020-10-01 17:09:40 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":536870912,"PieceCID":{"/":"baga6ea4seaqmp62gwjcgh2mxnvjriw4ugiti4g5mdntdd2mhybsgqj75aqvgegi"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacecxim6ysvg2kfk6ikz5i35w6mwleegz6f6vhjotfxfhodybw7ascq"},"DealID":355473,"DealSchedule":{"StartEpoch":117360,"EndEpoch":638369},"KeepUnsealed":false}}}}
2.  2020-10-01 17:57:10 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":536870912,"PieceCID":{"/":"baga6ea4seaqjug4kejx6ftg46qasy427cgbtdjoxqi5hvptvkrffbik72t7mgjy"}},"DealInfo":{"PublishCid":{"/":"bafy2bzaceaahl4o2kknslqzwam4rcyi6uxdaq5bytmhfhsdmi7z5kdx2nsbes"},"DealID":356644,"DealSchedule":{"StartEpoch":117457,"EndEpoch":638369},"KeepUnsealed":true}}}}
3.  2020-10-01 18:40:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":131072,"PieceCID":{"/":"baga6ea4seaqczvq3kgxg6p2rh7fmwvyfwvnhfxpbmzggg7peijl6ipull32hmba"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacebnsktqldoj5c4msz3lylwfctgrbrmae6qcr2doslalhffttw5vqu"},"DealID":358468,"DealSchedule":{"StartEpoch":111788,"EndEpoch":690209},"KeepUnsealed":true}}}}
4.  2020-10-01 19:36:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":131072,"PieceCID":{"/":"baga6ea4seaqi5hreap5iqthwen7wbxzf7a7oidokt3mht23pmnjncuee6wwq2py"}},"DealInfo":null}}}
5.  2020-10-01 19:36:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":262144,"PieceCID":{"/":"baga6ea4seaqhklmwsp5bm5jehfkhnyyxvgcyb4aji6x3piyfidlclkjjdtasuby"}},"DealInfo":null}}}
6.  2020-10-01 19:36:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":524288,"PieceCID":{"/":"baga6ea4seaqhaixwb57pnlp2c4ixuutbtyym5kbmnadvvxy4mz3yn3cqn3xs2gi"}},"DealInfo":null}}}
7.  2020-10-01 19:36:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":1048576,"PieceCID":{"/":"baga6ea4seaqntgehxfzvoouw4ejzgzcsg3axwh2moa2noi6hvgpxbg5u3jqrmky"}},"DealInfo":null}}}
8.  2020-10-01 19:36:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":2097152,"PieceCID":{"/":"baga6ea4seaqnbnjq3oylj4s4luxsukg752aiwu2bfibjghyyysm7lisubbvrgjq"}},"DealInfo":null}}}
9.  2020-10-01 19:36:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":4194304,"PieceCID":{"/":"baga6ea4seaqijqccdoqgqwqbx54vui2eazh6ijf5kku5eq3xwokp6tclivuoqei"}},"DealInfo":null}}}
10. 2020-10-01 19:36:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":8388608,"PieceCID":{"/":"baga6ea4seaqpym6sxmmlggt5us5axh4xxgwqzqaawsta2vwn5sa2zhottizosni"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacecmun4sybqdjqo6d4izxq6u4mrqdimazq6lpb5xed6f2g6b3my5xk"},"DealID":360851,"DealSchedule":{"StartEpoch":117660,"EndEpoch":638369},"KeepUnsealed":false}}}}
11. 2020-10-01 20:51:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":16777216,"PieceCID":{"/":"baga6ea4seaqkejdvbaufquewln7dgszre6ymaqvr2bdnyvcaee3we7gypgoocoq"}},"DealInfo":null}}}
12. 2020-10-01 20:51:31 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":33554432,"PieceCID":{"/":"baga6ea4seaqprgnhojniowmit7d7yjlxkga2ql27rxeisau7yx3bshbdxjaasaq"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacebxetqitpogfr6xbqlt5ydodrpewm642wbrj3kqugf5j5c3s7tmvw"},"DealID":364628,"DealSchedule":{"StartEpoch":117810,"EndEpoch":638369},"KeepUnsealed":false}}}}
13. 2020-10-01 20:56:31 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":67108864,"PieceCID":{"/":"baga6ea4seaqild2rn2eyaor6nrgsqzbqjq3wiu2bdm3vozpeupxseeeq7uyw4ja"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacedzyoomhlk6sfkmkptiay3qp225cqkasozjjyxzp5qnitp3tcvoci"},"DealID":364867,"DealSchedule":{"StartEpoch":117816,"EndEpoch":638369},"KeepUnsealed":false}}}}
14. 2020-10-01 20:58:01 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":67108864,"PieceCID":{"/":"baga6ea4seaqg5n7fss5j4wk4cy6fnfa2u56yqoaamfbxlwepkwr34xu5kisbgei"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacec7bbcth3rohqfrcmllga436c7iktc47xwawgps5qh2oincmwb75y"},"DealID":364948,"DealSchedule":{"StartEpoch":117816,"EndEpoch":638369},"KeepUnsealed":false}}}}
15. 2020-10-01 21:00:01 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":67108864,"PieceCID":{"/":"baga6ea4seaqnsqov4dldcsuzlqz77pkpxzurddlt2ts72lgtd4hxzbxl3ukoobq"}},"DealInfo":null}}}
16. 2020-10-01 21:00:06 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":268435456,"PieceCID":{"/":"baga6ea4seaqey2q3m74ni5aaqphl33ij2seafe7zzjcaox6tvlbyxewts6reakq"}},"DealInfo":{"PublishCid":{"/":"bafy2bzaceclkayh7m2e5y2h5p77xdjvx45kfmr55eibeaf4tvzphericsx74e"},"DealID":364993,"DealSchedule":{"StartEpoch":117816,"EndEpoch":638369},"KeepUnsealed":false}}}}
17. 2020-10-01 21:07:05 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":268435456,"PieceCID":{"/":"baga6ea4seaqbsyrqdt2hve6zpjkgt23uw5ek7btgdgqxkiw5re6afof56u5ncha"}},"DealInfo":{"PublishCid":{"/":"bafy2bzaceakhuglzewfqdbezbouoixfijsbisyczu7jipk6es2i7c3gqwnv76"},"DealID":365420,"DealSchedule":{"StartEpoch":117819,"EndEpoch":638369},"KeepUnsealed":false}}}}
18. 2020-10-01 21:09:31 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":33554432,"PieceCID":{"/":"baga6ea4seaqpel344nqupnacxxjyuthlbbarvttu3g3kbmo6yepb7r7i7ripaaq"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacea2vy3qp4hliqshfj2fiukgonfzfy3l7lyfei6ci73hvwrvyta3aa"},"DealID":365542,"DealSchedule":{"StartEpoch":117821,"EndEpoch":638369},"KeepUnsealed":false}}}}
19. 2020-10-01 21:14:31 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":33554432,"PieceCID":{"/":"baga6ea4seaqnv7nlnwutmrctyjwtg4tlt7x6gq56r6awjhwabgvnh6x7kbqxkca"}},"DealInfo":null}}}
20. 2020-10-01 21:14:32 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":67108864,"PieceCID":{"/":"baga6ea4seaqonzpzmpxt2ictyy3ci3z24tt2hhgtmn2pipriikz4l7jo753x2oq"}},"DealInfo":{"PublishCid":{"/":"bafy2bzaceb5lfrqrpwksavd23hajel65ilil74n5qjv3bnnbdj2uorolqpw3a"},"DealID":365811,"DealSchedule":{"StartEpoch":117821,"EndEpoch":638369},"KeepUnsealed":false}}}}
21. 2020-10-01 21:23:32 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":134217728,"PieceCID":{"/":"baga6ea4seaqfctcdlq6qju2juu3f7pkz77drgyurcf4fteobupctv4rapf2buly"}},"DealInfo":null}}}
22. 2020-10-01 21:23:37 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":268435456,"PieceCID":{"/":"baga6ea4seaqmx73krcxoiq4l6o2nhbglpfak3lj57jtk2rk7udpmz4sk37yxmni"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacebgpxjvtqwfxdohmiknwradseo5minyrjhmqlj5exansg5scpkfkg"},"DealID":366176,"DealSchedule":{"StartEpoch":117821,"EndEpoch":638369},"KeepUnsealed":false}}}}
23. 2020-10-01 21:28:06 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":268435456,"PieceCID":{"/":"baga6ea4seaqhilnuahhhm24rd5annv5lwpomglb3hkramjutayn4mvuvpn5z4oy"}},"DealInfo":{"PublishCid":{"/":"bafy2bzaceb4ujryrp66vx4otaum3acttmybqlxcw4efobf5bfa7ptifka2vze"},"DealID":366425,"DealSchedule":{"StartEpoch":117821,"EndEpoch":638369},"KeepUnsealed":false}}}}
24. 2020-10-01 21:29:00 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":16777216,"PieceCID":{"/":"baga6ea4seaqayknvgzwltlo7jdhqamu5usy4dsmnuu6qchrltugmzrdktuznula"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacedlkfns5jrk7nn5q2oeymgvorjlf2hndo3ncmmq2dftipkw7d26ls"},"DealID":366437,"DealSchedule":{"StartEpoch":117821,"EndEpoch":638369},"KeepUnsealed":false}}}}
25. 2020-10-01 21:34:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":16777216,"PieceCID":{"/":"baga6ea4seaqkejdvbaufquewln7dgszre6ymaqvr2bdnyvcaee3we7gypgoocoq"}},"DealInfo":null}}}
26. 2020-10-01 21:34:31 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":33554432,"PieceCID":{"/":"baga6ea4seaqnv7nlnwutmrctyjwtg4tlt7x6gq56r6awjhwabgvnh6x7kbqxkca"}},"DealInfo":null}}}
27. 2020-10-01 21:34:32 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":67108864,"PieceCID":{"/":"baga6ea4seaqnsqov4dldcsuzlqz77pkpxzurddlt2ts72lgtd4hxzbxl3ukoobq"}},"DealInfo":null}}}
28. 2020-10-01 21:34:34 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":134217728,"PieceCID":{"/":"baga6ea4seaqfctcdlq6qju2juu3f7pkz77drgyurcf4fteobupctv4rapf2buly"}},"DealInfo":null}}}
29. 2020-10-01 21:34:38 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":268435456,"PieceCID":{"/":"baga6ea4seaqiji3lefaraasym5hgkg22xt42m5aevnulecqzwnzvqbiuadapyhy"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacec757ghnw55mfovawcbn5nr23o2ra3xssm6mkiooywa4gysyb53tg"},"DealID":366646,"DealSchedule":{"StartEpoch":117821,"EndEpoch":638369},"KeepUnsealed":false}}}}
30. 2020-10-01 22:40:05 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":268435456,"PieceCID":{"/":"baga6ea4seaqfj6u3xqkpvwv3zwxebpbkss5dfblk7kpqcrzp6yznzjubnlvasji"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacea4ooiizso3jfraagfapwio2hrlzlk7m7tl4nmlc6v3sgun6bh5eq"},"DealID":369038,"DealSchedule":{"StartEpoch":117821,"EndEpoch":638369},"KeepUnsealed":false}}}}
31. 2020-10-01 23:16:35 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":268435456,"PieceCID":{"/":"baga6ea4seaqk2bufhfu5g7ju74eobh2wsmfevum2rhppmdf75z7b2m4byhtryny"}},"DealInfo":null}}}
32. 2020-10-01 23:16:42 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":536870912,"PieceCID":{"/":"baga6ea4seaqdsvqopmj2soyhujb72jza76t4wpq5fzifvm3ctz47iyytkewnubq"}},"DealInfo":null}}}
33. 2020-10-01 23:17:01 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":1073741824,"PieceCID":{"/":"baga6ea4seaqlxcmuvnhcxwlfeob2664f6bf5stglbv5odtg75p736dwfnvod6nq"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacecxi6abdlnqomxvpsj3ggqxn3fbske36bdaqmnaztw563dtlplw46"},"DealID":369641,"DealSchedule":{"StartEpoch":117913,"EndEpoch":638369},"KeepUnsealed":false}}}}
34. 2020-10-02 00:21:35 +0100 BST:  [event;sealing.SectorStartPacking]  {"User":{}}
35. 2020-10-02 00:28:53 +0100 BST:  [event;sealing.SectorPacked]    {"User":{"FillerPieces":[{"Size":1073741824,"PieceCID":{"/":"baga6ea4seaqmzq6acl23axubdiv37xipnaz3qqtvwr57ekoaauviescphqnfwpi"}},{"Size":2147483648,"PieceCID":{"/":"baga6ea4seaqh34u3nf3tdgpi6k2aw54rtucikcpo25uofrzjpmprinydj7b4mla"}},{"Size":8589934592,"PieceCID":{"/":"baga6ea4seaqnqyicdbbfvnpjlmokmi45fgroiigxa2uw6nz6f6ojveoxlhizwai"}},{"Size":17179869184,"PieceCID":{"/":"baga6ea4seaqg2nsld34emra2ljfgrbrdcswmbjdpaftrpzjuipudt3w7qpbikpa"}}]}}
36. 2020-10-02 05:28:23 +0100 BST:  [event;sealing.SectorPreCommit1]    {"User":{"PreCommit1Out":"eyJyZWdpc3RlcmVkX3Byb29mIjoiU3RhY2tlZERyZzMyR2lCVjEiLCJsYWJlbHMiOnsiU3RhY2tlZERyZzMyR2lCVjEiOnsibGFiZWxzIjpbeyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItMSIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItMiIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItMyIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItNCIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItNSIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItNiIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItNyIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItOCIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItOSIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItMTAiLCJzaXplIjoxMDczNzQxODI0LCJyb3dzX3RvX2Rpc2NhcmQiOjd9LHsicGF0aCI6Ii9tbnQvbnZtZTBuMS8ubG90dXNtaW5lci9jYWNoZS9zLXQwODQwMy0xMjkiLCJpZCI6ImxheWVyLTExIiwic2l6ZSI6MTA3Mzc0MTgyNCwicm93c190b19kaXNjYXJkIjo3fV0sIl9oIjpudWxsfX0sImNvbmZpZyI6eyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoidHJlZS1kIiwic2l6ZSI6MjE0NzQ4MzY0Nywicm93c190b19kaXNjYXJkIjo3fSwiY29tbV9kIjpbNzgsNzksMjE0LDIxNiw4MiwxNDQsMTkxLDM0LDE3OCw5MywyMTcsMTA3LDEwMCw2MCwyNTIsMjUzLDIzMCwxNzksMTc1LDExOCwxMDcsMTUsMTk1LDYzLDUyLDIxMCwxNTgsMTA1LDIzMywxNDIsMjYsNDddfQ==","TicketValue":"QnJq82Ae5+rnvbcipa/Ale2GkLxPYvrrjON08IpjW6Y=","TicketEpoch":108717}}
37. 2020-10-02 06:18:07 +0100 BST:  [event;sealing.SectorPreCommit2]    {"User":{"Sealed":{"/":"bagboea4b5abcbznott2nyuwn5yn23ylmxjrko5ts2dlm3jtxm5ydtbedzygkddlj"},"Unsealed":{"/":"baga6ea4seaqe4t6w3bjjbpzcwjo5s23eht6p3zvtv53gwd6dh42nfhtj5ghbuly"}}}
38. 2020-10-02 06:18:08 +0100 BST:  [event;sealing.SectorPreCommitted]  {"User":{"Message":{"/":"bafy2bzacecbshuledex32aqh33kgfszottcd7xsqv5ci2bfwis737hbxvliqc"},"PreCommitDeposit":"299095700588327276","PreCommitInfo":{"SealProof":3,"SectorNumber":129,"SealedCID":{"/":"bagboea4b5abcbznott2nyuwn5yn23ylmxjrko5ts2dlm3jtxm5ydtbedzygkddlj"},"SealRandEpoch":108717,"DealIDs":[355473,356644,358468,360851,364628,364867,364948,364993,365420,365542,365811,366176,366425,366437,366646,369038,369641],"Expiration":1631969,"ReplaceCapacity":true,"ReplaceSectorDeadline":0,"ReplaceSectorPartition":0,"ReplaceSectorNumber":83}}}
39. 2020-10-02 06:21:38 +0100 BST:  [event;sealing.SectorPreCommitLanded]   {"User":{"TipSet":"AXGg5AIgG6OaDlNqSV+IiUF2WXnv3BHzomK9yggV9jgJMCfPRn8BcaDkAiCIoNhVsVJYfOUfj2jrNtq7ceXHWoAPsqcD1M876vbjXAFxoOQCIBM/SiRP2rqCwyvBVe1XxqiPR/BiDeAlaFGWio+Sckn0"}}
40. 2020-10-02 07:36:30 +0100 BST:  [event;sealing.SectorSeedReady] {"User":{"SeedValue":"vi+x/trZOQnp3zIXIpbEGgaInqOATRWz09EOj5x7Y4A=","SeedEpoch":110467}}
41. 2020-10-02 09:05:29 +0100 BST:  [event;sealing.SectorCommitted] {"User":{"Proof":"rVDPL3SQt+Z51vHbC+1MwBod8mZD6M7qeuTlh4U6N9+qEup8XY+nccgJr65k4Fo2jP+1+aH2YidwQHLyEYS/tLXFghALuiF0s0Q8e2j+W2DTA3Sg9oSgOSL+UhfscMBwCslKt6BiNuPbuIDZVzOjCKX2RM4d4ks5+/29v8LtUmABEQtHmn0z3CE8vVDyHfanr6mfCQfc38wAlKkVFgr4A0QQtl+cYcv2+mFGDXdq4SOrl9C27ho2Y7h9bCECDoo+jRRTmANGQ/t0cqG4nr45Ufh8NUB30tG6dDYcHBxKLFF4p+pZvR6hWihTvw1kWtZiqDuuLRG+o6KIsotkbQ3iMAxOywg558/kVaRqptVt99tmoVAoZCdz71QRog0DnK9JFJLkVIzjZS+bsdNcgZAIQg0hV257rPRSna6xnjwkXQcjoKvVztBxA4q6G6YanemQsDLw+4OvicmCBv5SatHL/HhbXSIJFtbk/QuaHmL9R2K8rIgyD9dWnwdFYAa3YfuSqC6Ktj7W45PLNrziVeMTDK6GcoeZAya+aEyuGpEMKexF//fk/VZsWfBHD9neI17CiwU7KaC9LOs16SMaHph2WOP7+yryIi6yCXN0aT5Ht0rnKEYcj0WSt+srkQbNPlLjAdovu2hsSaXQAvf2d3I/5KRF94yqN3173WGNeRkUgFps66jzXQVyBMUEoKTvbFXsoMoBZEIzPmRpdOq8WzjOYj09LG1KCI8sBMkD/C3i4f+N31PC7nNoiQ5XyIcHW4Lbufl5g1yDtvGz3xRT4BdofJrPnoBIrp+TAc4ghjg8AlsBxP66KmyXqvaq93KJJcWmtDKpuDz+ya8lC+x9P1UtylKsusr8ue2NGbvcnzs9i/pYcupcXu2GkSEhZAWZj///C006lTNE193WNnWyhYqxQ2ukwMig0pv9FAwDq5XY+8OjCvBtvx8qyK+L8J5w0OJ9lppLpivXWgEtWaKF33pB5u6m88Dc1g9oXX+lYO70CDVXIPGVtvMtPSixxV9FIul2oitjTYMg/ufSkSYm4bYGBjunvwqkBaMukOx8F6AFafmtA2bWTMpo5J28Cx9J4Ss0pcPeKAVQtAM3pSvZ7N9pFGFQjBhEmynj3JZrRFsLkBr2W+d8lMu4QHxN/jQYtmBqDqxZFLXq8bgXGzv1J1zMX/nsFvxrqrht4JMCKwWW6VpdlsdW5sWh4eYHR6Kbo2zNoREM3fa5nq5dMtO7NCLxmTln5tPp2YHOBy388uYz0rkusVgt4WJoCxOU1XHqPvvtpHxO+YlfdsH/+4FWNaOFVSJqu6IP7nkHJPElD1UPFP8uJq5nc8t8aDnqGtgejkjvmIckoZ0WgDcHJXv0Fumit55S+zJSSPTQbbDOQr07KbRaGnpD1axopAZ+rmSqfpopCncBIt2soeAVPR2NQy3W/EG9w6nII9F+ofEzMzzRl0T31Jee74OnRC42sTWmlisvr17W+vq+6/AoCCQStHAHKRtMcEUeH8VORngEl4jNtQGA5UdaMxo1qkHHHjEUbEZmqDYV0Ze7NB6odqwV7eoDHhKf6dzXmAa5UKudBPqtKRu68Frw0Xq3dnFyTB+Ya1AquVgOFQoivTKlIswFaQgGgqlluf9l+Dte0MUBXUjh4TLOqPwcispODTvsPBjmvisTC0Awpamm8GNUgajAgn5xcoZrJ3hwFK7EANE/1S/kcsyU1fv3Sj+MSICcPXfLj7cghBlhHe8swYxtw2ZJbPk64Nb6qUqWSA0ZmXfoQwqb3NI6Gn7eJ/TiEqbazuepRt8orMcvvlAgdsp4N+Y99jeRWA91b/+0rjyCk2AgPvF+ZdBihhmQUl1UzPLsBx62Z0o9oQtyFX/Ss/bn8q1LDpKtmo1kONnLbwJE+W6TmaZlzHka+29E25RoMM8zxHbjyUTjFhewkGaVG34WuxY71nvS+WLrVKhFnh3WEzRN+Qh0654VIyPXer8o/Ui4O/oGFP9zsZRLwNdo6Qr7TXjPat6REGMht3ax29xXKNMaE4nL5wo0bj9XZni/r+8HaInpDxnQk/wuVWTOFL8VVb0hjeNy+At1JHNVJbH+Uq84evgpTLkDtc4xkFaJoG3lMWOpoY1mqtLqfXHs17+/Lpf/bApp4pJHWvVHKr4450vgLriCec9W9fNOIn3YnSMVqNoIGbBeDVyvLKC8W1NdUKQa8F6pcDgs9/w+QFNg/xzPE88hCpFdbvSYeOLHP8sCzQTK71Slg9/8bQuaSWFQAe4o0H6RJX4Ot1hCJCYF9DS5eC+/OtQBC4w5lDUVRXttJ44zEfkwovjQZat1D3CUg6fXTdvdGz12a+M2Rgpf8lZb0Xs7Dn0AW672CG6P+QnRg8SV2l+/qUZYxBS/8P871ExyZc9jqGyYjgCpvfbkcZ2DaGsDQZjNnIUGUxvNyWhDrQjtRcd/BoXKmydJTjDUYBjatQOALlncixDeYWDbpDiQSiBH6DBkVDw3mcB5LaTaAHr7IYmZsJ8/V/2nQuVkrNf5rCVSY0i8I9z5OpzhgTJN9HkIOtFzwiykZd4bU6r/zPHzdROK"}}
42. 2020-10-02 09:05:29 +0100 BST:  [event;sealing.SectorCommitSubmitted]   {"User":{"Message":{"/":"bafy2bzaceb6x4m3k7n5zmnsg3p2o43vsek5p4szg4rwzemxwlya5cvklslj34"}}}
43. 2020-10-02 09:08:30 +0100 BST:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
44. 2020-10-02 09:09:30 +0100 BST:  [event;sealing.SectorRetryComputeProof] {"User":{}}
45. 2020-10-02 09:09:30 +0100 BST:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
46. 2020-10-02 09:09:30 +0100 BST:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
47. 2020-10-02 09:10:30 +0100 BST:  [event;sealing.SectorRetryComputeProof] {"User":{}}
48. 2020-10-02 09:10:30 +0100 BST:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
49. 2020-10-02 09:10:30 +0100 BST:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
50. 2020-10-02 09:11:30 +0100 BST:  [event;sealing.SectorRetryComputeProof] {"User":{}}
51. 2020-10-02 09:11:30 +0100 BST:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
52. 2020-10-02 09:11:30 +0100 BST:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
53. 2020-10-02 09:12:30 +0100 BST:  [event;sealing.SectorRetryComputeProof] {"User":{}}
54. 2020-10-02 09:12:30 +0100 BST:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
55. 2020-10-02 09:12:30 +0100 BST:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
56. 2020-10-02 09:13:30 +0100 BST:  [event;sealing.SectorRetryComputeProof] {"User":{}}
57. 2020-10-02 09:13:30 +0100 BST:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
58. 2020-10-02 09:13:30 +0100 BST:  [event;sealing.SectorCommitFailed]  {"User":{}}
jennijuju commented 4 years ago

Same issue for first time here.

lotus version 0.8.1+git.1ebad94d

t08403

SectorID: 129
Status:       Removing
CIDcommD: baga6ea4seaqe4t6w3bjjbpzcwjo5s23eht6p3zvtv53gwd6dh42nfhtj5ghbuly
CIDcommR: bagboea4b5abcbznott2nyuwn5yn23ylmxjrko5ts2dlm3jtxm5ydtbedzygkddlj
Ticket:       42726af3601ee7eae7bdb722a5afc095ed8690bc4f62faeb8ce374f08a635ba6
TicketH:  108717
Seed:     be2fb1fedad93909e9df32172296c41a06889ea3804d15b3d3d10e8f9c7b6380
SeedH:        110467
Precommit:    bafy2bzacecbshuledex32aqh33kgfszottcd7xsqv5ci2bfwis737hbxvliqc
Commit:       bafy2bzaceb6x4m3k7n5zmnsg3p2o43vsek5p4szg4rwzemxwlya5cvklslj34
Proof:        ad50cf2f7490b7e679d6f1db0bed4cc01a1df26643e8ceea7ae4e587853a37dfaa12ea7c5d8fa771c809afae64e05a368cffb5f9a1f66227704072f21184bfb4b5c582100bba2174b3443c7b68fe5b60d30374a0f684a03922fe5217ec70c0700ac94ab7a06236e3dbb880d95733a308a5f644ce1de24b39fbfdbdbfc2ed526001110b479a7d33dc213cbd50f21df6a7afa99f0907dcdfcc0094a915160af8034410b65f9c61cbf6fa61460d776ae123ab97d0b6ee1a3663b87d6c21020e8a3e8d145398034643fb7472a1b89ebe3951f87c354077d2d1ba74361c1c1c4a2c5178a7ea59bd1ea15a2853bf0d645ad662a83bae2d11bea3a288b28b646d0de2300c4ecb0839e7cfe455a46aa6d56df7db66a15028642773ef5411a20d039caf491492e4548ce3652f9bb1d35c819008420d21576e7bacf4529daeb19e3c245d0723a0abd5ced071038aba1ba61a9de990b032f0fb83af89c98206fe526ad1cbfc785b5d220916d6e4fd0b9a1e62fd4762bcac88320fd7569f07456006b761fb92a82e8ab63ed6e393cb36bce255e3130cae867287990326be684cae1a910c29ec45fff7e4fd566c59f0470fd9de235ec28b053b29a0bd2ceb35e9231a1e987658e3fbfb2af2222eb2097374693e47b74ae728461c8f4592b7eb2b9106cd3e52e301da2fbb686c49a5d002f7f677723fe4a445f78caa377d7bdd618d791914805a6ceba8f35d057204c504a0a4ef6c55eca0ca016442333e646974eabc5b38ce623d3d2c6d4a088f2c04c903fc2de2e1ff8ddf53c2ee7368890e57c887075b82dbb9f979835c83b6f1b3df1453e017687c9acf9e8048ae9f9301ce2086383c025b01c4feba2a6c97aaf6aaf7728925c5a6b432a9b83cfec9af250bec7d3f552dca52acbacafcb9ed8d19bbdc9f3b3d8bfa5872ea5c5eed869121216405998fffff0b4d3a953344d7ddd63675b2858ab1436ba4c0c8a0d29bfd140c03ab95d8fbc3a30af06dbf1f2ac8af8bf09e70d0e27d969a4ba62bd75a012d59a285df7a41e6eea6f3c0dcd60f685d7fa560eef408355720f195b6f32d3d28b1c55f4522e976a22b634d8320fee7d2912626e1b606063ba7bf0aa405a32e90ec7c17a00569f9ad0366d64cca68e49dbc0b1f49e12b34a5c3de280550b40337a52bd9ecdf691461508c18449b29e3dc966b445b0b901af65be77c94cbb8407c4dfe3418b6606a0eac5914b5eaf1b8171b3bf5275ccc5ff9ec16fc6baab86de093022b0596e95a5d96c756e6c5a1e1e60747a29ba36ccda1110cddf6b99eae5d32d3bb3422f1993967e6d3e9d981ce072dfcf2e633d2b92eb1582de162680b1394d571ea3efbeda47c4ef9895f76c1fffb815635a38555226abba20fee790724f1250f550f14ff2e26ae6773cb7c6839ea1ad81e8e48ef988724a19d16803707257bf416e9a2b79e52fb325248f4d06db0ce42bd3b29b45a1a7a43d5ac68a4067eae64aa7e9a290a770122ddaca1e0153d1d8d432dd6fc41bdc3a9c823d17ea1f133333cd19744f7d4979eef83a7442e36b135a6962b2faf5ed6fafabeebf028082412b47007291b4c70451e1fc54e4678049788cdb50180e5475a331a35aa41c71e31146c4666a83615d197bb341ea876ac15edea031e129fe9dcd79806b950ab9d04faad291bbaf05af0d17ab77671724c1f986b502ab9580e150a22bd32a522cc0569080682a965b9ff65f83b5ed0c5015d48e1e132cea8fc1c8aca4e0d3bec3c18e6be2b130b4030a5a9a6f0635481a8c0827e7172866b27787014aec400d13fd52fe472cc94d5fbf74a3f8c48809c3d77cb8fb7208419611def2cc18c6dc366496cf93ae0d6faa94a96480d199977e8430a9bdcd23a1a7ede27f4e212a6dacee7a946df28acc72fbe502076ca7837e63df63791580f756fffb4ae3c829360203ef17e65d062861990525d54ccf2ec071eb6674a3da10b72157fd2b3f6e7f2ad4b0e92ad9a8d6438d9cb6f0244f96e9399a665cc791afb6f44db946830cf33c476e3c944e31617b09066951b7e16bb163bd67bd2f962eb54a8459e1dd613344df90874eb9e152323d77abf28fd48b83bfa0614ff73b1944bc0d768e90afb4d78cf6ade91106321b776b1dbdc5728d31a1389cbe70a346e3f576678bfafef076889e90f19d093fc2e5564ce14bf1555bd218de372f80b7524735525b1fe52af387af8294cb903b5ce31905689a06de53163a9a18d66aad2ea7d71ecd7bfbf2e97ff6c0a69e292475af5472abe38e74be02eb88279cf56f5f34e227dd89d2315a8da0819b05e0d5caf2ca0bc5b535d50a41af05ea970382cf7fc3e405360ff1ccf13cf210a915d6ef49878e2c73fcb02cd04caef54a583dffc6d0b9a49615001ee28d07e91257e0eb75842242605f434b9782fbf3ad4010b8c39943515457b6d278e3311f930a2f8d065ab750f709483a7d74ddbdd1b3d766be336460a5ff2565bd17b3b0e7d005baef6086e8ff909d183c495da5fbfa94658c414bff0ff3bd44c7265cf63a86c988e00a9bdf6e4719d83686b034198cd9c8506531bcdc96843ad08ed45c77f0685ca9b27494e30d46018dab503802e59dc8b10de6160dba438904a2047e83064543c3799c0792da4da007afb218999b09f3f57fda742e564acd7f9ac25526348bc23dcf93a9ce181324df479083ad173c22ca465de1b53aaffccf1f375138a
Deals:        [355473 356644 358468 0 0 0 0 0 0 360851 0 364628 364867 364948 0 364993 365420 365542 0 365811 0 366176 366425 366437 0 0 0 0 366646 369038 0 0 369641 0 0 0 0]
Retries:  239
--------
Event Log:
0.    2020-10-01 17:09:30 +0100 BST:  [event;sealing.SectorStart] {"User":{"ID":129,"SectorType":3}}
1.    2020-10-01 17:09:40 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":536870912,"PieceCID":{"/":"baga6ea4seaqmp62gwjcgh2mxnvjriw4ugiti4g5mdntdd2mhybsgqj75aqvgegi"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacecxim6ysvg2kfk6ikz5i35w6mwleegz6f6vhjotfxfhodybw7ascq"},"DealID":355473,"DealSchedule":{"StartEpoch":117360,"EndEpoch":638369},"KeepUnsealed":false}}}}
2.    2020-10-01 17:57:10 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":536870912,"PieceCID":{"/":"baga6ea4seaqjug4kejx6ftg46qasy427cgbtdjoxqi5hvptvkrffbik72t7mgjy"}},"DealInfo":{"PublishCid":{"/":"bafy2bzaceaahl4o2kknslqzwam4rcyi6uxdaq5bytmhfhsdmi7z5kdx2nsbes"},"DealID":356644,"DealSchedule":{"StartEpoch":117457,"EndEpoch":638369},"KeepUnsealed":true}}}}
3.    2020-10-01 18:40:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":131072,"PieceCID":{"/":"baga6ea4seaqczvq3kgxg6p2rh7fmwvyfwvnhfxpbmzggg7peijl6ipull32hmba"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacebnsktqldoj5c4msz3lylwfctgrbrmae6qcr2doslalhffttw5vqu"},"DealID":358468,"DealSchedule":{"StartEpoch":111788,"EndEpoch":690209},"KeepUnsealed":true}}}}
4.    2020-10-01 19:36:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":131072,"PieceCID":{"/":"baga6ea4seaqi5hreap5iqthwen7wbxzf7a7oidokt3mht23pmnjncuee6wwq2py"}},"DealInfo":null}}}
5.    2020-10-01 19:36:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":262144,"PieceCID":{"/":"baga6ea4seaqhklmwsp5bm5jehfkhnyyxvgcyb4aji6x3piyfidlclkjjdtasuby"}},"DealInfo":null}}}
6.    2020-10-01 19:36:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":524288,"PieceCID":{"/":"baga6ea4seaqhaixwb57pnlp2c4ixuutbtyym5kbmnadvvxy4mz3yn3cqn3xs2gi"}},"DealInfo":null}}}
7.    2020-10-01 19:36:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":1048576,"PieceCID":{"/":"baga6ea4seaqntgehxfzvoouw4ejzgzcsg3axwh2moa2noi6hvgpxbg5u3jqrmky"}},"DealInfo":null}}}
8.    2020-10-01 19:36:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":2097152,"PieceCID":{"/":"baga6ea4seaqnbnjq3oylj4s4luxsukg752aiwu2bfibjghyyysm7lisubbvrgjq"}},"DealInfo":null}}}
9.    2020-10-01 19:36:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":4194304,"PieceCID":{"/":"baga6ea4seaqijqccdoqgqwqbx54vui2eazh6ijf5kku5eq3xwokp6tclivuoqei"}},"DealInfo":null}}}
10.   2020-10-01 19:36:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":8388608,"PieceCID":{"/":"baga6ea4seaqpym6sxmmlggt5us5axh4xxgwqzqaawsta2vwn5sa2zhottizosni"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacecmun4sybqdjqo6d4izxq6u4mrqdimazq6lpb5xed6f2g6b3my5xk"},"DealID":360851,"DealSchedule":{"StartEpoch":117660,"EndEpoch":638369},"KeepUnsealed":false}}}}
11.   2020-10-01 20:51:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":16777216,"PieceCID":{"/":"baga6ea4seaqkejdvbaufquewln7dgszre6ymaqvr2bdnyvcaee3we7gypgoocoq"}},"DealInfo":null}}}
12.   2020-10-01 20:51:31 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":33554432,"PieceCID":{"/":"baga6ea4seaqprgnhojniowmit7d7yjlxkga2ql27rxeisau7yx3bshbdxjaasaq"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacebxetqitpogfr6xbqlt5ydodrpewm642wbrj3kqugf5j5c3s7tmvw"},"DealID":364628,"DealSchedule":{"StartEpoch":117810,"EndEpoch":638369},"KeepUnsealed":false}}}}
13.   2020-10-01 20:56:31 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":67108864,"PieceCID":{"/":"baga6ea4seaqild2rn2eyaor6nrgsqzbqjq3wiu2bdm3vozpeupxseeeq7uyw4ja"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacedzyoomhlk6sfkmkptiay3qp225cqkasozjjyxzp5qnitp3tcvoci"},"DealID":364867,"DealSchedule":{"StartEpoch":117816,"EndEpoch":638369},"KeepUnsealed":false}}}}
14.   2020-10-01 20:58:01 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":67108864,"PieceCID":{"/":"baga6ea4seaqg5n7fss5j4wk4cy6fnfa2u56yqoaamfbxlwepkwr34xu5kisbgei"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacec7bbcth3rohqfrcmllga436c7iktc47xwawgps5qh2oincmwb75y"},"DealID":364948,"DealSchedule":{"StartEpoch":117816,"EndEpoch":638369},"KeepUnsealed":false}}}}
15.   2020-10-01 21:00:01 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":67108864,"PieceCID":{"/":"baga6ea4seaqnsqov4dldcsuzlqz77pkpxzurddlt2ts72lgtd4hxzbxl3ukoobq"}},"DealInfo":null}}}
16.   2020-10-01 21:00:06 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":268435456,"PieceCID":{"/":"baga6ea4seaqey2q3m74ni5aaqphl33ij2seafe7zzjcaox6tvlbyxewts6reakq"}},"DealInfo":{"PublishCid":{"/":"bafy2bzaceclkayh7m2e5y2h5p77xdjvx45kfmr55eibeaf4tvzphericsx74e"},"DealID":364993,"DealSchedule":{"StartEpoch":117816,"EndEpoch":638369},"KeepUnsealed":false}}}}
17.   2020-10-01 21:07:05 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":268435456,"PieceCID":{"/":"baga6ea4seaqbsyrqdt2hve6zpjkgt23uw5ek7btgdgqxkiw5re6afof56u5ncha"}},"DealInfo":{"PublishCid":{"/":"bafy2bzaceakhuglzewfqdbezbouoixfijsbisyczu7jipk6es2i7c3gqwnv76"},"DealID":365420,"DealSchedule":{"StartEpoch":117819,"EndEpoch":638369},"KeepUnsealed":false}}}}
18.   2020-10-01 21:09:31 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":33554432,"PieceCID":{"/":"baga6ea4seaqpel344nqupnacxxjyuthlbbarvttu3g3kbmo6yepb7r7i7ripaaq"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacea2vy3qp4hliqshfj2fiukgonfzfy3l7lyfei6ci73hvwrvyta3aa"},"DealID":365542,"DealSchedule":{"StartEpoch":117821,"EndEpoch":638369},"KeepUnsealed":false}}}}
19.   2020-10-01 21:14:31 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":33554432,"PieceCID":{"/":"baga6ea4seaqnv7nlnwutmrctyjwtg4tlt7x6gq56r6awjhwabgvnh6x7kbqxkca"}},"DealInfo":null}}}
20.   2020-10-01 21:14:32 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":67108864,"PieceCID":{"/":"baga6ea4seaqonzpzmpxt2ictyy3ci3z24tt2hhgtmn2pipriikz4l7jo753x2oq"}},"DealInfo":{"PublishCid":{"/":"bafy2bzaceb5lfrqrpwksavd23hajel65ilil74n5qjv3bnnbdj2uorolqpw3a"},"DealID":365811,"DealSchedule":{"StartEpoch":117821,"EndEpoch":638369},"KeepUnsealed":false}}}}
21.   2020-10-01 21:23:32 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":134217728,"PieceCID":{"/":"baga6ea4seaqfctcdlq6qju2juu3f7pkz77drgyurcf4fteobupctv4rapf2buly"}},"DealInfo":null}}}
22.   2020-10-01 21:23:37 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":268435456,"PieceCID":{"/":"baga6ea4seaqmx73krcxoiq4l6o2nhbglpfak3lj57jtk2rk7udpmz4sk37yxmni"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacebgpxjvtqwfxdohmiknwradseo5minyrjhmqlj5exansg5scpkfkg"},"DealID":366176,"DealSchedule":{"StartEpoch":117821,"EndEpoch":638369},"KeepUnsealed":false}}}}
23.   2020-10-01 21:28:06 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":268435456,"PieceCID":{"/":"baga6ea4seaqhilnuahhhm24rd5annv5lwpomglb3hkramjutayn4mvuvpn5z4oy"}},"DealInfo":{"PublishCid":{"/":"bafy2bzaceb4ujryrp66vx4otaum3acttmybqlxcw4efobf5bfa7ptifka2vze"},"DealID":366425,"DealSchedule":{"StartEpoch":117821,"EndEpoch":638369},"KeepUnsealed":false}}}}
24.   2020-10-01 21:29:00 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":16777216,"PieceCID":{"/":"baga6ea4seaqayknvgzwltlo7jdhqamu5usy4dsmnuu6qchrltugmzrdktuznula"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacedlkfns5jrk7nn5q2oeymgvorjlf2hndo3ncmmq2dftipkw7d26ls"},"DealID":366437,"DealSchedule":{"StartEpoch":117821,"EndEpoch":638369},"KeepUnsealed":false}}}}
25.   2020-10-01 21:34:30 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":16777216,"PieceCID":{"/":"baga6ea4seaqkejdvbaufquewln7dgszre6ymaqvr2bdnyvcaee3we7gypgoocoq"}},"DealInfo":null}}}
26.   2020-10-01 21:34:31 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":33554432,"PieceCID":{"/":"baga6ea4seaqnv7nlnwutmrctyjwtg4tlt7x6gq56r6awjhwabgvnh6x7kbqxkca"}},"DealInfo":null}}}
27.   2020-10-01 21:34:32 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":67108864,"PieceCID":{"/":"baga6ea4seaqnsqov4dldcsuzlqz77pkpxzurddlt2ts72lgtd4hxzbxl3ukoobq"}},"DealInfo":null}}}
28.   2020-10-01 21:34:34 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":134217728,"PieceCID":{"/":"baga6ea4seaqfctcdlq6qju2juu3f7pkz77drgyurcf4fteobupctv4rapf2buly"}},"DealInfo":null}}}
29.   2020-10-01 21:34:38 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":268435456,"PieceCID":{"/":"baga6ea4seaqiji3lefaraasym5hgkg22xt42m5aevnulecqzwnzvqbiuadapyhy"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacec757ghnw55mfovawcbn5nr23o2ra3xssm6mkiooywa4gysyb53tg"},"DealID":366646,"DealSchedule":{"StartEpoch":117821,"EndEpoch":638369},"KeepUnsealed":false}}}}
30.   2020-10-01 22:40:05 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":268435456,"PieceCID":{"/":"baga6ea4seaqfj6u3xqkpvwv3zwxebpbkss5dfblk7kpqcrzp6yznzjubnlvasji"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacea4ooiizso3jfraagfapwio2hrlzlk7m7tl4nmlc6v3sgun6bh5eq"},"DealID":369038,"DealSchedule":{"StartEpoch":117821,"EndEpoch":638369},"KeepUnsealed":false}}}}
31.   2020-10-01 23:16:35 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":268435456,"PieceCID":{"/":"baga6ea4seaqk2bufhfu5g7ju74eobh2wsmfevum2rhppmdf75z7b2m4byhtryny"}},"DealInfo":null}}}
32.   2020-10-01 23:16:42 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":536870912,"PieceCID":{"/":"baga6ea4seaqdsvqopmj2soyhujb72jza76t4wpq5fzifvm3ctz47iyytkewnubq"}},"DealInfo":null}}}
33.   2020-10-01 23:17:01 +0100 BST:  [event;sealing.SectorAddPiece]  {"User":{"NewPiece":{"Piece":{"Size":1073741824,"PieceCID":{"/":"baga6ea4seaqlxcmuvnhcxwlfeob2664f6bf5stglbv5odtg75p736dwfnvod6nq"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacecxi6abdlnqomxvpsj3ggqxn3fbske36bdaqmnaztw563dtlplw46"},"DealID":369641,"DealSchedule":{"StartEpoch":117913,"EndEpoch":638369},"KeepUnsealed":false}}}}
34.   2020-10-02 00:21:35 +0100 BST:  [event;sealing.SectorStartPacking]  {"User":{}}
35.   2020-10-02 00:28:53 +0100 BST:  [event;sealing.SectorPacked]    {"User":{"FillerPieces":[{"Size":1073741824,"PieceCID":{"/":"baga6ea4seaqmzq6acl23axubdiv37xipnaz3qqtvwr57ekoaauviescphqnfwpi"}},{"Size":2147483648,"PieceCID":{"/":"baga6ea4seaqh34u3nf3tdgpi6k2aw54rtucikcpo25uofrzjpmprinydj7b4mla"}},{"Size":8589934592,"PieceCID":{"/":"baga6ea4seaqnqyicdbbfvnpjlmokmi45fgroiigxa2uw6nz6f6ojveoxlhizwai"}},{"Size":17179869184,"PieceCID":{"/":"baga6ea4seaqg2nsld34emra2ljfgrbrdcswmbjdpaftrpzjuipudt3w7qpbikpa"}}]}}
36.   2020-10-02 05:28:23 +0100 BST:  [event;sealing.SectorPreCommit1]    {"User":{"PreCommit1Out":"eyJyZWdpc3RlcmVkX3Byb29mIjoiU3RhY2tlZERyZzMyR2lCVjEiLCJsYWJlbHMiOnsiU3RhY2tlZERyZzMyR2lCVjEiOnsibGFiZWxzIjpbeyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItMSIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItMiIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItMyIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItNCIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItNSIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItNiIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItNyIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItOCIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItOSIsInNpemUiOjEwNzM3NDE4MjQsInJvd3NfdG9fZGlzY2FyZCI6N30seyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoibGF5ZXItMTAiLCJzaXplIjoxMDczNzQxODI0LCJyb3dzX3RvX2Rpc2NhcmQiOjd9LHsicGF0aCI6Ii9tbnQvbnZtZTBuMS8ubG90dXNtaW5lci9jYWNoZS9zLXQwODQwMy0xMjkiLCJpZCI6ImxheWVyLTExIiwic2l6ZSI6MTA3Mzc0MTgyNCwicm93c190b19kaXNjYXJkIjo3fV0sIl9oIjpudWxsfX0sImNvbmZpZyI6eyJwYXRoIjoiL21udC9udm1lMG4xLy5sb3R1c21pbmVyL2NhY2hlL3MtdDA4NDAzLTEyOSIsImlkIjoidHJlZS1kIiwic2l6ZSI6MjE0NzQ4MzY0Nywicm93c190b19kaXNjYXJkIjo3fSwiY29tbV9kIjpbNzgsNzksMjE0LDIxNiw4MiwxNDQsMTkxLDM0LDE3OCw5MywyMTcsMTA3LDEwMCw2MCwyNTIsMjUzLDIzMCwxNzksMTc1LDExOCwxMDcsMTUsMTk1LDYzLDUyLDIxMCwxNTgsMTA1LDIzMywxNDIsMjYsNDddfQ==","TicketValue":"QnJq82Ae5+rnvbcipa/Ale2GkLxPYvrrjON08IpjW6Y=","TicketEpoch":108717}}
37.   2020-10-02 06:18:07 +0100 BST:  [event;sealing.SectorPreCommit2]    {"User":{"Sealed":{"/":"bagboea4b5abcbznott2nyuwn5yn23ylmxjrko5ts2dlm3jtxm5ydtbedzygkddlj"},"Unsealed":{"/":"baga6ea4seaqe4t6w3bjjbpzcwjo5s23eht6p3zvtv53gwd6dh42nfhtj5ghbuly"}}}
38.   2020-10-02 06:18:08 +0100 BST:  [event;sealing.SectorPreCommitted]  {"User":{"Message":{"/":"bafy2bzacecbshuledex32aqh33kgfszottcd7xsqv5ci2bfwis737hbxvliqc"},"PreCommitDeposit":"299095700588327276","PreCommitInfo":{"SealProof":3,"SectorNumber":129,"SealedCID":{"/":"bagboea4b5abcbznott2nyuwn5yn23ylmxjrko5ts2dlm3jtxm5ydtbedzygkddlj"},"SealRandEpoch":108717,"DealIDs":[355473,356644,358468,360851,364628,364867,364948,364993,365420,365542,365811,366176,366425,366437,366646,369038,369641],"Expiration":1631969,"ReplaceCapacity":true,"ReplaceSectorDeadline":0,"ReplaceSectorPartition":0,"ReplaceSectorNumber":83}}}
39.   2020-10-02 06:21:38 +0100 BST:  [event;sealing.SectorPreCommitLanded]   {"User":{"TipSet":"AXGg5AIgG6OaDlNqSV+IiUF2WXnv3BHzomK9yggV9jgJMCfPRn8BcaDkAiCIoNhVsVJYfOUfj2jrNtq7ceXHWoAPsqcD1M876vbjXAFxoOQCIBM/SiRP2rqCwyvBVe1XxqiPR/BiDeAlaFGWio+Sckn0"}}
40.   2020-10-02 07:36:30 +0100 BST:  [event;sealing.SectorSeedReady] {"User":{"SeedValue":"vi+x/trZOQnp3zIXIpbEGgaInqOATRWz09EOj5x7Y4A=","SeedEpoch":110467}}
41.   2020-10-02 09:05:29 +0100 BST:  [event;sealing.SectorCommitted] {"User":{"Proof":"rVDPL3SQt+Z51vHbC+1MwBod8mZD6M7qeuTlh4U6N9+qEup8XY+nccgJr65k4Fo2jP+1+aH2YidwQHLyEYS/tLXFghALuiF0s0Q8e2j+W2DTA3Sg9oSgOSL+UhfscMBwCslKt6BiNuPbuIDZVzOjCKX2RM4d4ks5+/29v8LtUmABEQtHmn0z3CE8vVDyHfanr6mfCQfc38wAlKkVFgr4A0QQtl+cYcv2+mFGDXdq4SOrl9C27ho2Y7h9bCECDoo+jRRTmANGQ/t0cqG4nr45Ufh8NUB30tG6dDYcHBxKLFF4p+pZvR6hWihTvw1kWtZiqDuuLRG+o6KIsotkbQ3iMAxOywg558/kVaRqptVt99tmoVAoZCdz71QRog0DnK9JFJLkVIzjZS+bsdNcgZAIQg0hV257rPRSna6xnjwkXQcjoKvVztBxA4q6G6YanemQsDLw+4OvicmCBv5SatHL/HhbXSIJFtbk/QuaHmL9R2K8rIgyD9dWnwdFYAa3YfuSqC6Ktj7W45PLNrziVeMTDK6GcoeZAya+aEyuGpEMKexF//fk/VZsWfBHD9neI17CiwU7KaC9LOs16SMaHph2WOP7+yryIi6yCXN0aT5Ht0rnKEYcj0WSt+srkQbNPlLjAdovu2hsSaXQAvf2d3I/5KRF94yqN3173WGNeRkUgFps66jzXQVyBMUEoKTvbFXsoMoBZEIzPmRpdOq8WzjOYj09LG1KCI8sBMkD/C3i4f+N31PC7nNoiQ5XyIcHW4Lbufl5g1yDtvGz3xRT4BdofJrPnoBIrp+TAc4ghjg8AlsBxP66KmyXqvaq93KJJcWmtDKpuDz+ya8lC+x9P1UtylKsusr8ue2NGbvcnzs9i/pYcupcXu2GkSEhZAWZj///C006lTNE193WNnWyhYqxQ2ukwMig0pv9FAwDq5XY+8OjCvBtvx8qyK+L8J5w0OJ9lppLpivXWgEtWaKF33pB5u6m88Dc1g9oXX+lYO70CDVXIPGVtvMtPSixxV9FIul2oitjTYMg/ufSkSYm4bYGBjunvwqkBaMukOx8F6AFafmtA2bWTMpo5J28Cx9J4Ss0pcPeKAVQtAM3pSvZ7N9pFGFQjBhEmynj3JZrRFsLkBr2W+d8lMu4QHxN/jQYtmBqDqxZFLXq8bgXGzv1J1zMX/nsFvxrqrht4JMCKwWW6VpdlsdW5sWh4eYHR6Kbo2zNoREM3fa5nq5dMtO7NCLxmTln5tPp2YHOBy388uYz0rkusVgt4WJoCxOU1XHqPvvtpHxO+YlfdsH/+4FWNaOFVSJqu6IP7nkHJPElD1UPFP8uJq5nc8t8aDnqGtgejkjvmIckoZ0WgDcHJXv0Fumit55S+zJSSPTQbbDOQr07KbRaGnpD1axopAZ+rmSqfpopCncBIt2soeAVPR2NQy3W/EG9w6nII9F+ofEzMzzRl0T31Jee74OnRC42sTWmlisvr17W+vq+6/AoCCQStHAHKRtMcEUeH8VORngEl4jNtQGA5UdaMxo1qkHHHjEUbEZmqDYV0Ze7NB6odqwV7eoDHhKf6dzXmAa5UKudBPqtKRu68Frw0Xq3dnFyTB+Ya1AquVgOFQoivTKlIswFaQgGgqlluf9l+Dte0MUBXUjh4TLOqPwcispODTvsPBjmvisTC0Awpamm8GNUgajAgn5xcoZrJ3hwFK7EANE/1S/kcsyU1fv3Sj+MSICcPXfLj7cghBlhHe8swYxtw2ZJbPk64Nb6qUqWSA0ZmXfoQwqb3NI6Gn7eJ/TiEqbazuepRt8orMcvvlAgdsp4N+Y99jeRWA91b/+0rjyCk2AgPvF+ZdBihhmQUl1UzPLsBx62Z0o9oQtyFX/Ss/bn8q1LDpKtmo1kONnLbwJE+W6TmaZlzHka+29E25RoMM8zxHbjyUTjFhewkGaVG34WuxY71nvS+WLrVKhFnh3WEzRN+Qh0654VIyPXer8o/Ui4O/oGFP9zsZRLwNdo6Qr7TXjPat6REGMht3ax29xXKNMaE4nL5wo0bj9XZni/r+8HaInpDxnQk/wuVWTOFL8VVb0hjeNy+At1JHNVJbH+Uq84evgpTLkDtc4xkFaJoG3lMWOpoY1mqtLqfXHs17+/Lpf/bApp4pJHWvVHKr4450vgLriCec9W9fNOIn3YnSMVqNoIGbBeDVyvLKC8W1NdUKQa8F6pcDgs9/w+QFNg/xzPE88hCpFdbvSYeOLHP8sCzQTK71Slg9/8bQuaSWFQAe4o0H6RJX4Ot1hCJCYF9DS5eC+/OtQBC4w5lDUVRXttJ44zEfkwovjQZat1D3CUg6fXTdvdGz12a+M2Rgpf8lZb0Xs7Dn0AW672CG6P+QnRg8SV2l+/qUZYxBS/8P871ExyZc9jqGyYjgCpvfbkcZ2DaGsDQZjNnIUGUxvNyWhDrQjtRcd/BoXKmydJTjDUYBjatQOALlncixDeYWDbpDiQSiBH6DBkVDw3mcB5LaTaAHr7IYmZsJ8/V/2nQuVkrNf5rCVSY0i8I9z5OpzhgTJN9HkIOtFzwiykZd4bU6r/zPHzdROK"}}
42.   2020-10-02 09:05:29 +0100 BST:  [event;sealing.SectorCommitSubmitted]   {"User":{"Message":{"/":"bafy2bzaceb6x4m3k7n5zmnsg3p2o43vsek5p4szg4rwzemxwlya5cvklslj34"}}}
43.   2020-10-02 09:08:30 +0100 BST:  [event;sealing.SectorCommitFailed]  {"User":{}}
  proof validation failed, sector not found in sector set after cron
44.   2020-10-02 09:09:30 +0100 BST:  [event;sealing.SectorRetryComputeProof] {"User":{}}
45.   2020-10-02 09:09:30 +0100 BST:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
46.   2020-10-02 09:09:30 +0100 BST:  [event;sealing.SectorCommitFailed]  {"User":{}}
  proof validation failed, sector not found in sector set after cron
47.   2020-10-02 09:10:30 +0100 BST:  [event;sealing.SectorRetryComputeProof] {"User":{}}
48.   2020-10-02 09:10:30 +0100 BST:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
49.   2020-10-02 09:10:30 +0100 BST:  [event;sealing.SectorCommitFailed]  {"User":{}}
  proof validation failed, sector not found in sector set after cron
50.   2020-10-02 09:11:30 +0100 BST:  [event;sealing.SectorRetryComputeProof] {"User":{}}
51.   2020-10-02 09:11:30 +0100 BST:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
52.   2020-10-02 09:11:30 +0100 BST:  [event;sealing.SectorCommitFailed]  {"User":{}}
  proof validation failed, sector not found in sector set after cron
53.   2020-10-02 09:12:30 +0100 BST:  [event;sealing.SectorRetryComputeProof] {"User":{}}
54.   2020-10-02 09:12:30 +0100 BST:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
55.   2020-10-02 09:12:30 +0100 BST:  [event;sealing.SectorCommitFailed]  {"User":{}}
  proof validation failed, sector not found in sector set after cron
56.   2020-10-02 09:13:30 +0100 BST:  [event;sealing.SectorRetryComputeProof] {"User":{}}
57.   2020-10-02 09:13:30 +0100 BST:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
58.   2020-10-02 09:13:30 +0100 BST:  [event;sealing.SectorCommitFailed]  {"User":{}}

@yablu thanks for reporting!

Reiers commented 4 years ago

If you have this problem you can try: lotus-miner sectors update-state --really-do-it <sector> FinalizeSector Let the miner fetch sectors from workers, and restart the miner. Now they should be proving and active. It did work for us at least.

Daemon:  0.9.1+git.1401efb1+api0.16.0
Local: lotus version 0.9.1+git.1401efb1
askender commented 3 years ago

if SectorCommitFailed occur: lotus-miner sectors update-state --really-do-it <sector> FinalizeSector will result error if the committing msg is not online. lotus-miner sectors update-state --really-do-it <sector> Committing will help.

dispensable1988 commented 3 years ago

Same issue lotus-miner version 1.10.1 `98. 2021-09-18 02:07:21 +0000 UTC: [event;sealing.SectorRetrySubmitCommit] {"User":{}}

  1. 2021-09-18 02:07:21 +0000 UTC: [event;sealing.SectorSubmitCommitAggregate] {"User":{}}
  2. 2021-09-18 02:09:11 +0000 UTC: [event;sealing.SectorCommitAggregateSent] {"User":{"Message":{"/":"bafy2bzacebqopyifprfpxbfttjyhlj5dnof74gp4ujxww77yxlqytrermx6qe"}}}
  3. 2021-09-18 02:11:00 +0000 UTC: [event;sealing.SectorNoDealCommitFailed] {"User":{}} proof validation failed, sector not found in sector set after cron: %!w()`
supriya-premkumar commented 2 years ago

I am having the same issue with version 1.13.2

j1v commented 2 years ago

I'm also running into that with 1.13.2-rc4

Reiers commented 2 years ago

So after heavy repro on issue for past week. I manage to get 6 sectors in this state.

To get to this point, I needed to;

The sectors end up in a endless loop, and no matter what update-state I threw at it - it didn't stop. In the span of 12 hours the kvlog/metadata had grown to 1,1 TB - and thats with 6 sectors on constant loop. (If it filled up the whole /miner/path disk, it will not kill the miner process but it will corrupt the .log files and get unresponsive)

So the only way I found out of this was to lotus-miner sectors remove --really-do-it <sectornumb> (Keep in mind I had to run mix-matched lotus version so there might be a state that works)

5770.   2022-03-11 14:47:33 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
5771.   2022-03-11 14:47:33 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
5772.   2022-03-11 14:47:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
5773.   2022-03-11 14:47:34 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
5774.   2022-03-11 14:47:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
5775.   2022-03-11 14:47:34 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
5776.   2022-03-11 14:47:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
5777.   2022-03-11 14:47:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
5778.   2022-03-11 14:47:35 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
5779.   2022-03-11 14:47:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
5780.   2022-03-11 14:47:35 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
5781.   2022-03-11 14:47:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron

So proof validation failed, sector not found in sector set after cron is a issue on the latest release.

So we have 2 options:

  1. Figure out if its recoverable. Lotus sends it to a back to a working state and not in constant SectorRetryCommitWait loop
  2. Worst case - have Lotus remove automatically it since the constant loop will kill the miner process within 24/h due to lack of space and corrupted .log files. Sure you can recover from this, but that might result in more lost sectors.

Like I stated in the start - it took some doing getting these sector to this state, so you are must be really unlucky to get sectors in this state, it was 6 out of 16 sectors that ended in this state. The best practices while we look at this issue:

( This was reproduced on CC and Deal sectors)

Reiers commented 2 years ago

Update:

Log:

2022-03-11T22:24:19.790+0100    WARN    sectors storage-sealing/fsm.go:749  sector 48025 got error event sealing.SectorCommitFailed: proof validation failed, sector not found in sector set after cron

Cmd entered: lotus-miner sectors update-state --really-do-it 48025 PreCommit1 Log result:

2022-03-11T22:25:39.195+0100    DEBUG   advmgr  sector-storage/sched_worker.go:280  task done   {"workerid": "233cab9f-d84c-4f57-a04e-44f7b6078429"}
2022-03-11T22:25:39.196+0100    DEBUG   advmgr  sector-storage/sched.go:356 SCHED 0 queued; 32 open windows
2022-03-11T22:25:39.322+0100    INFO    stores  stores/http_handler.go:158  SERVE DELETE /remote/sealed/s-t022352-48025
2022-03-11T22:25:39.328+0100    INFO    stores  stores/local.go:639 remove /nfs/TechHedgeUnit4/sealed/s-t022352-48025
2022-03-11T22:25:41.287+0100    INFO    stores  stores/http_handler.go:158  SERVE DELETE /remote/cache/s-t022352-48025
2022-03-11T22:25:41.287+0100    INFO    stores  stores/local.go:639 remove /nfs/TechHedgeUnit4/cache/s-t022352-48025

Sector --log: this was in matter of seconds - look at all the events. 

lotus-miner sectors status --log 48025
SectorID:   48025
Status:     PreCommit1
CIDcommD:   baga6ea4seaql6eiei7fnpyeg4ayizqsr5pl5a5ageauezlpqhd4c5p3fgzj4slq
CIDcommR:   bagboea4b5abcaiyd6l5almuvzrqyfouukq4neljp2kiebiofddwba3bygjicmw2p
Ticket:     a34d11f858026a9e25b88e720671d09e49aeeb2de9769567a016965473237e68
TicketH:    1622531
Seed:       77c14076d8ffff55e8814752c6daf84a01adb713b182d9685dd3299e82e15189
SeedH:      1624186
Precommit:  bafy2bzacedt2e5h6fmptjoioc3tqig3j2yjs7vw3tan4awkxrmkflfcivp52w
Commit:     bafy2bzaceah4mv7ipx6kbc7q2pknjk54puvt7inozhm3y2wxnsnwlgatitoki
Deals:      [4434491]
Retries:    0
--------
Event Log:
0.  2022-03-11 15:30:15 +0100 CET:  [event;sealing.SectorStart] {"User":{"ID":48025,"SectorType":8}}
1.  2022-03-11 15:30:15 +0100 CET:  [event;sealing.SectorAddPiece]  {"User":{}}
2.  2022-03-11 15:35:38 +0100 CET:  [event;sealing.SectorPieceAdded]    {"User":{"NewPieces":[{"Piece":{"Size":34359738368,"PieceCID":{"/":"baga6ea4seaql6eiei7fnpyeg4ayizqsr5pl5a5ageauezlpqhd4c5p3fgzj4slq"}},"DealInfo":{"PublishCid":{"/":"bafy2bzacedra72cf72uuolexqoem3ojcqfzerw5cozs4evxf4khnba6x6uaii"},"DealID":4434491,"DealProposal":{"PieceCID":{"/":"baga6ea4seaql6eiei7fnpyeg4ayizqsr5pl5a5ageauezlpqhd4c5p3fgzj4slq"},"PieceSize":34359738368,"VerifiedDeal":true,"Client":"f1wdxdpqh3hirrhp353i4o6ld7bsw6evh3v7i5jtq","Provider":"f022352","Label":"mAXCg5AIgxDhpqNLYLZFKu9OaLkSaBTzIpfx02r/AU32kUdI7PJs","StartEpoch":1628442,"EndEpoch":2150950,"StoragePricePerEpoch":"0","ProviderCollateral":"6594327140982025","ClientCollateral":"0"},"DealSchedule":{"StartEpoch":1628442,"EndEpoch":2150950},"KeepUnsealed":true}}]}}
3.  2022-03-11 15:35:38 +0100 CET:  [event;sealing.SectorStartPacking]  {"User":{}}
4.  2022-03-11 15:35:38 +0100 CET:  [event;sealing.SectorPacked]    {"User":{"FillerPieces":null}}
5.  2022-03-11 15:35:38 +0100 CET:  [event;sealing.SectorTicket]    {"User":{"TicketValue":"o00R+FgCap4luI5yBnHQnkmu6y3pdpVnoBaWVHMjfmg=","TicketEpoch":1622531}}
6.  2022-03-11 20:05:51 +0100 CET:  [event;sealing.SectorPreCommit1]    {"User":{"PreCommit1Out":"eyJfbG90dXNfU2VhbFJhbmRvbW5lc3MiOiJvMDBSK0ZnQ2FwNGx1STV5Qm5IUW5rbXU2eTNwZHBWbm9CYVdWSE1qZm1nPSIsImNvbW1fZCI6WzE5MSwxNyw0LDcxLDIwMiwyMTUsMjI0LDEzNCwyMjQsNDgsMTQwLDE5NCw4MSwyMzUsMjE1LDIwOCwxMTYsNiwzMiw0MCw3NiwxNzMsMjQwLDU2LDI0OCw0NiwxOTEsMTAxLDU0LDgzLDIwMSw0Nl0sImNvbmZpZyI6eyJpZCI6InRyZWUtZCIsInBhdGgiOiIvbW50L1BjMVN0b3JlL2NhY2hlL3MtdDAyMjM1Mi00ODAyNSIsInJvd3NfdG9fZGlzY2FyZCI6Nywic2l6ZSI6MjE0NzQ4MzY0N30sImxhYmVscyI6eyJTdGFja2VkRHJnMzJHaUJWMSI6eyJfaCI6bnVsbCwibGFiZWxzIjpbeyJpZCI6ImxheWVyLTEiLCJwYXRoIjoiL21udC9QYzFTdG9yZS9jYWNoZS9zLXQwMjIzNTItNDgwMjUiLCJyb3dzX3RvX2Rpc2NhcmQiOjcsInNpemUiOjEwNzM3NDE4MjR9LHsiaWQiOiJsYXllci0yIiwicGF0aCI6Ii9tbnQvUGMxU3RvcmUvY2FjaGUvcy10MDIyMzUyLTQ4MDI1Iiwicm93c190b19kaXNjYXJkIjo3LCJzaXplIjoxMDczNzQxODI0fSx7ImlkIjoibGF5ZXItMyIsInBhdGgiOiIvbW50L1BjMVN0b3JlL2NhY2hlL3MtdDAyMjM1Mi00ODAyNSIsInJvd3NfdG9fZGlzY2FyZCI6Nywic2l6ZSI6MTA3Mzc0MTgyNH0seyJpZCI6ImxheWVyLTQiLCJwYXRoIjoiL21udC9QYzFTdG9yZS9jYWNoZS9zLXQwMjIzNTItNDgwMjUiLCJyb3dzX3RvX2Rpc2NhcmQiOjcsInNpemUiOjEwNzM3NDE4MjR9LHsiaWQiOiJsYXllci01IiwicGF0aCI6Ii9tbnQvUGMxU3RvcmUvY2FjaGUvcy10MDIyMzUyLTQ4MDI1Iiwicm93c190b19kaXNjYXJkIjo3LCJzaXplIjoxMDczNzQxODI0fSx7ImlkIjoibGF5ZXItNiIsInBhdGgiOiIvbW50L1BjMVN0b3JlL2NhY2hlL3MtdDAyMjM1Mi00ODAyNSIsInJvd3NfdG9fZGlzY2FyZCI6Nywic2l6ZSI6MTA3Mzc0MTgyNH0seyJpZCI6ImxheWVyLTciLCJwYXRoIjoiL21udC9QYzFTdG9yZS9jYWNoZS9zLXQwMjIzNTItNDgwMjUiLCJyb3dzX3RvX2Rpc2NhcmQiOjcsInNpemUiOjEwNzM3NDE4MjR9LHsiaWQiOiJsYXllci04IiwicGF0aCI6Ii9tbnQvUGMxU3RvcmUvY2FjaGUvcy10MDIyMzUyLTQ4MDI1Iiwicm93c190b19kaXNjYXJkIjo3LCJzaXplIjoxMDczNzQxODI0fSx7ImlkIjoibGF5ZXItOSIsInBhdGgiOiIvbW50L1BjMVN0b3JlL2NhY2hlL3MtdDAyMjM1Mi00ODAyNSIsInJvd3NfdG9fZGlzY2FyZCI6Nywic2l6ZSI6MTA3Mzc0MTgyNH0seyJpZCI6ImxheWVyLTEwIiwicGF0aCI6Ii9tbnQvUGMxU3RvcmUvY2FjaGUvcy10MDIyMzUyLTQ4MDI1Iiwicm93c190b19kaXNjYXJkIjo3LCJzaXplIjoxMDczNzQxODI0fSx7ImlkIjoibGF5ZXItMTEiLCJwYXRoIjoiL21udC9QYzFTdG9yZS9jYWNoZS9zLXQwMjIzNTItNDgwMjUiLCJyb3dzX3RvX2Rpc2NhcmQiOjcsInNpemUiOjEwNzM3NDE4MjR9XX19LCJyZWdpc3RlcmVkX3Byb29mIjoiU3RhY2tlZERyZzMyR2lCVjFfMSJ9"}}
7.  2022-03-11 20:37:45 +0100 CET:  [event;sealing.SectorPreCommit2]    {"User":{"Sealed":{"/":"bagboea4b5abcaiyd6l5almuvzrqyfouukq4neljp2kiebiofddwba3bygjicmw2p"},"Unsealed":{"/":"baga6ea4seaql6eiei7fnpyeg4ayizqsr5pl5a5ageauezlpqhd4c5p3fgzj4slq"}}}
8.  2022-03-11 20:37:45 +0100 CET:  [event;sealing.SectorPreCommitted]  {"User":{"Message":{"/":"bafy2bzacedt2e5h6fmptjoioc3tqig3j2yjs7vw3tan4awkxrmkflfcivp52w"},"PreCommitDeposit":"113032924231986218","PreCommitInfo":{"SealProof":8,"SectorNumber":48025,"SealedCID":{"/":"bagboea4b5abcaiyd6l5almuvzrqyfouukq4neljp2kiebiofddwba3bygjicmw2p"},"SealRandEpoch":1622531,"DealIDs":[4434491],"Expiration":2231261,"ReplaceCapacity":false,"ReplaceSectorDeadline":0,"ReplaceSectorPartition":0,"ReplaceSectorNumber":0}}}
9.  2022-03-11 20:41:00 +0100 CET:  [event;sealing.SectorPreCommitLanded]   {"User":{"TipSet":"AXGg5AIgyuKuarIxNb91HTtYyjUBpd6tico/wVbTdW1OCwYr6fgBcaDkAiAsJV8YChs5DX5xbyiPJ49UrbBCxMXcOq3UsyW7q/1/iAFxoOQCIHL22ARNNBZ6oO3p2JJrnbuD4QHHvqy1uY+7Ip33yP7hAXGg5AIgzfGqTNHbcxESJZ8fYstVvp9QWI8mR6/Gu1lbq+uDOgIBcaDkAiAMBCYNstSKOMu/1Wyb3LBXC9dsKY3HNf5jX5t0fXITHQFxoOQCIGNdV3+b6eRvSCBE+aD2qX9dEcDRgDipAteWzxTXZGli"}}
10. 2022-03-11 21:56:00 +0100 CET:  [event;sealing.SectorSeedReady] {"User":{"SeedValue":"d8FAdtj//1XogUdSxtr4SgGttxOxgtloXdMpnoLhUYk=","SeedEpoch":1624186}}
11. 2022-03-11 22:14:15 +0100 CET:  [event;sealing.SectorProofReady]    {"User":{"Proof":"hqYpkWNfgwxc1ZUqXVLW1dinzJF8IZDtTyyNiGTK8PCwzc0ytI4Jm8ciDimhT/pRkFcJmdGV8LC/Fpa0uESPlYplvcUPW1lN6vDccn1+GpDETNTNB7NIUqDTz8SARmPaGTkeu0y7hH6DkMPQoQfTV9MIV1EqNl44c8WL//7Mbq6o6WPAfiwMfWyOfiOyt8F5ry2oObztOdZQny1SD5cjiWi8Q83pOMNzdOmsgjiQ+eSM7ZCGWK6oQc3tRmaMpMI1lh2P6pSSJNBBuNa2S5Q2dxE7Hwc4MvLSOQIggQpiDXCJ99wDm3gDlEFO0Y3TVG5DufgXUGGLPDQ4C5grWA0tPHE6PFFBxap4ZgywZhsZv0EjDN0e7bTOOATKVyZQrz/TCu1yE/mTlp+YBqTKhE0fkHrwIrYAiQ3YNKBqPS1KLepiSvca+sYoVVZcO2Pn281vi5m2kyUrXfrC9CCbVysAXlTMDqI7ArKCKDvOOat8zrLAIEWVt/rNuoGirZB88FQ7uaV/tja2M53NyfMqkyVG5jb37iCEhSwGz8uQHv8QVX0L2kDRfXABeL6sYhsZHpsQmWnUJ2GmvxSHhxcD0acJN8htPmZM/YBdZRkt98RwS2LiB8QVhSmOBQOZpSETk80aBnIdWVcerL8bPfH0SOBbL6lBfNxPJfkaKy4YwRRkRqKbNX4VB97rEGy/gH1NianThLUyRMfQ33LzgXUnoRrVDbPD85nKqpvOhVB66gtygqKngK0vZ388i/88SU9XPeqtuOoojGeYo+hGHSyzIhKffj9KPn+WQYgCJreG3l43662t/QdW3eIBmb3/AP9Y603CjKgNkdizauT8O6DwHSRM6rGQnlxrObTkFLdJn6UDEGi6cftOFZyukl+QUxOwY1ZhFqzbhJI8PXsRJPzRP0xnZjz8UM5dQlxKosC8q0Y7fLTDtQkVJRNgauyyYSSx6OjElmdS6/uCp6WFLlEnxWqYI+uP6Gfv2dhHhyUuRmHR3VLQf2QjsSvD0ZVRoG/TZUxprn4Xservgq0/z3pP+3IUFCykFydMtCzLpjq0o8dfU1AuDKwWj6fZi/jEjZPGMJ1bigxy88u+2Wxs/SOHePlGPaSI1D4Bn/IL5LKvYh3BiGPsNBEGRawb1jALYvHfQoSWAGmCWVDZRY2G7TD4iLxkVxt2dfYFxqy10V6hoNOa2M2SUTwkkiHD1FfWQa34q+ASrk4JuCS0mWrHZuIHfutgON4YrCRovraQOYQadYoNoW5lzw97QQut40+L6BXrs7uyp+1Do+bOh/7/Q6gtiCy7y8OqYhwtVlHTLxR6CUNNErZZOo3WOmaoG86+cRrSExFvqbHmu26Tx+ZEl+aMFh1jnqeJJDVrt7/EyJc3c0xOpEjYCZm+gGV0uArrEhDVRhlFBkw+CmEgGXv5BlRWan1aiEXO7d5a05F+CypCFkziyz0AKj6b7S7WKj/qcRw2j74/kpA5cuLPMNbmXzXwtVNv8wCis6g8hsUZBfRLOE2VnEm0JMXuBDSrP7sGXYPCmUwVo4vluYzKmRy4rq7O8iGf8KhM1E6Z1vA05P4iGLobfZVNI8iTUFaA0gVDk5yAIaZ0ip/Be6p6hCIuvSQu6lgA9L1zyLjqUsdQkicOexnYsl7TlmD8NMt4L8BLv/TJtCFWGGBc2pMBrqwGT1J/9Arh15MBZw69cM5L2TPlWc1YZo0EDrf0PO/Pl5RbAH6NSowdlBcGsmkVDQ3UyB+eAla8ROeN+6/DZ7UIp0AxH7rQO/yPHgtPX7CsYiGTObRsBjVMpZ8xQYDLT8RhXopabDvSrVWg4MQSjzSKn0urGdNnjy6Qtoq6xgtTrFqiaJ2JXCi4uYnH2lhBUtmluz+DW1ux7P5E5MPjuQgrn9WnXhfc7v5Nv69RPcGfgKvYyV/3xPs1CKti8xfdOuAlntyJ4tQRP8IB4SWjpkAiBmjaMSOTAnxfn/7impyPMyLHAtiKz5KHhdxOokUQ4zhJQKrZM5A9u3zHCtlTlzABtVzYdXlISQSS1sQ5j1MDgX7SS8FLLoFSqHCPwjGA+NHy8m4m+jQDmCrsksgW6680PUTQLbBUYdTfDJ9IHUC6C9/SFeQjbx0LlVfS8Lu5jDzrKp+SBM6ER32xgSi0JRLO4IMGb+GvTbWbTwGRRjaZXtVLRe7FqowYB7wyfF1lDbei5fTANPMzbtbAigXS3JmBCXy1iyZ2mzrE3/+PvJ1jvWeayCtm2aeNit5W6s5qyLk9O9NS0hnqeB+Uid4dDauanTa4sTyu6Fz8f5fsydBnSXSQ0CGsd7Los9KCuNHZPWkfyJHiwnhzkW9H4HeR5ekTUT9WlJbswll24H9TdZQQwYsIDigANxX2r8ZuY9LDJZ3al4EP/aswCbxRfsvsMFPRpN+bm0c0Vau1gY04/UqTHGw+k68Kf/R0CPaF3yBmDzuNJk8UwwcMGER0WDR0Y/Xqr/q28Lp/f73DfDQLLKeVeOo6jBJAypC7uN17N0vCPlnv6SXBMqang7CTuZTGR4niPgYKI0HYW+uoLZi/OIMKt5I0Aqf40MlL"}}
12. 2022-03-11 22:19:25 +0100 CET:  [event;sealing.SectorFinalized] {"User":{}}
13. 2022-03-11 22:19:25 +0100 CET:  [event;sealing.SectorCommitSubmitted]   {"User":{"Message":{"/":"bafy2bzaceah4mv7ipx6kbc7q2pknjk54puvt7inozhm3y2wxnsnwlgatitoki"}}}
14. 2022-03-11 22:23:29 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
15. 2022-03-11 22:23:29 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
16. 2022-03-11 22:23:29 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
17. 2022-03-11 22:23:29 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
18. 2022-03-11 22:23:29 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
19. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
20. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
21. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
22. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
23. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
24. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
25. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
26. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
27. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
28. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
29. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
30. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
31. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
32. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
33. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
34. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
35. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
36. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
37. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
38. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
39. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
40. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
41. 2022-03-11 22:23:30 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
42. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
43. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
44. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
45. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
46. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
47. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
48. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
49. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
50. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
51. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
52. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
53. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
54. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
55. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
56. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
57. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
58. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
59. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
60. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
61. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
62. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
63. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
64. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
65. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
66. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
67. 2022-03-11 22:23:31 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
68. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
69. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
70. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
71. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
72. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
73. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
74. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
75. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
76. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
77. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
78. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
79. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
80. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
81. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
82. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
83. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
84. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
85. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
86. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
87. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
88. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
89. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
90. 2022-03-11 22:23:32 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
91. 2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
92. 2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
93. 2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
94. 2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
95. 2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
96. 2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
97. 2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
98. 2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
99. 2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
100.    2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
101.    2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
102.    2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
103.    2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
104.    2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
105.    2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
106.    2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
107.    2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
108.    2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
109.    2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
110.    2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
111.    2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
112.    2022-03-11 22:23:33 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
113.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
114.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
115.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
116.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
117.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
118.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
119.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
120.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
121.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
122.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
123.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
124.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
125.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
126.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
127.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
128.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
129.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
130.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
131.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
132.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
133.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
134.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
135.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
136.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
137.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
138.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
139.    2022-03-11 22:23:34 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
140.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
141.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
142.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
143.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
144.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
145.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
146.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
147.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
148.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
149.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
150.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
151.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
152.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
153.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
154.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
155.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
156.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
157.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
158.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
159.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
160.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
161.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
162.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
163.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
164.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
165.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
166.    2022-03-11 22:23:35 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
167.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
168.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
169.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
170.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
171.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
172.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
173.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
174.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
175.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
176.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
177.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
178.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
179.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
180.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
181.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
182.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
183.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
184.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
185.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
186.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
187.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
188.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
189.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
190.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
191.    2022-03-11 22:23:36 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
192.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
193.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
194.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
195.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
196.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
197.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
198.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
199.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
200.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
201.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
202.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
203.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
204.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
205.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
206.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
207.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
208.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
209.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
210.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
211.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
212.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
213.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
214.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
215.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
216.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
217.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
218.    2022-03-11 22:23:37 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
219.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
220.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
221.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
222.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
223.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
224.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
225.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
226.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
227.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
228.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
229.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
230.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
231.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
232.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
233.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
234.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
235.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
236.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
237.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
238.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
239.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
240.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
241.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
242.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
243.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
244.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
245.    2022-03-11 22:23:38 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
246.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
247.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
248.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
249.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
250.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
251.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
252.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
253.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
254.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
255.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
256.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
257.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
258.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
259.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
260.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
261.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
262.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
263.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
264.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
265.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
266.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
267.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
268.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
269.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
270.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
271.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
272.    2022-03-11 22:23:39 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
273.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
274.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
275.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
276.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
277.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
278.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
279.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
280.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
281.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
282.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
283.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
284.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
285.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
286.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
287.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
288.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
289.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
290.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
291.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
292.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
293.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
294.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
295.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
296.    2022-03-11 22:23:40 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
297.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
298.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
299.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
300.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
301.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
302.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
303.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
304.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
305.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
306.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
307.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
308.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
309.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
310.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
311.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
312.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
313.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
314.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
315.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
316.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
317.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
318.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
319.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
320.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
321.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
322.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
323.    2022-03-11 22:23:41 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
324.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
325.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
326.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
327.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
328.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
329.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
330.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
331.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
332.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
333.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
334.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
335.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
336.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
337.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
338.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
339.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
340.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
341.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
342.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
343.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
344.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
345.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
346.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
347.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
348.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
349.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
350.    2022-03-11 22:23:42 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
351.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
352.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
353.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
354.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
355.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
356.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
357.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
358.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
359.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
360.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
361.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
362.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
363.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
364.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
365.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
366.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
367.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
368.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
369.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
370.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
371.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
372.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
373.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
374.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
375.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
376.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
377.    2022-03-11 22:23:43 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
378.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
379.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
380.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
381.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
382.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
383.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
384.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
385.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
386.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
387.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
388.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
389.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
390.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
391.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
392.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
393.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
394.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
395.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
396.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
397.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
398.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
399.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
400.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
401.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
402.    2022-03-11 22:23:44 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
403.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
404.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
405.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
406.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
407.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
408.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
409.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
410.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
411.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
412.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
413.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
414.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
415.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
416.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
417.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
418.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
419.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
420.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
421.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
422.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
423.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
424.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
425.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
426.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
427.    2022-03-11 22:23:45 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
428.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
429.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
430.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
431.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
432.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
433.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
434.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
435.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
436.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
437.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
438.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
439.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
440.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
441.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
442.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
443.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
444.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
445.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
446.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
447.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
448.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
449.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
450.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
451.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
452.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
453.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
454.    2022-03-11 22:23:46 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
455.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
456.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
457.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
458.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
459.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
460.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
461.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
462.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
463.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
464.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
465.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
466.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
467.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
468.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
469.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
470.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
471.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
472.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
473.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
474.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
475.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
476.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
477.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
478.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
479.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
480.    2022-03-11 22:23:47 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
481.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
482.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
483.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
484.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
485.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
486.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
487.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
488.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
489.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
490.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
491.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
492.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
493.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
494.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
495.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
496.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
497.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
498.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
499.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
500.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
501.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
502.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
503.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
504.    2022-03-11 22:23:48 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
505.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
506.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
507.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
508.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
509.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
510.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
511.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
512.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
513.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
514.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
515.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
516.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
517.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
518.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
519.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
520.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
521.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
522.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
523.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
524.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
525.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
526.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
527.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
528.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
529.    2022-03-11 22:23:49 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
530.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
531.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
532.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
533.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
534.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
535.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
536.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
537.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
538.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
539.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
540.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
541.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
542.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
543.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
544.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
545.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
546.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
547.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
548.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
549.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
550.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
551.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
552.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
553.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
554.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
555.    2022-03-11 22:23:50 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
556.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
557.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
558.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
559.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
560.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
561.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
562.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
563.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
564.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
565.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
566.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
567.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
568.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
569.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
570.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
571.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
572.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
573.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
574.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
575.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
576.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
577.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
578.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
579.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
580.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
581.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
582.    2022-03-11 22:23:51 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
583.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
584.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
585.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
586.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
587.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
588.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
589.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
590.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
591.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
592.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
593.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
594.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
595.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
596.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
597.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
598.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
599.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
600.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
601.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
602.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
603.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
604.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
605.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
606.    2022-03-11 22:23:52 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
607.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
608.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
609.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
610.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
611.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
612.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
613.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
614.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
615.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
616.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
617.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
618.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
619.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
620.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
621.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
622.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
623.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
624.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
625.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
626.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
627.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
628.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
629.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
630.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
631.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
632.    2022-03-11 22:23:53 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
633.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
634.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
635.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
636.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
637.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
638.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
639.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
640.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
641.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
642.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
643.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
644.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
645.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
646.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
647.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
648.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
649.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
650.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
651.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
652.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
653.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
654.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
655.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
656.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
657.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
658.    2022-03-11 22:23:54 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
659.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
660.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
661.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
662.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
663.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
664.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
665.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
666.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
667.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
668.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
669.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
670.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
671.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
672.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
673.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
674.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
675.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
676.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
677.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
678.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
679.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
680.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
681.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
682.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
683.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
684.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
685.    2022-03-11 22:23:55 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
686.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
687.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
688.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
689.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
690.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
691.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
692.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
693.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
694.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
695.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
696.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
697.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
698.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
699.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
700.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
701.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
702.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
703.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
704.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
705.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
706.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
707.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
708.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
709.    2022-03-11 22:23:56 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
710.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
711.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
712.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
713.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
714.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
715.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
716.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
717.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
718.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
719.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
720.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
721.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
722.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
723.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
724.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
725.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
726.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
727.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
728.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
729.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
730.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
731.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
732.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
733.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
734.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
735.    2022-03-11 22:23:57 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
736.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
737.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
738.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
739.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
740.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
741.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
742.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
743.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
744.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
745.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
746.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
747.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
748.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
749.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
750.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
751.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
752.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
753.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
754.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
755.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
756.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
757.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
758.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
759.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
760.    2022-03-11 22:23:58 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
761.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
762.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
763.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
764.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
765.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
766.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
767.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
768.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
769.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
770.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
771.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
772.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
773.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
774.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
775.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
776.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
777.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
778.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
779.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
780.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
781.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
782.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
783.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
784.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
785.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
786.    2022-03-11 22:23:59 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
787.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
788.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
789.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
790.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
791.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
792.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
793.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
794.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
795.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
796.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
797.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
798.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
799.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
800.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
801.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
802.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
803.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
804.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
805.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
806.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
807.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
808.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
809.    2022-03-11 22:24:00 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
810.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
811.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
812.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
813.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
814.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
815.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
816.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
817.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
818.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
819.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
820.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
821.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
822.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
823.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
824.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
825.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
826.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
827.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
828.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
829.    2022-03-11 22:24:01 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
830.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
831.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
832.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
833.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
834.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
835.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
836.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
837.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
838.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
839.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
840.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
841.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
842.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
843.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
844.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
845.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
846.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
847.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
848.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
849.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
850.    2022-03-11 22:24:02 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
851.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
852.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
853.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
854.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
855.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
856.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
857.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
858.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
859.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
860.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
861.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
862.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
863.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
864.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
865.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
866.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
867.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
868.    2022-03-11 22:24:03 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
869.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
870.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
871.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
872.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
873.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
874.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
875.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
876.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
877.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
878.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
879.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
880.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
881.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
882.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
883.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
884.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
885.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
886.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
887.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
888.    2022-03-11 22:24:04 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
889.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
890.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
891.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
892.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
893.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
894.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
895.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
896.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
897.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
898.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
899.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
900.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
901.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
902.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
903.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
904.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
905.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
906.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
907.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
908.    2022-03-11 22:24:05 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
909.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
910.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
911.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
912.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
913.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
914.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
915.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
916.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
917.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
918.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
919.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
920.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
921.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
922.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
923.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
924.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
925.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
926.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
927.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
928.    2022-03-11 22:24:06 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
929.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
930.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
931.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
932.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
933.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
934.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
935.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
936.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
937.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
938.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
939.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
940.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
941.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
942.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
943.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
944.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
945.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
946.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
947.    2022-03-11 22:24:07 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
948.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
949.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
950.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
951.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
952.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
953.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
954.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
955.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
956.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
957.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
958.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
959.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
960.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
961.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
962.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
963.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
964.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
965.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
966.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
967.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
968.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
969.    2022-03-11 22:24:08 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
970.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
971.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
972.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
973.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
974.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
975.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
976.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
977.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
978.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
979.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
980.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
981.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
982.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
983.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
984.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
985.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
986.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
987.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
988.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
989.    2022-03-11 22:24:09 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
990.    2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
991.    2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
992.    2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
993.    2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
994.    2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
995.    2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
996.    2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
997.    2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
998.    2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
999.    2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1000.   2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1001.   2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1002.   2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1003.   2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1004.   2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1005.   2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1006.   2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1007.   2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1008.   2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1009.   2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1010.   2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1011.   2022-03-11 22:24:10 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1012.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1013.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1014.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1015.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1016.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1017.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1018.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1019.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1020.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1021.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1022.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1023.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1024.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1025.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1026.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1027.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1028.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1029.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1030.   2022-03-11 22:24:11 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1031.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1032.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1033.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1034.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1035.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1036.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1037.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1038.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1039.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1040.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1041.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1042.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1043.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1044.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1045.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1046.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1047.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1048.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1049.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1050.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1051.   2022-03-11 22:24:12 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1052.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1053.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1054.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1055.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1056.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1057.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1058.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1059.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1060.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1061.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1062.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1063.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1064.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1065.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1066.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1067.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1068.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1069.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1070.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1071.   2022-03-11 22:24:13 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1072.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1073.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1074.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1075.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1076.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1077.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1078.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1079.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1080.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1081.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1082.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1083.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1084.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1085.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1086.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1087.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1088.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1089.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1090.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1091.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1092.   2022-03-11 22:24:14 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1093.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1094.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1095.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1096.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1097.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1098.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1099.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1100.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1101.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1102.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1103.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1104.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1105.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1106.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1107.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1108.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1109.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1110.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1111.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1112.   2022-03-11 22:24:15 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1113.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1114.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1115.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1116.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1117.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1118.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1119.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1120.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1121.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1122.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1123.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1124.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1125.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1126.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1127.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1128.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1129.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1130.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1131.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1132.   2022-03-11 22:24:16 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1133.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1134.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1135.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1136.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1137.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1138.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1139.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1140.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1141.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1142.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1143.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1144.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1145.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1146.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1147.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1148.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1149.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1150.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1151.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1152.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1153.   2022-03-11 22:24:17 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1154.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1155.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1156.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1157.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1158.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1159.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1160.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1161.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1162.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1163.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1164.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1165.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1166.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1167.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1168.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1169.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1170.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1171.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1172.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1173.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1174.   2022-03-11 22:24:18 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1175.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1176.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1177.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1178.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1179.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1180.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1181.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1182.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1183.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1184.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1185.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1186.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1187.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1188.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1189.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
1190.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorCommitFailed]  {"User":{}}
    proof validation failed, sector not found in sector set after cron
1191.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorForceState]    {"User":{"State":"PreCommit1"}}
1192.   2022-03-11 22:24:19 +0100 CET:  [event;sealing.SectorRetryCommitWait]   {"User":{}}
RobQuistNL commented 2 years ago

I don't know why this happens and your reproduction methods are concerning. Because;

dont upgrade while have production line of workers sealing

We never do that

Upgrade all workers/daemon/miner/markets together

We always do that

Start daemon up before anything else

We always do that

Backup miner frequently (config.toml etc.)

We also do that

Reiers commented 2 years ago

My repro is concerning because I came with some friendly advise on best practices while we trying to figure this out. How Im I suppose to reproduce then? to gather logs? @RobQuistNL We are currently looking at the issue and need more logs than above. I put my own production miner on the line for this - so please share how I can gather logs faster so this can get resolved.

Reiers commented 2 years ago

I have never encounter this issue - so I needed do something to trigger it. and your description is: Sector is stuck in a loop (have 2 of these) and your logs from 0.5.4 is not there anymore - and you dont know what triggers it.

So I ask again, what is wrong with my method of repro? and is there a better way? Because I have spent a week now, gathering logs.

RobQuistNL commented 2 years ago

What? I'm not saying anything's wrong - I just find it concerning that it has to be done this way. I mean I think there's something else deep down in lotus / the workflow thats causing this - I find it concerning that your method of reproduction yields the same result as us regularly running lotus (while not doing anything funky like your repro steps).

Reiers commented 2 years ago

Okay, I misunderstood your message then. But that lotus users are running into this without my "funky" repro is concerning, agreed. @RobQuistNL

The good news is that Im narrowing it down with logs - with setting the right kind of log level and running trace. So will we find the cause eventually :) (The only reason I cant share all those logs in the thread here, is because it gives out a lot of information about my SP)

RobQuistNL commented 2 years ago

No worries.

BTW, these big logs;

You can throw them in a spoiler :+) ```bash all your logs go here ```

How to:

<details>
  <summary>You can throw them in a spoiler :+)</summary>

  ```bash
all your logs go here

Reiers commented 2 years ago

Hi @RobQuistNL

Thanks for the report. This issue will most likely be resolved in the upcoming release.

Closing ticket.

If you encounter this issue in the current release (v1.15.2+) - please open a new ticket ! Thanks

lanpin commented 2 years ago

Problem reproduces

Reiers commented 2 years ago

Do you happen to have any logs to look at @lanpin ? If you have reproduced the the issue, and have some pointer to where to look so we can find the root cause that would be great.

Your issue ticket does not provide any more information and you are running a .dirty version of lotus