[X] This is not a security-related bug/issue. If it is, please follow please follow the security policy.
[X] This is not a question or a support request. If you have any lotus related questions, please ask in the lotus forum.
[X] This is not a new feature request. If it is, please file a feature request instead.
[X] This is not an enhancement request. If it is, please file a improvement suggestion instead.
[X] I have searched on the issue tracker and the lotus forum, and there is no existing related issue or discussion.
[X] I am running the Latest release, or the most recent RC(release canadiate) for the upcoming release or the dev branch(master), or have an issue updating to any of these.
[ ] I did not make any code changes to lotus.
Lotus component
[ ] lotus daemon - chain sync
[ ] lotus miner - mining and block production
[ ] lotus miner/worker - sealing
[X] lotus miner - proving(WindowPoSt)
[ ] lotus miner/market - storage deal
[ ] lotus miner/market - retrieval deal
[ ] lotus miner/market - data transfer
[ ] lotus client
[ ] lotus JSON-RPC API
[ ] lotus message management (mpool)
[ ] Other
Lotus Version
Local: lotus version 1.17.1+mainnet+git.8db6a939c+api1.5.0
lotus-miner version 1.17.1+mainnet+git.8db6a939c+api1.5.0
Describe the Bug
Operation: I move a long-term storage sector and cache to another long-term storage disk. After the miner starts, I execute lotus miner providing compute window post 0. ERROR: running window post failed: Rust panic: Lazy instance has previously been compromised
root@root:/RAID/filecoin/lotusminertwo# lotus-miner proving check --slow 0
deadline partition sector status
0 0 29 good
0 0 168 good
0 0 88 good
0 0 120 good
0 0 145 good
0 0 150 good
0 0 115 good
0 0 117 good
0 0 174 good
0 0 187 good
0 0 60 good
0 0 131 good
0 0 160 good
0 0 185 good
0 0 68 good
0 0 71 good
0 0 78 good
0 0 90 good
0 0 129 good
0 0 164 good
0 0 13 good
0 0 19 good
0 0 70 good
0 0 92 good
0 0 104 good
0 0 24 good
0 0 99 good
0 0 100 good
0 0 142 good
0 0 165 good
0 0 169 good
0 0 179 good
0 0 84 good
0 0 94 good
0 0 21 good
0 0 28 good
0 0 69 good
0 0 79 good
0 0 146 good
0 0 170 good
0 0 6 good
0 0 15 good
0 0 17 good
0 0 30 good
0 0 80 good
0 0 148 good
0 0 11 good
0 0 55 good
0 0 58 good
0 0 63 good
0 0 77 good
0 0 114 good
0 0 147 good
0 0 18 good
0 0 20 good
0 0 53 good
0 0 74 good
0 0 75 good
0 0 184 good
0 0 23 good
0 0 76 good
0 0 113 good
0 0 157 good
0 0 89 good
0 0 101 good
0 0 175 good
0 0 3 good
0 0 73 good
0 0 154 good
0 0 155 good
0 0 182 good
0 0 186 good
0 0 97 good
0 0 109 good
0 0 116 good
0 0 130 good
0 0 137 good
0 0 172 good
0 0 14 good
0 0 26 good
0 0 51 good
0 0 108 good
0 0 176 good
0 0 98 good
0 0 103 good
0 0 143 good
0 0 144 good
0 0 156 good
0 0 161 good
0 0 7 good
0 0 82 good
0 0 87 good
0 0 125 good
0 0 139 good
0 0 181 good
0 0 65 good
0 0 152 good
0 0 138 good
0 0 50 good
0 0 61 good
0 0 83 good
0 0 121 good
0 0 133 good
0 0 135 good
0 0 166 good
0 0 85 good
0 0 86 good
0 0 132 good
0 0 173 good
0 0 4 good
0 0 5 good
0 0 16 good
0 0 56 good
0 0 64 good
0 0 105 good
0 0 110 good
0 0 151 good
0 0 158 good
0 0 159 good
0 0 12 good
0 0 93 good
0 0 134 good
0 0 136 good
0 0 124 good
0 0 149 good
0 0 8 good
0 0 54 good
0 0 96 good
0 0 140 good
0 0 118 good
0 0 10 good
0 0 49 good
0 0 66 good
0 0 81 good
0 0 153 good
0 0 162 good
0 0 167 good
0 0 171 good
0 0 9 good
0 0 52 good
0 0 95 good
0 0 177 good
0 0 183 good
0 0 27 good
0 0 62 good
0 0 67 good
0 0 127 good
0 0 180 good
0 0 102 good
0 0 112 good
0 0 128 good
0 0 141 good
0 0 163 good
0 0 178 good
root@root:/RAID/filecoin/lotusminertwo# lotus-miner proving check --slow 1
deadline partition sector status
1 0 31 good
1 0 59 good
root@root:/RAID/filecoin/lotusminertwo# lotus-miner proving check --slow 2
deadline partition sector status
2 0 91 good
root@root:/RAID/filecoin/lotusminertwo# lotus-miner storage find 31
In 62fbba7f-f94d-464b-ab74-3565ff03570d (Sealed, Cache)
Sealing: false; Storage: false
Local (/RAID/filecoin/storage)
URL: http://10.1.80.4:2345/remote/sealed/s-t01904162-31
root@root:/RAID/filecoin/lotusminertwo# lotus-miner storage find 59
In e4956cd2-0908-48bc-a17d-b4a05e960ba5 (Sealed, Cache)
Sealing: false; Storage: true
Local (/RAID1/data01)
URL: http://10.1.80.4:2345/remote/sealed/s-t01904162-59
root@root:/RAID/filecoin/lotusminertwo# lotus-miner storage find 91
In e4956cd2-0908-48bc-a17d-b4a05e960ba5 (Sealed, Cache)
Sealing: false; Storage: true
Local (/RAID1/data01)
URL: http://10.1.80.4:2345/remote/sealed/s-t01904162-91
root@root:/RAID/filecoin/lotusminertwo# lotus-miner proving compute window-post 0
Took 1m5.681708627s
ERROR: running window post failed: Rust panic: Lazy instance has previously been poisoned
I restored the moved sector to the original location and used the cbor file backed up by miner to restore miner, but the problem is still unresolved
I restored the moved sector to the original location and used the cbor file backed up by the miner to restore the miner, but the problem was still unresolved. Finally, I decided to revert the miner version from 1.17.1 to 1.17.0, and then executed lotus miner providing compute window post 0 again. The result was successful. At this time, the remote lotus version was 1.17.1, and the local miner version was 1.17.0
root@root:/RAID/filecoin# lotus-miner proving compute window-post 1
Took 4m35.89821116s
[{"Deadline":1,"Partitions":[{"Index":0,"Skipped":[0]}],"Proofs":[{"PoStProof":8,"ProofBytes":"obJ14wTG2xBCeXqz+pzFsM/J+w4A0NK02AsPYKnI04DWzhkHLHxi0704g8y+pP7gqf1DE2Ha1DUsQvlfce4E5B+mWFwEVdCyRj9OkIyqqaqwDWvmOVvG6VK2C6UShNnhBXRVgyEH8CQICJoN+nbTJvrlEAz6HQP/vcnvOTRvAkpUWWHpdjzcy3bs5jVULFO4qjJA1qQGbc8v5u6PX9WNQNuc1ZibQIEM3r9Xb/KLMqxPNEyoovly8qUfKjh56/yT"}],"ChainCommitEpoch":0,"ChainCommitRand":null}]
root@root:/RAID/filecoin# lotus-miner proving compute window-post 0
Took 4m32.88303155s
[{"Deadline":0,"Partitions":[{"Index":0,"Skipped":[0]}],"Proofs":[{"PoStProof":8,"ProofBytes":"sg8GHUn/G4YUmSLqntad4dsCZJ8vIdX/Yx/xCHSSJuGhUNgL/m2zdTVz6t9e7p74teQGqpO/eCUr6L5vZiS5UR3Mct8F+lJrdzna3B6zWnnb8kqNOrqbFpmvGCPQVXrWD+olh7GX90ZZJIfUtk67bd0rM/bBD7p5F40iGOeuAjCLEFaNd9lFX3wg4+UQ9lEVmFfyRcvKBYerkoiZ6ckn+ag7Uv8pVzPthZt1V/XmrOvxGQCIrpjPCEL36t7WqOQN"}],"ChainCommitEpoch":0,"ChainCommitRand":null}]
root@root:/RAID/filecoin# lotus-miner proving compute window-post 2
Took 4m25.117250726s
[{"Deadline":2,"Partitions":[{"Index":0,"Skipped":[0]}],"Proofs":[{"PoStProof":8,"ProofBytes":"t4Cjg9GUsM4zP2XIY3f74ShH0CeLer5U/2QcgdZcm0ojoei1SVfSSx4cppHsi1O/i7t2RP6L90DhsdVSXE5/ghQ05KsLNsqdoh2Et6QZt9hZIRih1u2wOheNlx5elbUYBNimJq4oypJL+LpKLEPTLy+rn24hNyVofq0x41jcW+42/LcnylSoxYnSnKwBPe+urB6BVeH/0GX6P+aoi8Lzb9NIFCO+BlwnITCLbNtcm9vnqVtIIrZNdqXNT1yBzxoh"}],"ChainCommitEpoch":0,"ChainCommitRand":null}]
Logging Information
Local: lotus version 1.17.1+mainnet+git.8db6a939c+api1.5.0
lotus-miner version 1.17.1+mainnet+git.8db6a939c+api1.5.0
root@root:/RAID/filecoin/lotusminertwo# lotus-miner proving compute window-post 0
Took 1m5.681708627s
ERROR: running window post failed: Rust panic: Lazy instance has previously been poisoned
After success
Local: lotus version 1.17.1+mainnet+git.8db6a939c+api1.5.0
lotus-miner version version 1.17.0+mainnet+git.2830429ad
root@root:/RAID/filecoin# lotus-miner proving compute window-post 1
Took 4m35.89821116s
[{"Deadline":1,"Partitions":[{"Index":0,"Skipped":[0]}],"Proofs":[{"PoStProof":8,"ProofBytes":"obJ14wTG2xBCeXqz+pzFsM/J+w4A0NK02AsPYKnI04DWzhkHLHxi0704g8y+pP7gqf1DE2Ha1DUsQvlfce4E5B+mWFwEVdCyRj9OkIyqqaqwDWvmOVvG6VK2C6UShNnhBXRVgyEH8CQICJoN+nbTJvrlEAz6HQP/vcnvOTRvAkpUWWHpdjzcy3bs5jVULFO4qjJA1qQGbc8v5u6PX9WNQNuc1ZibQIEM3r9Xb/KLMqxPNEyoovly8qUfKjh56/yT"}],"ChainCommitEpoch":0,"ChainCommitRand":null}]
root@root:/RAID/filecoin# lotus-miner proving compute window-post 0
Took 4m32.88303155s
[{"Deadline":0,"Partitions":[{"Index":0,"Skipped":[0]}],"Proofs":[{"PoStProof":8,"ProofBytes":"sg8GHUn/G4YUmSLqntad4dsCZJ8vIdX/Yx/xCHSSJuGhUNgL/m2zdTVz6t9e7p74teQGqpO/eCUr6L5vZiS5UR3Mct8F+lJrdzna3B6zWnnb8kqNOrqbFpmvGCPQVXrWD+olh7GX90ZZJIfUtk67bd0rM/bBD7p5F40iGOeuAjCLEFaNd9lFX3wg4+UQ9lEVmFfyRcvKBYerkoiZ6ckn+ag7Uv8pVzPthZt1V/XmrOvxGQCIrpjPCEL36t7WqOQN"}],"ChainCommitEpoch":0,"ChainCommitRand":null}]
root@root:/RAID/filecoin# lotus-miner proving compute window-post 2
Took 4m25.117250726s
[{"Deadline":2,"Partitions":[{"Index":0,"Skipped":[0]}],"Proofs":[{"PoStProof":8,"ProofBytes":"t4Cjg9GUsM4zP2XIY3f74ShH0CeLer5U/2QcgdZcm0ojoei1SVfSSx4cppHsi1O/i7t2RP6L90DhsdVSXE5/ghQ05KsLNsqdoh2Et6QZt9hZIRih1u2wOheNlx5elbUYBNimJq4oypJL+LpKLEPTLy+rn24hNyVofq0x41jcW+42/LcnylSoxYnSnKwBPe+urB6BVeH/0GX6P+aoi8Lzb9NIFCO+BlwnITCLbNtcm9vnqVtIIrZNdqXNT1yBzxoh"}],"ChainCommitEpoch":0,"ChainCommitRand":null}]
TippyFlitsUK
commented
2 years ago
Checklist
Latest release, or the most recent RC(release canadiate) for the upcoming release or the dev branch(master), or have an issue updating to any of these.Lotus component
Lotus Version
Describe the Bug
Operation: I move a long-term storage sector and cache to another long-term storage disk. After the miner starts, I execute lotus miner providing compute window post 0. ERROR: running window post failed: Rust panic: Lazy instance has previously been compromised
root@root:/RAID/filecoin/lotusminertwo# lotus-miner proving check --slow 0 deadline partition sector status 0 0 29 good 0 0 168 good 0 0 88 good 0 0 120 good 0 0 145 good 0 0 150 good 0 0 115 good 0 0 117 good 0 0 174 good 0 0 187 good 0 0 60 good 0 0 131 good 0 0 160 good 0 0 185 good 0 0 68 good 0 0 71 good 0 0 78 good 0 0 90 good 0 0 129 good 0 0 164 good 0 0 13 good 0 0 19 good 0 0 70 good 0 0 92 good 0 0 104 good 0 0 24 good 0 0 99 good 0 0 100 good 0 0 142 good 0 0 165 good 0 0 169 good 0 0 179 good 0 0 84 good 0 0 94 good 0 0 21 good 0 0 28 good 0 0 69 good 0 0 79 good 0 0 146 good 0 0 170 good 0 0 6 good 0 0 15 good 0 0 17 good 0 0 30 good 0 0 80 good 0 0 148 good 0 0 11 good 0 0 55 good 0 0 58 good 0 0 63 good 0 0 77 good 0 0 114 good 0 0 147 good 0 0 18 good 0 0 20 good 0 0 53 good 0 0 74 good 0 0 75 good 0 0 184 good 0 0 23 good 0 0 76 good 0 0 113 good 0 0 157 good 0 0 89 good 0 0 101 good 0 0 175 good 0 0 3 good 0 0 73 good 0 0 154 good 0 0 155 good 0 0 182 good 0 0 186 good 0 0 97 good 0 0 109 good 0 0 116 good 0 0 130 good 0 0 137 good 0 0 172 good 0 0 14 good 0 0 26 good 0 0 51 good 0 0 108 good 0 0 176 good 0 0 98 good 0 0 103 good 0 0 143 good 0 0 144 good 0 0 156 good 0 0 161 good 0 0 7 good 0 0 82 good 0 0 87 good 0 0 125 good 0 0 139 good 0 0 181 good 0 0 65 good 0 0 152 good 0 0 138 good 0 0 50 good 0 0 61 good 0 0 83 good 0 0 121 good 0 0 133 good 0 0 135 good 0 0 166 good 0 0 85 good 0 0 86 good 0 0 132 good 0 0 173 good 0 0 4 good 0 0 5 good 0 0 16 good 0 0 56 good 0 0 64 good 0 0 105 good 0 0 110 good 0 0 151 good 0 0 158 good 0 0 159 good 0 0 12 good 0 0 93 good 0 0 134 good 0 0 136 good 0 0 124 good 0 0 149 good 0 0 8 good 0 0 54 good 0 0 96 good 0 0 140 good 0 0 118 good 0 0 10 good 0 0 49 good 0 0 66 good 0 0 81 good 0 0 153 good 0 0 162 good 0 0 167 good 0 0 171 good 0 0 9 good 0 0 52 good 0 0 95 good 0 0 177 good 0 0 183 good 0 0 27 good 0 0 62 good 0 0 67 good 0 0 127 good 0 0 180 good 0 0 102 good 0 0 112 good 0 0 128 good 0 0 141 good 0 0 163 good 0 0 178 good root@root:/RAID/filecoin/lotusminertwo# lotus-miner proving check --slow 1 deadline partition sector status 1 0 31 good 1 0 59 good root@root:/RAID/filecoin/lotusminertwo# lotus-miner proving check --slow 2 deadline partition sector status 2 0 91 good root@root:/RAID/filecoin/lotusminertwo# lotus-miner storage find 31 In 62fbba7f-f94d-464b-ab74-3565ff03570d (Sealed, Cache) Sealing: false; Storage: false Local (/RAID/filecoin/storage) URL: http://10.1.80.4:2345/remote/sealed/s-t01904162-31 root@root:/RAID/filecoin/lotusminertwo# lotus-miner storage find 59 In e4956cd2-0908-48bc-a17d-b4a05e960ba5 (Sealed, Cache) Sealing: false; Storage: true Local (/RAID1/data01) URL: http://10.1.80.4:2345/remote/sealed/s-t01904162-59 root@root:/RAID/filecoin/lotusminertwo# lotus-miner storage find 91 In e4956cd2-0908-48bc-a17d-b4a05e960ba5 (Sealed, Cache) Sealing: false; Storage: true Local (/RAID1/data01) URL: http://10.1.80.4:2345/remote/sealed/s-t01904162-91 root@root:/RAID/filecoin/lotusminertwo# lotus-miner proving compute window-post 0 Took 1m5.681708627s ERROR: running window post failed: Rust panic: Lazy instance has previously been poisoned
I restored the moved sector to the original location and used the cbor file backed up by miner to restore miner, but the problem is still unresolved I restored the moved sector to the original location and used the cbor file backed up by the miner to restore the miner, but the problem was still unresolved. Finally, I decided to revert the miner version from 1.17.1 to 1.17.0, and then executed lotus miner providing compute window post 0 again. The result was successful. At this time, the remote lotus version was 1.17.1, and the local miner version was 1.17.0
root@root:/RAID/filecoin# lotus-miner proving compute window-post 1 Took 4m35.89821116s [{"Deadline":1,"Partitions":[{"Index":0,"Skipped":[0]}],"Proofs":[{"PoStProof":8,"ProofBytes":"obJ14wTG2xBCeXqz+pzFsM/J+w4A0NK02AsPYKnI04DWzhkHLHxi0704g8y+pP7gqf1DE2Ha1DUsQvlfce4E5B+mWFwEVdCyRj9OkIyqqaqwDWvmOVvG6VK2C6UShNnhBXRVgyEH8CQICJoN+nbTJvrlEAz6HQP/vcnvOTRvAkpUWWHpdjzcy3bs5jVULFO4qjJA1qQGbc8v5u6PX9WNQNuc1ZibQIEM3r9Xb/KLMqxPNEyoovly8qUfKjh56/yT"}],"ChainCommitEpoch":0,"ChainCommitRand":null}] root@root:/RAID/filecoin# lotus-miner proving compute window-post 0 Took 4m32.88303155s [{"Deadline":0,"Partitions":[{"Index":0,"Skipped":[0]}],"Proofs":[{"PoStProof":8,"ProofBytes":"sg8GHUn/G4YUmSLqntad4dsCZJ8vIdX/Yx/xCHSSJuGhUNgL/m2zdTVz6t9e7p74teQGqpO/eCUr6L5vZiS5UR3Mct8F+lJrdzna3B6zWnnb8kqNOrqbFpmvGCPQVXrWD+olh7GX90ZZJIfUtk67bd0rM/bBD7p5F40iGOeuAjCLEFaNd9lFX3wg4+UQ9lEVmFfyRcvKBYerkoiZ6ckn+ag7Uv8pVzPthZt1V/XmrOvxGQCIrpjPCEL36t7WqOQN"}],"ChainCommitEpoch":0,"ChainCommitRand":null}] root@root:/RAID/filecoin# lotus-miner proving compute window-post 2 Took 4m25.117250726s [{"Deadline":2,"Partitions":[{"Index":0,"Skipped":[0]}],"Proofs":[{"PoStProof":8,"ProofBytes":"t4Cjg9GUsM4zP2XIY3f74ShH0CeLer5U/2QcgdZcm0ojoei1SVfSSx4cppHsi1O/i7t2RP6L90DhsdVSXE5/ghQ05KsLNsqdoh2Et6QZt9hZIRih1u2wOheNlx5elbUYBNimJq4oypJL+LpKLEPTLy+rn24hNyVofq0x41jcW+42/LcnylSoxYnSnKwBPe+urB6BVeH/0GX6P+aoi8Lzb9NIFCO+BlwnITCLbNtcm9vnqVtIIrZNdqXNT1yBzxoh"}],"ChainCommitEpoch":0,"ChainCommitRand":null}]
Logging Information
Repo Steps