edge case: unstaked tokens haven't been released/can't be unstaked while unstaking after a jailing by downtime after the upgrade from v0.9.13 to v0.10.0

McDaan commented 1 month ago

Description and context

I found out what it looks like an edge case while I was providing technical support today to one folk at Discord. He was trying to unstake his stack because his validator became jailed due downtime after the upgrade from v0.9.13 to v0.10.0 happened. He finally didn't upgrade - he remained in v0.9.12, trying to figure out how to upgrade, restarting multiple times the server, and even deleting the entire data folder, starting then resyncing from scratch, which caused the jailing by downtime.

So, at first try, according to a transaction hash he provided, he has been able to run successfully the unstake command https://testnet.storyscan.xyz/tx/0x3e53cf3bd5bb341cd8819568ee816ab522a92255db3a9242cc62aa0b13771db3?tab=index, but after the unbonding period (3600s, i.e. 1 hour) the tokens didn't reach out his validator wallet.

The odd thing comes now, at second try, after waiting for the unbonding period and then retrying, he got this error telling that the "validator does not exist", but, guess what? Yes, it does exist, refer to https://api-story-testnet.trusted-point.com/cosmos/staking/v1beta1/validators/storyvaloper1xazxfvwgk3u8zmas9p4yjxgcph07xzq3065dx0

https://testnet.story.explorers.guru/validator/374464B1C8B478716FB0286A4919180DDFE30811

Error log provided by him:

ERRO !! Fatal error occurred, app died️ unexpectedly !! err="failed to estimate gas: execution reverted: IPTokenStaking: Validator does not exist" stacktrace="[errors.go:39 transaction.go:118 transaction.go:48 validator.go:567 validator.go:520 validator.go:239 validator.go:296 command.go:983 command.go:1115 command.go:1039 command.go:1032 cmd.go:34 main.go:10 proc.go:271 asm_amd64.s:1695]"

The command he was using it's the same he used at first try, but leaving it here, just in case:

story validator unstake \
  --validator-pubkey A9r+sVF6aMdPH2BJ0i0F9XQTR3xIcKCMLREI1OOA2ar+ \
  --unstake 7413000000000000000000 \
  --private-key xxx

One thing to note is that a prior he used the 7413000000000000000000 at first despite of having a little less balance (according to https://testnet.story.explorers.guru/account/story1xazxfvwgk3u8zmas9p4yjxgcph07xzq3p4qvdy). Anyway, he then tried with 7412000000000000000000 several times after I suggested so (even suggested to use other RPC endpoint, with the same result).

Extra information that may be related to this issue

Another folk, which validator is also in the inactive set due to jailing by downtime and with staked amoun, wanted to unstake but experienced this error despite of having sufficient staked amount to be unstaked (https://testnet.story.explorers.guru/validator/D5258A5CF5A85707DAA6B9488D631DFD2A45EE97):

ERRO !! Fatal error occurred, app died️ unexpectedly !! err="failed to estimate gas: execution reverted: IPTokenStaking: Insufficient staked amount" stacktrace="[errors.go:39 transaction.go:118 transaction.go:48 validator.go:567 validator.go:520 validator.go:239 validator.go:296 command.go:983 command.go:1115 command.go:1039 command.go:1032 cmd.go:34 main.go:10 proc.go:271 asm_amd64.s:1695]

He then tried to unstake, especifically, an amount less than 1 IP 1000000 (to be precise) and unstake transaction went through.

Experienced behavior

The unstaked tokens haven't reached out the validator wallet at first try after the unbonding period (3600s, i.e. 1 hour), and at second try an error log telling that his "validator does not exist" suddenly shown up.

Expected behavior

The unstaked tokens must have been reached out the validator wallet at first try and after the unbonding period (3600s, i.e. 1 hour).

edisonz0718 commented 1 month ago

I believe there are 2 issues here:

Since he is jailed, that's why the unstaking wasn't successful, which is the expected behavior. Ideally he needs to first send a tx to unjail and then unstake. However, there is a bug in our unjail logic. It's going to be fixed in the next hard fork release that to be announced soon.
validator does not exist: This is a known issue on our side. The issue is that our staking smart contract loses track of what happens in the validator node since your friends unstake twice. We plan to remove this checking logic later.

McDaan commented 1 month ago

I believe there are 2 issues here:

Since he is jailed, that's why the unstaking wasn't successful, which is the expected behavior. Ideally he needs to first send a tx to unjail and then unstake. However, there is a bug in our unjail logic. It's going to be fixed in the next hard fork release that to be announced soon.

validator does not exist: This is a known issue on our side. The issue is that our staking smart contract loses track of what happens in the validator node since your friends unstake twice. We plan to remove this checking logic later.

Great. Hope to see both issues I/we found out resolved. And they aren't strictly my friends, I mean, I'm reporting these issues as a contributor of Story Protocol - they are issues from "folks" in general that I found out while providing continuous technical support in the validator channels :)

Have a great day @edisonz0718!

piplabs / story