search

vegaprotocol / data-node

A rich API server for Vega Protocol
https://vega.xyz
Other
3 stars 1 forks source link

Double counting delegations (sometimes) #697

Closed wwestgarth closed 2 years ago

wwestgarth commented 2 years ago

Problem encountered

During this overnight run some delegation tests failed.

Looking at the logs for test_a_selfDelegation, we see the following:

Running staking/self-delegations for 5e7f1efdef84df790ac231d7910d478c6528a0db256ec51989cd7b2b5311ff7e (2 VEGA token), start stake 3760000000000000000
Delegate Submission {'delegateSubmission': {'nodeId': 'b9a675e839ed7335acf3654742f0e0ab61125db3e784d59f467c7930c7f88463', 'amount': '2000000000000000000'}}
delegation not seen in data node expected {start_stake + int(str_tokens)} got {int(updated_stakedByOperator)}"
AssertionError: delegation not seen in data node expected 5760000000000000000 got 9520000000000000000

So we start with a stake of 376, then delegate 200 more but end up with 952 = (2 * 376) + 200 instead of 576 = 376 + 200. So I believe the datanode might be going strange and double counting something?

The events reported by vega look correct in that we have delegation balances of 376 for a few epochs, and then 576 for a few epochs: https://jenkins.ops.vega.xyz/job/common/job/system-tests/2740/artifact/test_logs/workspace.tests.staking_delegation.test_selfDelegation/test_a_selfDelegation/BUS_EVENT_TYPE_DELEGATION_BALANCE.txt

Interestingly in the subsequent test test_b_node_undelegate which tries to undelegate we report a start stake of 952 from the previous test, and undelegate 100 and so expect a resultant stake of 852. Instead we get 476 which is 376 + 200 - 100 the actual true value you would expect from this chain of tests. Like its popped back again and self-healed.

Observed behaviour

Delegation reporting from the data-node sometimes goes wrong.

Expected behaviour

Delegation reporting from the data-node never goes wrong.

System response

Describe what the system response was, include the output from the command, automation, or else.

Steps to reproduce

Manual

Steps to reproduce the behaviour manually:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Automation

Link to automation and explanation on how to run it to reproduce the problem/bug

Evidence

Logs

If applicable, add logs and/or screenshots to help explain your problem.

Additional context

Add any other context about the problem here including; system version numbers, components affected.

Definition of Done

ℹ️ Not every issue will need every item checked, however, every item on this list should be properly considered and actioned to meet the DoD.

Before Merging

After Merging

wwestgarth commented 2 years ago

This is still happening: https://jenkins.ops.vega.xyz/blue/organizations/jenkins/common%2Fsystem-tests/detail/system-tests/2885/pipeline

https://github.com/vegaprotocol/vega/issues/5491

vega-paul commented 2 years ago

We also see multiple delegation objects for same party/object in the /nodes response, for e.g.

      {
          "party": "21ddd19abb3ed5924bb6d807f167d03cdc5b8b455f76207f0bd2dd9f15682b9f",
          "nodeId": "02726c00bf1434b1efb7701c328bacb352f6bf2310d2591ba8db19c14aec6eb2",
          "amount": "1000000000000000000000000",
          "epochSeq": "78"
        },
        {
          "party": "21ddd19abb3ed5924bb6d807f167d03cdc5b8b455f76207f0bd2dd9f15682b9f",
          "nodeId": "02726c00bf1434b1efb7701c328bacb352f6bf2310d2591ba8db19c14aec6eb2",
          "amount": "1000000000000000000000000",
          "epochSeq": "78"
        },
        {
          "party": "21ddd19abb3ed5924bb6d807f167d03cdc5b8b455f76207f0bd2dd9f15682b9f",
          "nodeId": "02726c00bf1434b1efb7701c328bacb352f6bf2310d2591ba8db19c14aec6eb2",
          "amount": "1000000000000000000000000",
          "epochSeq": "78"
        },
vega-paul commented 2 years ago

Another issue which may be related

When a user delegates, the API nodes response shows an increase in stakedByDelegates X 2 and stakedByOperator is updated with the original staked value in this field so we end up with stakedByOperator X 2

for e.g.

Before values

      "stakedByOperator": "1000000000000000000",
      "stakedByDelegates": "0",
      "stakedTotal": "1000000000000000000",

user delegates 

Delegate Submission {'delegateSubmission': {'nodeId': '091df5a1af9febf5bbedfcab641dabe0f585eadd19f4bcb70176537489d7df08', 'amount': '1000000000000000000000000'}} in block 324 epoch 43
2022-06-21 16:38:12.627:[INFO] MainProcess walletsMod.py:wallet_sign_submit_txn_v2:234 ----- Signing transaction for trader EC59E8AEE4 pubkey 7db5cefdf2ad2b613c1b68db064f317acd223f08320e6f001c77ef8d8484eb4b
2022-06-21 16:38:12.748:[INFO] MainProcess contractMod.py:Delegate_Submission:1150 ----- Wallet signing {'txHash': '82BC425C6756F0FFED01CF215F744AF7FFB8F31BFF7C04A62A9335E4A2C0ECA4', 'receivedAt': '2022-06-21T16:38:12.691326+01:00', 'sentAt': '2022-06-21T16:38:12.709635+01:00', 'txId': 'UJCtitHyCPR7wRyYVKL0', 'tx': {'input_data': 'CLS54PuNn5et1AEQxQKCP10KQDA5MWRmNWExYWY5ZmViZjViYmVkZmNhYjY0MWRhYmUwZjU4NWVhZGQxOWY0YmNiNzAxNzY1Mzc0ODlkN2RmMDgSGTEwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDA=', 'signature': {'value': 'e20e817bb7bf3b38117b19cc42887304544dccc9530e4e27c2f121489472db3d047b42489e26405d29897816399ecfba26b48671b75f394edde77f02247fbb0f', 'algo': 'vega/ed25519', 'version': 1}, 'From': {'PubKey': '7db5cefdf2ad2b613c1b68db064f317acd223f08320e6f001c77ef8d8484eb4b'}, 'version': 2, 'pow': {'tid': '19671E9B12ACA89BEAE83098A89B002CD5D5725E525FFBD946E0DFE15E07E51D', 'nonce': 3}}}```

node values

  "stakedByOperator": "2000000000000000000",
  "stakedByDelegates": "2000000000000000000000000",
  "stakedTotal": "2000002000000000000000000",