Rosetta API and near CLI are reporting 2 different balances for given account

[asmolarek-bc]

Describe the bug The scenario is as follows:

I have account with ~10k near and I am doing 5 transfers 500 NEAR each to implicit accounts that do not exists at the time of transferring NEAR, transaction will create them (transactions are created with rosetta api)

after that I am transferring funds from all of those accounts to single implicit account that does not exists yet and will be created with first transfer, lets call this account Alice Account

for those transactions I am using rosetta api as well, I am requesting rosetta to prepare transaction that will move 500NEAR

All transactions are executed and accepted, after that I am asking rosetta api for balance of Alice account, also I am checking Alice account with near cli

those are the responses I am getting:

Rosetta returns 2499900737200000000000000000 Near cli returns 2499917281000000000000000000

they do not sum up to 2500near, as expected because there are some transactions fees that have to be paid

what is wrong is that they are not the same

Version (please complete the following information):

• nearcore tag 1.21.0
• rustc 1.51.0
• rustup 1.24.3 (ce5817a94 2021-05-31)
• cargo 1.51.0
• docker Docker version 20.10.8, build 3967b7d
• local env with local network

node was compiled with --features rosetta_rpc and started with rosetta api enabled in conf file

[bowenwang1996]

@frol could you take a look at this issue?

[frol]

@asmolarek-bc Rosetta RPC reports the balances in the following way:

https://github.com/near/nearcore/blob/0defc3c9bf480adaf05aae545e12d0b9f58c1663/chain/rosetta-rpc/src/lib.rs#L386-L400

1. main account (without subaccount) gives you the liquid balance
2. LIQUID_BALANCE_FOR_STORAGE subaccount under the main account exposes the balance that is locked for the storage
3. LOCKED subaccount under the main account represents the natively staked tokens

JSON RPC has only locked and "the rest" breakdown.

Thus, 2499917281000000000000000000 (near CLI) = 2499900737200000000000000000 (Rosetta main account balance) + small amount of NEAR locked for storage (Rosetta LIQUID_BALANCE_FOR_STORAGE subaccount balance)

[asmolarek-bc]

yes, I am aware of sub accounts for rosetta, but from what I see is that I am getting only one balance, the one that I posted, and near CLI and near RPC are returning as balance this 2499917281000000000000000000 value as amount of tokens hold by that account

@frol you pointed interesting thing, does it mean that there is storage staking in case of transferring funds to newly created account? I mean when I transfer 500 NEAR to implicit account what should be the balance of that newly created implicit account? 500? or 500 - LIQUID_BALANCE_FOR_STORAGE? if the second option is correct how is LIQUID_BALANCE_FOR_STORAGE determined?

[asmolarek-bc]

to double check I executed the test again and this is what I am getting from the near cli:

{
  amount: '2499917281000000000000000000',
  block_hash: 'BZNm24u48zpbGSA1iZz27qtQXsJ1GDSJKnZtBAHXuHd',
  block_height: 621,
  code_hash: '11111111111111111111111111111111',
  locked: '0',
  storage_paid_at: 0,
  storage_usage: 182,
  formattedAmount: '2,499.917281'
}

no indication of any locked funds

[asmolarek-bc]

Are the LIQUID_BALANCE_FOR_STORAGE funds truly locked, or is this that rosetta interprets them as locked and always keeps that LIQUID_BALANCE_FOR_STORAGE amount on the account?

[asmolarek-bc]

to be sure that the integration is working as expected I installed rosetta cli (separate tool for debugging) and checked if there is any additional account

rosetta-cli view:balance '{"address":"account.hex.address"}' 450
loaded configuration file: /rosetta-cli.json
Amounts: [
 {
  "value": "2499900737200000000000000000",
  "currency": {
   "symbol": "NEAR",
   "decimals": 24
  }
 }
]
2021/10/27 10:34:14 Metadata: null
2021/10/27 10:34:14 Balance Fetched At: {
 "index": 450,
 "hash": "Ci6i1Qb7UWX42YgAbHzMjiFgDL1yFBoLRJiBfsYkLrTy"
}

[bowenwang1996]

@frol you pointed interesting thing, does it mean that there is storage staking in case of transferring funds to newly created account?

Yes. Storage staking applies to all accounts.

I mean when I transfer 500 NEAR to implicit account what should be the balance of that newly created implicit account?

For an account with one access key, you need at least 0.0182N on the account.

Also the Rosetta subaccounts are virtual and we use that as a way to represent balances. There are no actual subaccounts with those balances on chain.

[asmolarek-bc]

Also the Rosetta subaccounts are virtual and we use that as a way to represent balances. There are no actual subaccounts with those balances on chain.

thx for the answer

what is the best way to get the balance on virtual sub account, the one that is for LIQUID_BALANCE_FOR_STORAGE ?

I posted output from rosetta cli and it is not there, rosetta-cli view:balance reports only that json that you se above, LIQUID_BALANCE_FOR_STORAGE is not reported by near cli nor by near rpc

it is important for me to have 2 ways of checking the account balance first to reconstruct given balance from transactions / receipts that I see in the blocks second to ask node what is the balance on chain

with those 2 I have a way to verify if I am processing transactions correctly

[mikedotexe]

Having discussions with the team, sounds like we'd like to reopen this issue.

[frol]

I will take a closer look next week

[frol]

what is the best way to get the balance on virtual sub account, the one that is for LIQUID_BALANCE_FOR_STORAGE ?

If your questions are about rosetta-cli or API usage, Rosetta support channels are better to be asked. I also was confused by the help of rosetta-cli utility, but I somehow remembered how I used it (basically, it expects raw JSON input for /account/balance endpoint):

$ ./rosetta-cli-0.7.2-linux-amd64 --configuration-file ./rosetta.cfg view:balance '{"address": "frol4.testnet"}'
loaded configuration file: ./rosetta.cfg
2021/11/10 12:10:28 Amounts: [
 {
  "value": "383348266956168680085420257",
  "currency": {
   "symbol": "NEAR",
   "decimals": 24
  }
 }
]
2021/11/10 12:10:28 Metadata: null
2021/11/10 12:10:28 Balance Fetched At: {
 "index": 70799385,
 "hash": "83SWfsF8HFnZvrBYWnpMgoJ4zAuqCBpUaSLbhfRvkMPR"
}

$ ./rosetta-cli-0.7.2-linux-amd64 --configuration-file ./rosetta.cfg view:balance '{"address": "frol4.testnet", "sub_account": {"address": "LIQUID_BALANCE_FOR_STORAGE"}}'
loaded configuration file: ./rosetta.cfg
2021/11/10 12:12:49 Amounts: [
 {
  "value": "17100000000000000000000",
  "currency": {
   "symbol": "NEAR",
   "decimals": 24
  }
 }
]
2021/11/10 12:12:49 Metadata: null
2021/11/10 12:12:49 Balance Fetched At: {
 "index": 70799598,
 "hash": "EJswSoMgy56eJyrzvWFJuxRy7JCn9LcbmdDLrg3QVueq"
}

Let's compare it to JSON RPC response:

$ curl https://rpc.testnet.near.org -H 'Content-Type: application/json' --data '{"jsonrpc": "2.0", "method": "query", "params": {"request_type": "view_account", "account_id": "frol4.testnet", "finality": "final"}, "id": "dontcare"}'
{
  "jsonrpc": "2.0",
  "result": {
    "amount": "383365366956168680085420257",
    "block_hash": "BH1KuDcYDFMeW8ijtpv8TrGuGxVeA2HDbtxE15smNKMw",
    "block_height": 70799922,
    "code_hash": "11111111111111111111111111111111",
    "locked": "0",
    "storage_paid_at": 0,
    "storage_usage": 1710
  },
  "id": "dontcare"
}

The math adds up: 383348266956168680085420257 + 17100000000000000000000 = 383365366956168680085420257

[asmolarek-bc]

@frol thank you for the response

no, question was not about rosetta cli usage, I used rosetta to debug if my integration is working, and I provided it as an example that can be reproduced without the code base I am working on, thx for the explanation

[asmolarek-bc]

rosetta-cli is debug tool, when I passed the JSON you posted it returned balance of the sub account

rosetta-cli view:balance '{"address":"account.hex.address", "sub_account": {"address": "LIQUID_BALANCE_FOR_STORAGE"}}' 1450
loaded configuration file: /rosetta-cli.json
2021/11/10 13:15:52 Amounts: [
 {
  "value": "16543800000000000000000",
  "currency": {
   "symbol": "NEAR",
   "decimals": 24
  }
 }
]
2021/11/10 13:15:52 Metadata: null
2021/11/10 13:15:52 Balance Fetched At: {
 "index": 1450,
 "hash": "CBMhZnZWmUbAVvzLw8FuczbbFRmmXLCSrxrMDrNUhpTa"
}

it requires to specify explicitly sub account, which was not there when I was testing it the first time

[asmolarek-bc]

what is a bit off is that sub account returned 16543800000000000000000 and storage_usage for this account is 182 which means that the storage_amount_per_byte is 9.09 × 10^19 but near RPC returns storage_amount_per_byte 10^19 is there a configuration that has to be set when running rosetta on local env? this is described more here https://github.com/near/nearcore/issues/5129

[frol]

which means that the storage_amount_per_byte is 9.09 × 10^19

Well, that is weird. Which network is that? If you look at my snippets (tested on testnet), there is 1710 bytes used and 17100000000000000000000 yoctoNEAR locked.

[asmolarek-bc]

yes exactly I see that in your case it is different, my network is local net that I am starting like this

nearcore/target/release/neard --home ~/.near/testnet run

and .near is created with

nearcore/target/release/neard --home ~/.near/testnet init

[frol]

@asmolarek-bc from your #5129 I noticed that you use 1.21.0 release, which indeed produces a genesis file with "storage_amount_per_byte": "90900000000000000000",. I am using 1.22.0-rc.4, which does not generate this section to genesis config at all (it is a separate question), but the storage price there is 10^20.

Either edit your genesis file or adjust your expectations accordingly :smile:

[asmolarek-bc]

thx for the explanation, I see that it is set in genesis, I can update it accordingly to be the same as in the docs, but why despite it is set to 90900000000000000000 in the genesis it is reported as 10^19 via near RPC Is it possible that this value evolves while blocks are produced and rosetta api only check the value from the genesis and do not update it accordingly?

[frol]

why despite it is set to 90900000000000000000 in the genesis it is reported as 10^19 via near RPC

It is a separate problem, indeed. How do you check that value?

[asmolarek-bc]

I see this value 90900000000000000000 as amount that rosetta is reporting as LIQUID_BALANCE_FOR_STORAGE (182 * 9.09 × 10^19)

the other value I see from near rpc endpoint EXPERIMENTAL_protocol_config

[frol]

@asmolarek-bc I have reported it as a separate issue: #5197

[asmolarek-bc]

Thank you