cannot query account balances before the mainnet upgrade height 5629650

[andy108369]

cannot query account balances before the mainnet upgrade

I am using full (archive) RPC node (tried different ones, but mainly using our primary AMS1 full RPC node) as well as different akash binary, v0.14.x, v0.15.x and v0.16.4.

Cannot query balances before the mainnet upgrade, whilst can query the market module without issues.

5629650 mainnet upgrade height - 2 months ago (2022-04-26 19:45:09) in prop akash-network/node#12.

# akash query bank balances --height 5629700 akash1vky0uh4wayh9npd74uqesglpaxwymynnspf6a4
balances:
- amount: "14619232085"
  denom: uakt
pagination:
  next_key: null
  total: "0"

# akash query bank balances --height 5629600 akash1vky0uh4wayh9npd74uqesglpaxwymynnspf6a4
balances: []
pagination:
  next_key: null
  total: "0"

Can query leases without issues though:

# akash query market lease list --height 5629000 --provider akash1vky0uh4wayh9npd74uqesglpaxwymynnspf6a4 --gseq 0 --oseq 0 --page 1 --limit 200 --state active -o json | jq -r '.leases | length'
98

Could be that there is a similar to https://github.com/ovrclk/akash/pull/1605/files fix is needed?

---

Precision falls by 100 for older than 25 days blocks

Update: fixed by using the node synced with pruning=nothing (see below)

Another interesting thing I've noticed is that the precision falls by 100 for older than 25 days blocks. I.e. when I query the RPC node for 63893XX height where XX is anything but not 00 would return no items, example:

# akash query bank balances --height 6389320 akash1vky0uh4wayh9npd74uqesglpaxwymynnspf6a4
balances: []
pagination:
  next_key: null
  total: "0"

# akash query bank balances --height 6389300 akash1vky0uh4wayh9npd74uqesglpaxwymynnspf6a4
balances:
- amount: "14502093352"
  denom: uakt
pagination:
  next_key: null
  total: "0"

Another example, querying the market module:

root@primary:~# akash query market lease list --height 5629000 --provider akash1vky0uh4wayh9npd74uqesglpaxwymynnspf6a4 --gseq 0 --oseq 0 --page 1 --limit 200 --state active -o json | jq -r '.leases | length'
98

root@primary:~# akash query market lease list --height 5629001 --provider akash1vky0uh4wayh9npd74uqesglpaxwymynnspf6a4 --gseq 0 --oseq 0 --page 1 --limit 200 --state active -o json | jq -r '.leases | length'
0

I am dropping the precision by 100, i.e. BLOCK=$(($BLOCK - $(expr $BLOCK % 100))) to make sure I am always getting the results when querying the blockchain for historical events (i.e. older than 25 days).

I can open a separate issue for this one, let me know.

[andy108369]

On the fallen precision, figured that this is how non-archiving node would behave with other than pruning = "nothing" setting:

• ~/.akash/config/app.toml:

# default: the last 100 states are kept in addition to every 500th state; pruning at 10 block intervals
# nothing: all historic states will be saved, nothing will be deleted (i.e. archiving node)
# everything: all saved states will be deleted, storing only the current state; 
pruning at 10 block intervals
# custom: allow pruning options to be manually specified through 'pruning-keep-recent', 'pruning-keep-every', and 'pruning-interval'
pruning = "default"

pruning should be set to nothing for the archiving node so the older balances can be queried.

I'm recreating the archiving node (pruning = nothing) right now from height=0, will close this case once fully confirm it.

[andy108369]

And it should be possible to query the older than height=5629650 transactions using akash 0.14.1 since this is where the SW upgrade has occurred:

$ akash query gov proposals | grep -A5 -B1 height:
    plan:
      height: "455200"
      info: ""
      name: akashnet-2-upgrade-1
      time: "0001-01-01T00:00:00Z"
      upgraded_client_state: null
    title: Akash Netwok' Software Upgrade Proposal
--
    plan:
      height: "5629650"
      info: ""
      name: akash_v0.15.0_cosmos_v0.44.x
      time: "0001-01-01T00:00:00Z"
      upgraded_client_state: null
    title: Akash Network - mainnet upgrade proposal

• at height=455200 it expects akash 0.12.0 - 0.14.1:

  
  $ git grep akashnet-2-upgrade-1 v0.12.0
  v0.12.0:app/app.go:     app.keeper.upgrade.SetUpgradeHandler("akashnet-2-upgrade-1", func(ctx sdk.Context, plan upgradetypes.Plan) {

$ git grep akashnet-2-upgrade-1 v0.14.1 v0.14.1:app/app.go: app.keeper.upgrade.SetUpgradeHandler("akashnet-2-upgrade-1", func(ctx sdk.Context, plan upgradetypes.Plan) {

- at `height=5629650` it expects akash `0.16.0` and above:

$ git grep akash_v0.15.0_cosmos_v0.44.x v0.16.0 v0.16.0:app/app.go: app.keeper.upgrade.SetUpgradeHandler("akash_v0.15.0_cosmos_v0.44.x", func(ctx sdk.Context, v0.16.0:app/app.go: if upgradeInfo.Name == "akash_v0.15.0_cosmos_v0.44.x" && !app.keeper.upgrade.IsSkipHeight(upgradeInfo.Height) {

[andy108369]

• RPC node akash 0.16.4, synced from height=0 and pruning=nothing;

On the fallen precision, can confirm that pruning=nothing solves this. Have tried akash client of either 0.16.3 or 0.16.4 - both are working.

$ akash-0.16.3 query bank balances --height 6389300 akash1vky0uh4wayh9npd74uqesglpaxwymynnspf6a4
balances:
- amount: "14502093352"
  denom: uakt
pagination:
  next_key: null
  total: "0"

$ akash-0.16.3 query bank balances --height 6389320 akash1vky0uh4wayh9npd74uqesglpaxwymynnspf6a4
balances:
- amount: "14502093352"
  denom: uakt
pagination:
  next_key: null
  total: "0"

Can also query the market lease module before the mainnet upgrade height 5629650 but only when running RPC node on akash 0.16.4 after this fix, it does not work when RPC is running 0.16.3.

$ akash-0.16.3 query market lease list --height 5629000 --provider akash1vky0uh4wayh9npd74uqesglpaxwymynnspf6a4 --gseq 0 --oseq 0 --page 1 --limit 200 --state active -o json | jq -r '.leases | length'
98

$ akash-0.16.3 query market lease list --height 5629001 --provider akash1vky0uh4wayh9npd74uqesglpaxwymynnspf6a4 --gseq 0 --oseq 0 --page 1 --limit 200 --state active -o json | jq -r '.leases | length'
98

Can even query the very old height:

$ akash-0.16.3 query market lease list --height 2921234 --provider akash1vky0uh4wayh9npd74uqesglpaxwymynnspf6a4 --gseq 0 --oseq 0 --page 1 --limit 200 --state active -o json | jq -r '.leases | length'
119

$ akash-0.16.3 query market lease list --height 90012 --provider akash1f6gmtjpx4r8qda9nxjwq26fp5mcjyqmaq5m6j7 --gseq 0 --oseq 0 --page 1 --limit 200 --state active -o json | jq -r '.leases | length'
6

But not the balances before the mainnet upgrade height 5629650:

$ akash-0.16.3 query bank balances --height 90012 akash1f6gmtjpx4r8qda9nxjwq26fp5mcjyqmaq5m6j7
balances: []
pagination:
  next_key: null
  total: "0"

$ akash-0.16.3 query bank balances --height 5629649 akash1f6gmtjpx4r8qda9nxjwq26fp5mcjyqmaq5m6j7
balances: []
pagination:
  next_key: null
  total: "0"

$ akash-0.16.3 query bank balances --height 5629650 akash1f6gmtjpx4r8qda9nxjwq26fp5mcjyqmaq5m6j7
balances:
- amount: "4789217453"
  denom: uakt
pagination:
  next_key: null
  total: "0"

---

However, still cannot query the account balances before the mainnet upgrade height 5629650.

• checking the pre-mainnet upgrade height 5629650:

$ akash-0.16.4 query bank balances --height 5629650 akash1365yvmc4s7awdyj3n2sav7xfx76adc6dnmlx63
balances:
- amount: "2932790"
  denom: uakt
pagination:
  next_key: null
  total: "0"

$ akash-0.16.4 query bank balances --height 5629649 akash1365yvmc4s7awdyj3n2sav7xfx76adc6dnmlx63
balances: []
pagination:
  next_key: null
  total: "0"

Have tried the akash 0.16.4, 0.16.3, 0.16.2, 0.14.1, 0.10.1 - no success :/

[boz]

There could have been a migration in the upgrade that affected the table(s) that the bank balances query uses. A current node looking at pre-migrated database versions wouldn't see anything in the place that it expects to see things.

[andy108369]

That explains, thanks @boz ! Would it be difficult to add the support for querying the pre-upgrade balances?

[boz]

That explains, thanks @boz ! Would it be difficult to add the support for querying the pre-upgrade balances?

yeah, it's not practical to do at all.

[andy108369]

That explains, thanks @boz ! Would it be difficult to add the support for querying the pre-upgrade balances?

yeah, it's not practical to do at all.

What would be the most practical way to query the pre-upgrade balances, other than querying the TXs with akash query txs --events "tx.height=11" --limit 1000 and re-indexing each?

Running 0.14.1 would just stop at height=5629650, unless maybe one syncs it with AKASH_HALT_HEIGHT=5629649 (1 block before the mainnet upgrade height) and then restarts it in the following way:

• addrbook.json removed;
• seeds, persistent_peers, unconditional_peer_ids, private_peer_ids, AKASH_HALT_HEIGHT unset;
• pex=false;
• max_num_inbound_peers=0;