Unable to query the oasis node

[Swornatoya]

The node stopped syncing and I had upgraded the oasis node to v2.6.0. When I check the logs of the node it is syncing fine as shown in the below image.

When I try to run the below curl query against the node. curl --location 'http://************/block' \ --header 'Content-Type: application/json' \ --data '{ "network_identifier": { "blockchain": "Oasis", "network": "b11b369e0da5bb230b220127f5e7b242d385ef8c6f54906243f30af63c815535" }, "block_identifier": { "index": 16840161 } }'

I am getting response as { "code": 0, "message": "network identifier {\"blockchain\":\"Oasis\",\"network\":\"b11b369e0da5bb230b220127f5e7b242d385ef8c6f54906243f30af63c815535\"} is not supported: requestNetwork is not supported", "retriable": false }

Please let me know if the network identifier has been updated.

[pro-wh]

yup, it's been updated. use the new network bb3d748def55bdfb797a2ac53ee6ee141e54cd2ab2dc2375f4a0703a178e6e55

from https://docs.oasis.io/node/mainnet/

[Swornatoya]

@pro-wh Thanks for your response.

When I try querying the latest block synced using the upgraded node I am getting the response. But if I try to query the older blocks it does not work and I get the below response.

Curl Query: curl --location 'http://10.213.67.23:7332/block' \ --header 'Content-Type: application/json' \ --data '{ "network_identifier": { "blockchain": "Oasis", "network": "bb3d748def55bdfb797a2ac53ee6ee141e54cd2ab2dc2375f4a0703a178e6e55" }, "block_identifier": { "index": 16810161 } }'

Response: { "code": 12, "message": "unable to get block", "retriable": true }

[pro-wh]

that latest synced block index can't be right. the bb3d748def55bdfb797a2ac53ee6ee141e54cd2ab2dc2375f4a0703a178e6e55 mainnet started at block 16817956. that mainnet can't answer about blocks from before it started

[Swornatoya]

@pro-wh : Then how do we get the data prior to the block 16817956 from our node.

[pro-wh]

run a separate node from the prior network, and run a separate oasis rosetta gateway connected to that node to answer for blocks in that network. it's a lot of trouble, we're aware. sorry about that

[Swornatoya]

@pro-wh : Tried running the old node with the network b11b369e0da5bb230b220127f5e7b242d385ef8c6f54906243f30af63c815535 . But the node is not syncing and when try to check the latest block query it returns the genesis block and also getting the below error in the node.

{"caller":"pex_reactor.go:531","level":"info","module":"tendermint:pex","msg":"No addresses to dial. Falling back to seeds","ts":"2023-12-11T10:58:09.941068528Z"} {"address":"e27f6b7a350b4cc2b48a6cbe94b0a02b0dcb0bf3@35.199.49.168:26656","caller":"switch.go:716","level":"info","module":"tendermint:p2p","msg":"Dialing peer","ts":"2023-12-11T10:58:09.941084628Z"} {"caller":"pex_reactor.go:632","err":"incompatible: peer is on a different network. Got bb3d748def55bdfb797a2ac53ee6ee141e54cd2ab2dc2375f4, expected b11b369e0da5bb230b220127f5e7b242d385ef8c6f54906243","level":"error","module":"tendermint:p2p","msg":"Error dialing seed","seed":"e27f6b7a350b4cc2b48a6cbe94b0a02b0dcb0bf3@35.199.49.168:26656","ts":"2023-12-11T10:58:10.065524383Z"}

And when I try to query the block data from the node with the below curl query curl --location 'http://********/block' \ --header 'Content-Type: application/json' \ --data '{ "network_identifier": { "blockchain": "Oasis", "network": "b11b369e0da5bb230b220127f5e7b242d385ef8c6f54906243f30af63c815535" }, "block_identifier": { "index": 8048956 } }' Getting the output as : {"code":12,"message":"unable to get block","retriable":true}

[pro-wh]

the seed nodes are now on the new network, so you won't be able to connect to a network using it. you can run a node in "archive" mode, where it won't connect to the p2p network. you'll need to run it with your own data, i.e. from before the wipe in the upgrade steps, as it won't sync data from a network.

[Swornatoya]

We are running the oasis node using the below docker compose file Even after mapping the volume of either :/data or :/node, we don’t get the volumes at NewDisk23Machine/node-oasis-rosetta/ But if you enter inside the container (with docker exec -it), it shows files and data is present inside Ideally all this content should get mapped to host machine volume path as given in compose file. But it doesn’t. Instead, this container data goes to docker storage volumes and sits there. So every time the node container restarts, the volume is lost and node starts from scratch. And now with the new network id and upgraded node we are unable to get the data before the block 16817956

[pro-wh]

the volume for /data is commented out in the screenshot above

[Swornatoya]

@pro-wh : Even when we uncomment the volume mapping for /data we don't get the volumes mapped to our node folder

[pro-wh]

could it be a permissions issue? the mounted directory needs to be www-data:www-data (33) and be mode 0700. check the container logs for diagnostics. here's how I got it working locally:

mkdir -p /tmp/rosetta-data/etc
chmod 700 /tmp/rosetta-data
cp config.yml /tmp/rosetta-data/etc
wget -O /tmp/rosetta-data/etc/genesis.json https://github.com/oasisprotocol/mainnet-artifacts/releases/download/2023-11-29/genesis.json
sudo chown -R www-data:www-data /tmp/rosetta-data
docker run -it -v /tmp/rosetta-data:/data rosetta-priv:latest

(edit: chown later for convenience)

then

sudo ls -al /tmp/rosetta-data

shows files

[Swornatoya]

@pro-wh : The image rosetta-priv:latest that you are using is it the latest and we need to use this image. As because the image we are currently using is with version v2.6.0 and this we had build using the repo from the github.

[pro-wh]

tried with 2.6.0, works with that too. there are no changes to how the docker image uses that directory between 2.6.0 and master

[Swornatoya]

@pro-wh : Tried to start the node with the backup that we had to read the block data before 16817956. But getting error as below: {"Version":"22.2.8-gitc159d0364","caller":"node.go:554","level":"info","module":"oasis-node","msg":"Starting oasis-node","ts":"2024-01-19T12:31:49.083307905Z"} {"caller":"helpers.go:40","level":"info","module":"common/persistent","msg":"All 1 tables opened in 0s","ts":"2024-01-19T12:31:49.092521167Z"} {"caller":"helpers.go:40","level":"info","module":"common/persistent","msg":"Discard stats nextEmptySlot: 0","ts":"2024-01-19T12:31:49.096822896Z"} {"caller":"helpers.go:40","level":"info","module":"common/persistent","msg":"Set nextTxnTs to 17295617","ts":"2024-01-19T12:31:49.097009697Z"} {"caller":"helpers.go:40","level":"info","module":"common/persistent","msg":"Deleting empty file: /data/persistent-store.badger.db/050796.vlog","ts":"2024-01-19T12:31:49.097095498Z"} {"caller":"upgrade.go:171","err":"binary consensus version not compatible: own: 6.0.0, required: 7.0.0","handler":"mainnet-upgrade-2023-11-29","level":"error","log_event":"upgrade/incompatible-binary","module":"upgrade","msg":"incompatible binary version for upgrade","ts":"2024-01-19T12:31:49.097705402Z"} {"caller":"node.go:599","err":"binary consensus version not compatible: own: 6.0.0, required: 7.0.0","level":"error","module":"oasis-node","msg":"failed to initialize upgrade backend","ts":"2024-01-19T12:31:49.097754102Z"} {"caller":"helpers.go:40","level":"info","module":"common/persistent","msg":"Lifetime L0 stalled for: 0s","ts":"2024-01-19T12:31:49.097771802Z"} {"caller":"helpers.go:40","level":"info","module":"common/persistent","msg":"Level 0 [ ]: NumTables: 01. Size: 3.3 MiB of 0 B. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 64 MiB\nLevel 1 [ ]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB\nLevel 2 [ ]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB\nLevel 3 [ ]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB\nLevel 4 [ ]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB\nLevel 5 [ ]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB\nLevel 6 [B]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB\nLevel Done","ts":"2024-01-19T12:31:49.103776442Z"}

[pro-wh]

from our team:

did they enable archive mode?

https://docs.oasis.io/node/run-your-node/archive-node#configuration

[pro-wh]

quick correction to the docs:

The setting is now just mode (top level, and not consensus.tendermind.mode as the docs say)

edit: docs updated too