search

oasisprotocol / oasis-web3-gateway

Web3 Gateway for the Oasis SDK EVM ParaTime module.
GNU General Public License v3.0
27 stars 14 forks source link

web3.eth.getTransactionCount hangs #161

Open bugbuzz opened 2 years ago

bugbuzz commented 2 years ago

Hi! web3.eth.getTransactionCount from web3.js hangs and I don't see it in my emerald-web3-gateway logs.

What I'm trying to do is

await web3.eth.getTransactionCount('0x0000000000000000000000000000000088888888', 'latest')

Meanwhile 

await web3.eth.getBlock('latest')

works pretty well.

Also, some eth_calls hangs too, but in this case I can see these calls in my gateway logs. I will create another ticket for this case if I won't figure out the issue.

My configs

I have 

config:
  "000000000000000000000000000000000000000000000000e2eaa99fc008f87f":
    allow_expensive_queries: true

in my oasis-node config.

And

database:
  dial_timeout: 5
  read_timeout: 10
  write_timeout: 5
  max_open_conns: 0

gateway:
  method_limits:
    get_logs_max_rounds: 200

in my gateway config.

oasis-node --version
Software version: 21.3.8
Consensus:
  Consensus protocol version: 4.0.0
Runtime:
  Host protocol version:      4.0.0
  Committee protocol version: 3.0.0
Go toolchain version: 1.16.13

emerald-paratime: v6.1.0

emerald-web3-gateway --version
Software version: 1.1.0
Oasis SDK version: v0.1.1-0.20211203131800-7bc6c485ad2f
Go toolchain version: go1.17.6
kostko commented 2 years ago

What does oasis-node control status -a unix:/path/to/datadir/internal.sock say?

bugbuzz commented 2 years ago 
oasis-node control status -a unix:/home/ubuntu/oasis/data/internal.sock
{
  "software_version": "21.3.8",
  "identity": {
    "node": "OyhgWXbOih3w8VRWsr3QRISBBCRsiKuzcJb+fONf5dQ=",
    "p2p": "2u8EHJ3TR+xLIRuYULtXCwGCigRDnrsuD2NC4mJjcqU=",
    "consensus": "Lkf4Hxny1b30zyVnYGzHTjOlAQb5s0264aoJSpmdqZI=",
    "tls": [
      "zbvbXAxeMvlwP/P7F5deXU0R0rlWo+AwoymVhYcOYrs=",
      "9X0F8CaT/8T3IRa7I+HHEZW6TNYk1zj1mYnRvmPUNO8="
    ]
  },
  "consensus": {
    "version": {
      "major": 4
    },
    "backend": "tendermint",
    "features": 3,
    "node_peers": [
      "dcdb7c94bc29d4e5b5591380ae838ff53c927c58@95.216.23.223:36656",
      "e2d7720df78f98ef28ced72728d853ae75ace5af@185.107.80.40:26656",
      "b22d7ed700f22390e0f4e3de051561b032a80818@152.228.227.65:26656",
      "16dd39659dac59950ecb861bc92dab3ed0052663@164.68.120.119:26656",
      "5f40ab6f70a4ba09f4553521926948705f5f3e89@194.163.169.33:26656",
      "3e5c6ae956cf079f4158dfd130b9b5d3308d1a33@91.90.179.51:26656",
      "7fecaa21b4cdeb14785a1327104813fc426efd5d@95.216.241.124:26656",
      "9d29bf3d541b64b5470b9f8aa76ff8b2c8587f98@34.86.108.137:26656",
      "f889a426e598b127cdbea2d57ce1ab310d796cca@142.44.137.137:26656",
      "2900fe0aefe45819df89e11102d31d0df429a81d@20.51.115.71:26656",
      "2f8d01f566bb5e6f75a6321d01367eb53671f9c1@159.138.147.15:21656",
      "d6fb7f7e3a7f74cf1d9ce9a31a0b08305e88e15b@18.183.66.241:26657",
      "7f3aea932edcfbe219be89780317b815927aca9c@159.89.91.252:26656",
      "ed1acdfa10acd4206d5131a7286697d183c9dc25@209.126.6.224:26656",
      "34e63fcb16b782ece5f28ca781de5c88a947dedf@131.153.99.14:26656",
      "8847c280b3d7dfc8cb59f15bf5dcaa8b270e9792@178.154.228.42:26656",
      "82321eea39873a9fa33e793605a250c7649ee92c@95.216.45.250:20656",
      "261cb7910da6b73c5d60abe205b78b570b6025d2@23.88.105.64:26656",
      "d8b549e447292d1b36987a70836e0b59e3b75121@54.208.50.80:26656"
    ],
    "latest_height": 6989104,
    "latest_hash": "08f547127ddeaaeee11edac0a6bc5300cd2753530fb6f616490fc9ba066e5224",
    "latest_time": "2022-01-27T11:50:29+01:00",
    "latest_epoch": 11638,
    "latest_state_root": {
      "ns": "0000000000000000000000000000000000000000000000000000000000000000",
      "version": 6989103,
      "root_type": 1,
      "hash": "7f5a6feb54f0817030ba40c06c05b39067a2139f650dcedbc6faafb5f3803da7"
    },
    "genesis_height": 3027601,
    "genesis_hash": "67045351274dc3d5be3703cfd4ec6e1b58441d19dd96f5df0cea502c4f6d95f4",
    "last_retained_height": 3027601,
    "last_retained_hash": "67045351274dc3d5be3703cfd4ec6e1b58441d19dd96f5df0cea502c4f6d95f4",
    "chain_context": "53852332637bacb61b91b6411ab4095168ba02a50be4c3f82448438826f23898",
    "is_validator": false
  },
  "runtimes": {
    "000000000000000000000000000000000000000000000000e2eaa99fc008f87f": {
      "descriptor": {
        "v": 2,
        "id": "000000000000000000000000000000000000000000000000e2eaa99fc008f87f",
        "entity_id": "TAv9qXjV4yBphnKLJcNkzois1TLoYUjaRPrMfY58Apo=",
        "genesis": {
          "state_root": "c672b8d1ef56ed28ab87c3622c5114069bdd3ad7b8f9737498d0c01ecef0967a",
          "state": null,
          "storage_receipts": null,
          "round": 0
        },
        "kind": 1,
        "tee_hardware": 0,
        "versions": {
          "version": {
            "major": 6
          }
        },
        "executor": {
          "group_size": 10,
          "group_backup_size": 30,
          "allowed_stragglers": 1,
          "round_timeout": 5,
          "max_messages": 256
        },
        "txn_scheduler": {
          "algorithm": "simple",
          "batch_flush_timeout": 1000000000,
          "max_batch_size": 1000,
          "max_batch_size_bytes": 1048576,
          "propose_batch_timeout": 2
        },
        "storage": {
          "group_size": 10,
          "min_write_replication": 3,
          "max_apply_write_log_entries": 10000,
          "max_apply_ops": 2,
          "checkpoint_interval": 1000,
          "checkpoint_num_kept": 2,
          "checkpoint_chunk_size": 8388608
        },
        "admission_policy": {
          "any_node": {}
        },
        "constraints": {
          "executor": {
            "backup-worker": {
              "validator_set": {},
              "max_nodes": {
                "limit": 1
              },
              "min_pool_size": {
                "limit": 30
              }
            },
            "worker": {
              "validator_set": {},
              "max_nodes": {
                "limit": 1
              },
              "min_pool_size": {
                "limit": 30
              }
            }
          },
          "storage": {
            "worker": {
              "validator_set": {},
              "max_nodes": {
                "limit": 1
              },
              "min_pool_size": {
                "limit": 10
              }
            }
          }
        },
        "staking": {},
        "governance_model": "entity"
      },
      "latest_round": 240437,
      "latest_hash": "12ccff658600cf0cd8ef7e62050e829bcd6502bff9d99af91a677a6adf01be9b",
      "latest_time": "2022-01-27T11:50:00+01:00",
      "latest_state_root": {
        "ns": "000000000000000000000000000000000000000000000000e2eaa99fc008f87f",
        "version": 240437,
        "root_type": 1,
        "hash": "b77309ccc459847fe6b3869f276b9080807444b2cfabc2417dfaa463c80d7c73"
      },
      "genesis_round": 0,
      "genesis_hash": "34a5ce9b2ff76fa7b062ceabb5f64d451761c72b8443445b0c9df693d067b5c1",
      "last_retained_round": 0,
      "last_retained_hash": "34a5ce9b2ff76fa7b062ceabb5f64d451761c72b8443445b0c9df693d067b5c1",
      "committee": {
        "latest_round": 240437,
        "latest_height": 6989099,
        "last_committee_update_height": 6988791,
        "executor_roles": null,
        "storage_roles": null,
        "is_txn_scheduler": false,
        "peers": [
          "/ip4/136.243.103.147/tcp/40002/p2p/12D3KooWBizfYwusJJbccD66ZfavAs5Nn2wjb8JF89gC1U9LJD64",
          "/ip4/88.99.215.94/tcp/35002/p2p/12D3KooWJzgmDUt5v5XUSdbyb2AHnVfjQnCkKfevzuhmoK38CFNQ",
          "/ip4/135.181.130.59/tcp/30002/p2p/12D3KooWDzq86WGEqEHcVCYfhukxuYKPq7cjrCxwsYg5euBpTcfi",
          "/ip4/65.21.78.105/tcp/30002/p2p/12D3KooWHGmGJv8gbk6BjN3jHKNb6HQqgiASsrX59LLBPxtz4cCx",
          "/ip4/95.216.23.223/tcp/40002/p2p/12D3KooWSzJQ6h7zQ8yphLaLj5fT9g6Pf32cDpNdNbd6iNhzCKcq",
          "/ip4/52.252.232.158/tcp/30002/p2p/12D3KooWD7hWofy2fYds7utQrb88yM8tQG2nSzgbXL9B2KyMGvsQ",
          "/ip4/157.90.178.24/tcp/43002/p2p/12D3KooWCM5fJRX7SZP2LzBqs9wQ1xdkf79u4HZjYVzPVUKYFhbq",
          "/ip4/141.94.130.89/tcp/30002/p2p/12D3KooWJeJEggbXJgmvLa49tuX4VcXA4dgjmzg3RNEx3eqn99uU",
          "/ip4/95.217.119.229/tcp/30002/p2p/12D3KooWEvfS5EdnpcC8etv8eiE3oSkhWid2FEXb5oAisZT3AJyo",
          "/ip4/35.245.249.189/tcp/30002/p2p/12D3KooW9whHFUP4G5apBgKNDMc1Uc4yb3E3opvPgqVrPC9T6Kvu",
          "/ip4/172.31.18.97/tcp/30002/p2p/12D3KooWAkKBfHtD5wTXTSMA4oJHi8mFVXfBydmFBQ9UzJas6M7X",
          "/ip4/95.217.85.213/tcp/40002/p2p/12D3KooWCs2xp2XmenpNDDCuM5a5VksxFFHNgqQfNjeWWnDu5Xbb",
          "/ip4/94.130.164.172/tcp/50002/p2p/12D3KooWBXCepnySWH6yM3qwNTNVqRBvGFcsCnsbkhBEaBHw6o6G",
          "/ip4/135.181.139.153/tcp/40002/p2p/12D3KooWC9xtWD9eMLCgkHpoFAYTyrkrUHZzbbkDkbVTNXMNgVVE",
          "/ip4/95.216.219.29/tcp/30002/p2p/12D3KooWSLBMzgYQViZbRZ5siHgRL6QQ2QaT1x46XSL89W4SH4nG",
          "/ip4/65.108.43.199/tcp/35002/p2p/12D3KooWML9Lyua1yqZa34FJpcydEYUzWNMHLrRj1NsnPbG3dvqH",
          "/ip4/51.222.154.19/tcp/30002/p2p/12D3KooWQShu4eMSPDiaWmfVTcpqfKWumBrbcovEo6AznhcUfheN",
          "/ip4/144.137.17.159/tcp/30002/p2p/12D3KooWCixcEb3YyZqQnANYX9YLVuAxQ2tNaRapQuYh9TeTTFZV",
          "/ip4/141.95.35.31/tcp/30002/p2p/12D3KooWCyjYceAstcSNeUXP5dBcLfVaEjb1KzWZKQ343jaVnVQA",
          "/ip4/185.107.80.40/tcp/30002/p2p/12D3KooWCi6Cgpn1kfgPGB5hfsv1mfRiqTRQFuFCCR7XWgwhYGcH",
          "/ip4/51.81.107.146/tcp/30002/p2p/12D3KooWCSzSjZRbWYG79suVVeyvBUj1FA5L4hJ7HPt2aSDnCQH3",
          "/ip4/138.68.55.162/tcp/30002/p2p/12D3KooWMi7GwhRh3diSNpopzdtpWvW81dnwUv2xWVgPxsb9fbEM",
          "/ip4/23.88.105.64/tcp/30002/p2p/12D3KooWJzEi7z5bu4EroUBezW3BtygoiSYdWGrr8DqrBdoKA9mP",
          "/ip4/95.217.118.121/tcp/10002/p2p/12D3KooWF15NEgUFnh25SzhY2HJ6EdsywUFy4KLCUSN9X4eXhobL",
          "/ip4/95.216.25.173/tcp/40002/p2p/12D3KooWT2HEn17eHdkhVq7igY2uoLA2k7vT6ccosfKNM4fcHdQF",
          "/ip4/65.108.40.48/tcp/30002/p2p/12D3KooWS74Wqr1ZYLWafymznv4ib48DzDdmgZzVqMkM2ntNg5DZ",
          "/ip4/176.9.16.60/tcp/40002/p2p/12D3KooWM7m5RfjPGcGbPAx6uy21XFYzkn1SYA2vXisRK8nC3hhw",
          "/ip4/135.181.179.38/tcp/10002/p2p/12D3KooWNYpvGHuWr2V1zdLTbaYQXBLysh6gVqSwsmGJgRUqDEtD",
          "/ip4/95.217.74.107/tcp/30002/p2p/12D3KooWQowfSY9AVrt6DZE6TPwYVRRjWEp15Kd4Ns7mSQ3Go795",
          "/ip4/54.203.41.63/tcp/30002/p2p/12D3KooWDjvLwGspDSANLvwZGuoPPhMuUqfDKBtnqWqPwZDZZMve",
          "/ip4/88.99.100.32/tcp/30002/p2p/12D3KooWHcwXSe1cxnGjcyF92A1woBCujFxuUbT5TfcrWMtvAh2k",
          "/ip4/51.195.88.99/tcp/40002/p2p/12D3KooWN7ZKkLKD2MfneFzQUQv7q8q2LFcfdE17szMGFkyE5Y1L",
          "/ip4/135.181.215.247/tcp/35002/p2p/12D3KooWG3FhE4wPgnsWiFMJAymeAGZAwdUjG6uL2mS8nYtzDKWU",
          "/ip4/135.181.1.160/tcp/10002/p2p/12D3KooWK1mkdF77pp3CxK99uR1LfZuUngkgpZ6nEKPHMuk9V1FS",
          "/ip4/80.64.208.66/tcp/30002/p2p/12D3KooWHpcFG5dMyXCdg7VQWZty5dH2wigKPgrbeNJCptTZe7FH",
          "/ip4/173.230.140.135/tcp/30002/p2p/12D3KooWNPuJcZJwiNn1PHgRDN1hjJGpYwA9q8zPN36Ua5DqUdpB",
          "/ip4/94.130.34.15/tcp/45002/p2p/12D3KooWF76yZj3NpLw24qLFABvs3n7eaM2aFyAoT7a4isGLV1mq",
          "/ip4/95.217.62.175/tcp/30002/p2p/12D3KooWBzxL7TcysL4UNVaDQJ2Xu33XUZfqEjr6zSVuG1h24VnL",
          "/ip4/13.212.213.171/tcp/30002/p2p/12D3KooWRyHLK4eD2atnYZAnNxCEbTCpnQ44xLet14XvFfee1LaT",
          "/ip4/95.217.62.146/tcp/30002/p2p/12D3KooWDzch2bdRmp1tQM13r9cQz3VPUo1ZMjFeLiynK7GQpFw9",
          "/ip4/95.216.241.124/tcp/30002/p2p/12D3KooWSJ4WnjUY76cfiifujynaxD15QnCdQfzxuJsyqzat24rp",
          "/ip4/95.217.119.85/tcp/30002/p2p/12D3KooWQaNYjegtZyFtyrRNCF1yxB3sgGtBgY8asgEpFEcXPJiS",
          "/ip4/95.217.144.50/tcp/30002/p2p/12D3KooWKeo5Rhx4vr9A9Khaz7qpwyq1X3quPbbF3teBvmFUZwPt",
          "/ip4/80.64.208.65/tcp/30002/p2p/12D3KooWG1P4iJX2mqPUJF3bREgVjb2t9ezMULm1YmKkjTWpEUhH",
          "/ip4/91.90.179.51/tcp/30002/p2p/12D3KooWQZXoU1GFSSWsmqWALRz2nvukxENCPBKJGndTca3gfW8o",
          "/ip4/95.217.62.249/tcp/40002/p2p/12D3KooWEnHPNLM1Cfw9jK8ktYA1KVsaV5BKXLsU6bci8btzfg95",
          "/ip4/95.216.37.80/tcp/40002/p2p/12D3KooWPW3yJRiL54Wnz9XirNsDdT66dAJvitKd4EpTZTdTYLh4",
          "/ip4/95.217.43.207/tcp/30002/p2p/12D3KooWCMXSZAARpfhCS87VzcLKfZQ77bmpZzmPyBVBofcu94FL",
          "/ip4/103.195.100.122/tcp/30002/p2p/12D3KooWAUKaFcPSBvXWPyq8fYjT6SDYW2LfcWc7mgKx9aWJaC73",
          "/ip4/178.170.39.152/tcp/30002/p2p/12D3KooWEufLfHyEZbfCFZxuAcSy4Z4Cp12CFU4TWQaf8y7bq4uE",
          "/ip4/65.21.120.160/tcp/30002/p2p/12D3KooWN8GWEiCcMrLbXFRRinKErtL8yz9xBYEH3zd1o55MspSZ",
          "/ip4/131.153.99.14/tcp/30002/p2p/12D3KooWKPMS72iRgPDWczMVzKyYa3DSZmc771XDNXnMWVPnjfmJ"
        ]
      },
      "storage": {
        "last_finalized_round": 240437
      }
    }
  },
  "registration": {
    "last_registration": "0001-01-01T00:00:00Z"
  },
  "pending_upgrades": []
}
bugbuzz commented 2 years ago

btw, I have synced oasis-node without any paratime first and extended it with emerald later

kostko commented 2 years ago

The status looks ok, can you share your full node config as well and some parts from both your node and gateway logs around the time that you do the query that hangs?

bugbuzz commented 2 years ago

my oasis node config is

datadir: /home/ubuntu/oasis/data

log:
  level:
    default: info
    tendermint: info
    tendermint/context: error
  format: JSON

genesis:
  file: /home/ubuntu/oasis/etc/genesis.json

consensus:
  tendermint:
    p2p:
      # List of seed nodes to connect to.
      # NOTE: You can add additional seed nodes to this list if you want.
      seed:
        - "E27F6B7A350B4CC2B48A6CBE94B0A02B0DCB0BF3@35.199.49.168:26656"

runtime:
  supported:
    # List of ParaTimes that the node should support.
    - "000000000000000000000000000000000000000000000000e2eaa99fc008f87f"

  paths:
    # Paths to ParaTime non-SGX binaries for all of the supported ParaTimes.
    "000000000000000000000000000000000000000000000000e2eaa99fc008f87f": /home/ubuntu/oasis/runtimes/emerald-paratime

config:
  "000000000000000000000000000000000000000000000000e2eaa99fc008f87f":
    allow_expensive_queries: true

worker:
  storage:
    enabled: true
    checkpoint_sync:
      disabled: true

  p2p:
    # External P2P configuration.
    enabled: true

gateway config is

runtime_id: 000000000000000000000000000000000000000000000000e2eaa99fc008f87f
node_address: "unix:/home/ubuntu/oasis/data/internal.sock"
enable_pruning: false
pruning_step: 100000

log:
  level: debug
  format: json

database:
  host: localhost
  port: 5432
  db: oasis
  user: oasis
  password: oasis
  dial_timeout: 5
  read_timeout: 10
  write_timeout: 5
  max_open_conns: 0

gateway:
  chain_id: 42262
  http:
    host: 0.0.0.0
    port: 22545
    origins: "*"
    cors: ["moz-extension://*", "chrome-extension://*"]
  ws:
    host: 0.0.0.0
    port: 22546
    origins: "*"
    cors: ["moz-extension://*", "chrome-extension://*"]
  method_limits:
    get_logs_max_rounds: 200
bugbuzz commented 2 years ago

Logs of the gateway. I've put some personal console logs and used 0xd371EfD1ba557ED3d3026aFd8037B1cBcf7526cE as an address to query tx count for:

Jan 27 13:27:42 emerald-web3-gateway[508653]: {"caller":"utils.go:327","height":241594,"level":"debug","module":"indexer","msg":"sent chain event to event system","ts":"2022-01-27T13:27:42.641744032Z"}
Jan 27 13:27:42 emerald-web3-gateway[508653]: {"caller":"backend.go:109","level":"info","module":"indexer","msg":"indexed block","round":241594,"ts":"2022-01-27T13:27:42.641778687Z"}
Jan 27 13:27:45 emerald-web3-gateway[508653]: {"caller":"indexer.go:123","latest_indexed":241594,"latest_round":241594,"level":"debug","module":"gateway/indexer","msg":"checking health","runtime_id":"000000000000000000000000000000000000000000000000e2eaa99fc008f87f","ts":"2022-01-27T13:27:45.145719487Z"}
Jan 27 13:27:54 emerald-web3-gateway[508653]: {"block_number":"latest","caller":"api.go:138","full_tx":false,"level":"debug","method":"eth_getBlockByNumber","module":"eth_rpc","msg":"request","ts":"2022-01-27T13:27:54.428050801Z"}
Jan 27 13:27:54 emerald-web3-gateway[508653]: {"address":"0xd371efd1ba557ed3d3026afd8037b1cbcf7526ce","block_number":"latest","caller":"api.go:253","level":"debug","method":"eth_getBlockTransactionCount","module":"eth_rpc","msg":"request","ts":"2022-01-27T13:27:54.501546372Z"}
Jan 27 13:27:54 emerald-web3-gateway[508653]: {"18446744073709551615":"(MISSING)","address":"0xd371efd1ba557ed3d3026afd8037b1cbcf7526ce","block_number":"latest","caller":"api.go:262","level":"debug","method":"eth_getBlockTransactionCount","module":"eth_rpc","msg":"before accounts.Nonce, round is","ts":"2022-01-27T13:27:54.501594472Z"}
Jan 27 13:27:54 emerald-web3-gateway[508653]: {"address":"0xd371efd1ba557ed3d3026afd8037b1cbcf7526ce","block_number":"latest","caller":"api.go:263","level":"debug","method":"eth_getBlockTransactionCount","module":"eth_rpc","msg":"before accounts.Nonce, accountsAddr is","oasis1qr6hkwskznrj7gg7aflt4gx8gv5nwpw64vw9mj3z":"(MISSING)","ts":"2022-01-27T13:27:54.501623868Z"}
Jan 27 13:27:54 emerald-web3-gateway[508653]: >>>>>> nonce query
Jan 27 13:27:55 emerald-web3-gateway[508653]: {"caller":"indexer.go:123","latest_indexed":241594,"latest_round":241594,"level":"debug","module":"gateway/indexer","msg":"checking health","runtime_id":"000000000000000000000000000000000000000000000000e2eaa99fc008f87f","ts":"2022-01-27T13:27:55.147090519Z"}

^^^ here I'm expecting <<<<< nonce query console log which i've put right after this line, but it seems that the execution doesn't reach this line

oasis-node logs:

Jan 27 13:27:53 oasis-node[433290]: {"caller":"state.go:1833","level":"info","module":"tendermint:consensus","msg":"received proposal","proposal":"Proposal{6990668/0 (A4EA4F13E1A574A1415605D7204A09A3140154EDB8C9F2BE2E903A27CF4ACD3D:1:BF310F85D4FE, -1) 651FE22A0BC8 @ 2022-01-27T13:27:53.113780489Z}","ts":"2022-01-27T13:27:53.4169693Z"}
Jan 27 13:27:53 oasis-node[433290]: {"caller":"state.go:1892","hash":"A4EA4F13E1A574A1415605D7204A09A3140154EDB8C9F2BE2E903A27CF4ACD3D","height":6990668,"level":"info","module":"tendermint:consensus","msg":"received complete proposal block","ts":"2022-01-27T13:27:53.476318098Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"state.go:1587","hash":"A4EA4F13E1A574A1415605D7204A09A3140154EDB8C9F2BE2E903A27CF4ACD3D","height":6990668,"level":"info","module":"tendermint:consensus","msg":"finalizing commit of block","num_txs":0,"root":"76CB7167E8D1EF111DDA7E774306601D3C6E494961A259352B5F0FD2ABE4BE30","ts":"2022-01-27T13:27:54.038444103Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"execution.go:333","height":6990668,"level":"info","module":"tendermint:state","msg":"executed block","num_invalid_txs":0,"num_valid_txs":0,"ts":"2022-01-27T13:27:54.059730285Z"}
Jan 27 13:27:54 oasis-node[433290]: {"app_hash":"59EEC4B6FC3B6FB3E4D947DC6A3207AF77253F2D2DF5657D9BE562749D9F4CC2","caller":"execution.go:235","height":6990668,"level":"info","module":"tendermint:state","msg":"committed state","num_txs":0,"ts":"2022-01-27T13:27:54.059948205Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"indexer_service.go:82","height":6990668,"level":"info","module":"tendermint:txindex","msg":"indexed block","ts":"2022-01-27T13:27:54.060207632Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[core] Subchannel Connectivity change to IDLE\n","ts":"2022-01-27T13:27:54.385449423Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[balancer] base.baseBalancer: handle SubConn state change: 0xc040f1b9d0, IDLE\n","ts":"2022-01-27T13:27:54.385501652Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[core] Subchannel Connectivity change to CONNECTING\n","ts":"2022-01-27T13:27:54.385527751Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[core] Subchannel picks a new address \"88.99.242.44:30001\" to connect\n","ts":"2022-01-27T13:27:54.385566503Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[balancer] base.baseBalancer: handle SubConn state change: 0xc040f1b9d0, CONNECTING\n","ts":"2022-01-27T13:27:54.385648828Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"component.go:41","level":"warn","module":"grpc","msg":"[core] grpc: addrConn.createTransport failed to connect to {88.99.242.44:30001  <nil> 0 <nil>}. Err: connection error: desc = \"transport: Error while dialing dial tcp 88.99.242.44:30001: connect: connection refused\"\n","ts":"2022-01-27T13:27:54.38592665Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[core] Subchannel Connectivity change to TRANSIENT_FAILURE\n","ts":"2022-01-27T13:27:54.385965884Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[balancer] base.baseBalancer: handle SubConn state change: 0xc040f1b9d0, TRANSIENT_FAILURE\n","ts":"2022-01-27T13:27:54.385986333Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[core] Subchannel Connectivity change to IDLE\n","ts":"2022-01-27T13:27:54.622714473Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[balancer] base.baseBalancer: handle SubConn state change: 0xc040f0a6b0, IDLE\n","ts":"2022-01-27T13:27:54.622745972Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[core] Subchannel Connectivity change to CONNECTING\n","ts":"2022-01-27T13:27:54.622770318Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[core] Subchannel picks a new address \"141.95.145.168:30001\" to connect\n","ts":"2022-01-27T13:27:54.622801206Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[balancer] base.baseBalancer: handle SubConn state change: 0xc040f0a6b0, CONNECTING\n","ts":"2022-01-27T13:27:54.622865367Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"component.go:41","level":"warn","module":"grpc","msg":"[core] grpc: addrConn.createTransport failed to connect to {141.95.145.168:30001  <nil> 0 <nil>}. Err: connection error: desc = \"transport: Error while dialing dial tcp 141.95.145.168:30001: connect: connection refused\"\n","ts":"2022-01-27T13:27:54.630633969Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[core] Subchannel Connectivity change to TRANSIENT_FAILURE\n","ts":"2022-01-27T13:27:54.630667402Z"}
Jan 27 13:27:54 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[balancer] base.baseBalancer: handle SubConn state change: 0xc040f0a6b0, TRANSIENT_FAILURE\n","ts":"2022-01-27T13:27:54.630686288Z"}
Jan 27 13:27:56 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[core] Subchannel Connectivity change to IDLE\n","ts":"2022-01-27T13:27:56.222194094Z"}
Jan 27 13:27:56 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[balancer] base.baseBalancer: handle SubConn state change: 0xc0420a2f20, IDLE\n","ts":"2022-01-27T13:27:56.222249468Z"}
Jan 27 13:27:56 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[core] Subchannel Connectivity change to CONNECTING\n","ts":"2022-01-27T13:27:56.222279634Z"}
Jan 27 13:27:56 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[core] Subchannel picks a new address \"94.130.136.136:32001\" to connect\n","ts":"2022-01-27T13:27:56.222313077Z"}
Jan 27 13:27:56 oasis-node[433290]: {"caller":"component.go:36","level":"info","module":"grpc","msg":"[balancer] base.baseBalancer: handle SubConn state change: 0xc0420a2f20, CONNECTING\n","ts":"2022-01-27T13:27:56.222380805Z"}