search

kwilteam / kwil-db

Kwil DB, the database for web3
https://www.kwil.com/
Other
36 stars 12 forks source link

Added migrate network-status, proposal-status commands, new status fe… #1040

Closed charithabandi closed 1 month ago

charithabandi commented 1 month ago

This PR includes:

charithabandi commented 1 month ago

Tested out the changes, can be reviewed

brennanjl commented 1 month ago

Tested out the changes, can be reviewed

Will take a look momentarily

jchappelow commented 1 month ago 
$  kwil-admin migrate list 
Migrations:
5eaf09ab-906f-5a0d-9e3f-b2d9efe3222d:
    activationPeriod: 50
    migrationDuration: 1200
    timestamp: 2024-10-01 01:11:27.433294926 +0000 UTC m=+0.075445430

$  kwil-admin migrate approve 5eaf09ab-906f-5a0d-9e3f-b2d9efe3222d 
TxHash: 96c6e38ed30246175309df10be0f1375f2401d715b439558469d4b9dbfeaa35c

$  kwil-admin migrate list 
No migrations found.

$  kwil-admin migrate proposal-status 
Error: accepts 1 arg(s), received 0
accepts 1 arg(s), received 0

 $  kwil-admin migrate network-status 
Migration Status: ActivationPeriod
Start Height: 163
End Height: 1363
Current Block: 119
Available Commands:
  approve         Approve a migration proposal.
  genesis-state   Download the genesis state corresponding to the ongoing migration.
  list            List all the pending migration proposals.
  network-status  Get the migration status of the network.
  proposal-status Get the status of the pending migration proposal.
  propose         Submit a migration proposal.

So, in my mind, hypothetically, list or status should list active migrations (and their status), while list-proposals or just proposals should list migration proposals.

jchappelow commented 1 month ago

Just for completeness as I go through this:

{
  "genesis_time": "0001-01-01T00:00:00Z",
  "chain_id": "",
  "initial_height": 163,
  "app_hash": "/Edi0bWPmCyqC6AtNOrGMik1iS2C7lr3EeyiGQSdBZE=",
  "activations": null,
  "consensus_params": {
    "block": {
      "max_bytes": 0,
      "max_gas": 0,
      "abci_max_bytes": false
    },
    "evidence": {
      "max_age_num_blocks": 0,
      "max_age_duration": 0,
      "max_bytes": 0
    },
    "version": {
      "app": 0
    },
    "validator": {
      "pub_key_types": null,
      "join_expiry": 0
    },
    "votes": {
      "vote_expiry": 0,
      "max_votes_per_tx": 0
    },
    "abci": {
      "vote_extensions_enable_height": 0
    },
    "migration": {
      "start_height": 163,
      "end_height": 1363
    },
    "without_gas_costs": false
  },
  "validators": [
    {
      "pub_key": "997ccd99f95ae68327609938b62201caad61227bb3c5e04e0bedf529d5e306c9",
      "power": 1,
      "name": "validator-0"
    },
    {
      "pub_key": "46076447d6767f76fff57229c9a10f835d22c6367c7b4d78b96ba4f7e7169cbe",
      "power": 1,
      "name": "validator-1"
    }
  ]
}
jchappelow commented 1 month ago

On new network node, I've edited as follows in hopes that it will get the genesis state (and its apphash), and use my validators, initial height, and chain ID:

{
  "genesis_time": "2024-09-30T12:46:48.412531807Z",
  "chain_id": "kwil-chain-19",
  "initial_height": 1364,
  "app_hash": null,
  "activations": null,
  "consensus_params": {
    "block": {
      "max_bytes": 6291456,
      "max_gas": -1,
      "abci_max_bytes": false
    },
    "evidence": {
      "max_age_num_blocks": 100000,
      "max_age_duration": 172800000000000,
      "max_bytes": 1048576
    },
    "version": {
      "app": 0
    },
    "validator": {
      "pub_key_types": [
        "ed25519"
      ],
      "join_expiry": 14400
    },
    "votes": {
      "vote_expiry": 14400,
      "max_votes_per_tx": 100
    },
    "abci": {
      "vote_extensions_enable_height": 0
    },
    "migration": {
      "start_height": 163,
      "end_height": 1363
    },
    "without_gas_costs": false
  },
  "validators": [
    {
      "pub_key": "997ccd99f95ae68327609938b62201caad61227bb3c5e04e0bedf529d5e306c9",
      "power": 1,
      "name": "validator-0"
    }
  ]
}
jchappelow commented 1 month ago

On the new network node, I've set:

# Path to the snapshot file to restore the database from.
# Used during the network migration process.
# Might be either absolute path or path related to the kwild root directory.
# genesis_state = "/data/kwild/migrate-old-chain/snapshot.sql.gz"
genesis_state = ""

# The listening address of the node to migrate the app state from.
# mandatory if the start_height and end_height are provided in the genesis file.
migrate_from = "http://23.20.126.14:8484"

My understanding is that the above is how to get it polling, given the following in genesis.json:

    "migration": {
      "start_height": 163,
      "end_height": 1363
    },

What I got on startup was:

2024-10-01T01:27:32.98Z warn    kwild.listener-manager.migrations       no migration config provided, skipping migration listener
2024-10-01T01:27:32.98Z debug   kwild.listener-manager  Event listener stopped (cleanly)        {"listener": "migrations"}

What went wrong? Well, I forgot there was a new [migration] section and I still had app.migrate_from (and no migrate.enable of course). It just started making blocks all on it's own without actually migrating. Trying again...

jchappelow commented 1 month ago

discrepancy:

migration.migrate_from in config.toml

type MigrationConfig struct {
        Enable bool `mapstructure:"enable"`
        // MigrateFrom is the JSON-RPC listening address of the node to replicate the state from.
        MigrateFrom string `mapstructure:"migrate_from"`
}

migration.from in flags:

      --migration.enable                                 Enable migration
      --migration.from string                            kwild JSON-RPC listening address of the node to replicate the state from.
jchappelow commented 1 month ago

Fixed config.toml and got this:

2024-10-01T01:37:21.398Z        info    kwild   Entering migration mode {"migrate_from": "http://23.20.126.14:8484"}
Error: failed to prepare for migration: migration genesis config should not have initial height set

I thought I could/should set initial height, but nbd, set it back to 1:

2024-10-01T01:39:52.344Z        info    kwild   version 0.9.0-pre+d9bb85d1a commit d9bb85d1a
2024-10-01T01:39:52.344Z        info    kwild   Entering migration mode {"migrate_from": "http://23.20.126.14:8484"}
Error: failed to prepare for migration: migration genesis config should not have migration info set

OK, it now gets the whole migration section from the provider if (cases?), so removed it and tried again:

2024-10-01T01:41:05.991Z        info    kwild   version 0.9.0-pre+d9bb85d1a commit d9bb85d1a
2024-10-01T01:41:05.991Z        info    kwild   Entering migration mode {"migrate_from": "http://23.20.126.14:8484"}
2024-10-01T01:41:05.993Z        info    kwild   Requesting genesis state from the old chain     {"listen_address": "http://23.20.126.14:8484"}
2024-10-01T01:41:05.994Z        info    kwild   Genesis state available for download
2024-10-01T01:41:06.126Z        warn    kwild   Validators already set in the genesis config. Ignoring the validators from the old chain
2024-10-01T01:41:06.126Z        info    kwild   Genesis state not available     {"error": "failed to save genesis config: open /data/kwild/genesis.json: permission denied", "retry after(sec)": 30}

It took my customized validators. :heavy_check_mark:

Couldn't write to my genesis.json.

-rw-r--r-- 1 root  root       896 Oct  1 01:41 genesis.json

but kwild was running as the user kwild.

Changed owner of genesis.json to kwild and try again:

2024-10-01T01:42:57.879Z        info    kwild   version 0.9.0-pre+d9bb85d1a commit d9bb85d1a
2024-10-01T01:42:57.879Z        info    kwild   Entering migration mode {"migrate_from": "http://23.20.126.14:8484"}
2024-10-01T01:42:57.88Z info    kwild   Genesis state already downloaded        {"genesis snapshot": "/data/kwild/genesis-state.sql.gz"}
Error: failed to prepare for migration: app hash does not match the genesis state

My genesis.json has: "app_hash": null,.

Decided to delete /data/kwild/genesis-state.sql.gz and try again.

Got this:

2024-10-01T01:45:47.954Z        info    kwild   Genesis state available for download
2024-10-01T01:45:48.027Z        warn    kwild   Validators already set in the genesis config. Ignoring the validators from the old chain
2024-10-01T01:45:48.028Z        info    kwild   Genesis state not available     {"error": "failed to save kwild config: open /data/kwild/config.toml: permission denied", "retry after(sec)": 30

That really surprised me. Why would it modify my config file? That seems incorrect.

Still, I them made it writable by kwild user, reset the apphash and initialheight in genesis.json, and tried again:

2024-10-01T01:49:03.358Z        info    kwild   version 0.9.0-pre+d9bb85d1a commit d9bb85d1a
2024-10-01T01:49:03.358Z        info    kwild   Entering migration mode {"migrate_from": "http://23.20.126.14:8484"}
Error: failed to prepare for migration: migration genesis config should not have migration info set

Ok, I guess I was not supposed to have the "migrations" section in genesis.json any more. Remove and try again:

2024-10-01T01:50:14.005Z        info    kwild   version 0.9.0-pre+d9bb85d1a commit d9bb85d1a                                                                                                                                         
2024-10-01T01:50:14.005Z        info    kwild   Entering migration mode {"migrate_from": "http://23.20.126.14:8484"}                                                                                                                 
2024-10-01T01:50:14.019Z        info    kwild   Requesting genesis state from the old chain     {"listen_address": "http://23.20.126.14:8484"}                                                                                       
2024-10-01T01:50:14.021Z        info    kwild   Genesis state available for download                                                                                                                                                 
2024-10-01T01:50:14.114Z        warn    kwild   Validators already set in the genesis config. Ignoring the validators from the old chain                                                                                             
2024-10-01T01:50:14.114Z        info    kwild   Genesis state downloaded successfully   {"genesis snapshot": "/data/kwild/genesis-state.sql.gz"}              
2024-10-01T01:50:14.114Z        info    kwild   Entering startup mode                                     
Started with 0 configured hard fork heights:                                                                                                                                                                                         
- halt: <nil> (disabled)                                                                                                                                                                                                             
2024-10-01T01:50:14.115Z        debug   kwild   loading TLS key pair for gRPC servers   {"key_file": "d.cfg.AdminTLSKeyFile", "cert_file": "d.cfg.AdminTLSCertFile"}                                                                 
2024-10-01T01:50:14.168Z        warn    kwild.pg        ERROR [42P01]: relation "kwild_voting.voters" does not exist                                                                                                                 
2024-10-01T01:50:14.168Z        info    kwild   Restore DB:     {"command": "/usr/bin/psql --username kwild --host 127.0.0.1 --port 5432 --dbname kwild --no-password"}                                                              
2024-10-01T01:50:16.411Z        info    kwild.pg        Connected to PostgreSQL 16.2 (Debian 16.2-1.pgdg120+2) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
...
2024-10-01T01:50:16.851Z        info    kwild.cometbft  service start   {"module": "txindex", "msg": "Starting IndexerService service", "impl": "IndexerService"}                                                                    
2024-10-01T01:50:16.852Z        info    kwild.abci      ABCI application is ready       {"height": 0}           
2024-10-01T01:50:16.852Z        info    kwild.cometbft  ABCI Handshake App Info {"module": "consensus", "height": 0, "hash": "", "software-version": "0.9.0-pre+d9bb85d1a", "protocol-version": 0}                                   
2024-10-01T01:50:16.852Z        info    kwild.cometbft  ABCI Replay Blocks      {"module": "consensus", "appHeight": 0, "storeHeight": 0, "stateHeight": 0}
2024-10-01T01:50:16.852Z        debug   kwild.abci              {"stage": "ABCI InitChain", "height": 163, "ChainId": "kwil-chain-19"}        
2024-10-01T01:50:16.853Z        info    kwild.abci      initialized chain       {"stage": "ABCI InitChain", "height": 163, "app hash": "94929c23533695af84f2ed6d24e79b37e7671416ad3761c4b811ba614f19988c"}                           
2024-10-01T01:50:16.858Z        info    kwild.cometbft  Completed ABCI Handshake - CometBFT and App are synced  {"module": "consensus", "appHeight": 0, "appHash": "94929C23533695AF84F2ED6D24E79B37E7671416AD3761C4B811BA614F19988C"
}                                                                                                                                                                                                                                    
2024-10-01T01:50:16.858Z        info    kwild.cometbft  Version info    {"tendermint_version": "0.38.12", "abci": "2.0.0", "block": 11, "p2p": 8, "commit_hash": ""}                                                                 
2024-10-01T01:50:16.858Z        info    kwild.cometbft  This node is a validator        {"module": "consensus", "addr": "643AF2A0462CEA175047C4B8A18817DD9429BB0B", "pubKey": "PubKeyEd25519{997CCD99F95AE68327609938B62201CAAD61227B
B3C5E04E0BEDF529D5E306C9}"}                                                                                                                                                                                                          
2024-10-01T01:50:16.875Z        info    kwild.cometbft  P2P Node ID     {"module": "p2p", "ID": "643af2a0462cea175047c4b8a18817dd9429bb0b", "file": "/data/kwild/abci"}
...
2024-10-01T01:50:17.377Z        info    kwild.listener-manager  Node is a validator and caught up with the network, starting listeners                                                                                               
2024-10-01T01:50:17.377Z        warn    kwild.listener-manager.eth_deposits     no eth_deposit configuration found, eth_deposit oracle will not start                                                                                
2024-10-01T01:50:17.377Z        debug   kwild.listener-manager  Event listener stopped (cleanly)        {"listener": "eth_deposits"}                                                                                                 
2024-10-01T01:50:17.38Z info    kwild.listener-manager.migrations       start syncing changesets from old chain {"startHeight": 163}                                                                                                 
2024-10-01T01:50:17.382Z        info    kwild.listener-manager.migrations       received changeset metadata     {"height": 163, "numChunks": 0, "chunkSizes": []}                                                                    
2024-10-01T01:50:17.382Z        debug   kwild.listener-manager.migrations       empty changesets for height     {"height": 163}                                                                                                      
2024-10-01T01:50:18.386Z        info    kwild.listener-manager.migrations       received changeset metadata     {"height": 164, "numChunks": 0, "chunkSizes": []}                                                                    
2024-10-01T01:50:18.386Z        debug   kwild.listener-manager.migrations       empty changesets for height     {"height": 164}                                                                                                      
2024-10-01T01:50:19.39Z info    kwild.listener-manager.migrations       received changeset metadata     {"height": 165, "numChunks": 0, "chunkSizes": []}                                                                            
2024-10-01T01:50:19.39Z debug   kwild.listener-manager.migrations       empty changesets for height     {"height": 165}
...
2024-10-01T01:50:20.396Z        info    kwild.listener-manager.migrations       received changeset metadata     {"height": 166, "numChunks": 0, "chunkSizes": []}
2024-10-01T01:50:20.396Z        debug   kwild.listener-manager.migrations       empty changesets for height     {"height": 166}                                       
2024-10-01T01:50:21.405Z        info    kwild.listener-manager.migrations       received changeset metadata     {"height": 167, "numChunks": 0, "chunkSizes": []}                                                                    
2024-10-01T01:50:21.405Z        debug   kwild.listener-manager.migrations       empty changesets for height     {"height": 167}                                                                                                      2024-10-01T01:50:22.409Z        info    kwild.listener-manager.migrations       received changeset metadata     {"height": 168, "numChunks": 0, "chunkSizes": []}                                                                    
2024-10-01T01:50:22.409Z        debug   kwild.listener-manager.migrations       empty changesets for height     {"height": 168}                                                                                                      
2024-10-01T01:50:22.876Z        debug   kwild.txapp     no events to propose    {"height": 163}                                                                                                                                      
2024-10-01T01:50:22.897Z        info    kwild.cometbft  finalizing commit of block      {"module": "consensus", "height": 163, "hash": "C2FD5EAF7DA224815363351747B1A6B1FFCD3AF572133A0E4DA1CCDA44AE7156", "root": "94929C23533695AF8
4F2ED6D24E79B37E7671416AD3761C4B811BA614F19988C", "num_txs": 0}                                                                                                                                                                      
2024-10-01T01:50:22.904Z        debug   kwild.event-broadcaster local node is current block proposer, skipping voteID broadcast                                                                                                      
2024-10-01T01:50:22.905Z        debug   kwild.abci      Finalize(start) {"height": 0, "appHash": ""}                                                                                                                                 
2024-10-01T01:50:22.916Z        info    kwild.cometbft  finalized block {"module": "state", "height": 163, "num_txs_res": 0, "num_val_updates": 0, "block_app_hash": "5DF6E0E2761359D30A8275058E299FCC0381534545F55CF43E41983F5D4C945
6"}

Migration started.

jchappelow commented 1 month ago

The end looks normal on the new chain:

2024-10-01T03:32:05.071Z        info    kwild.listener-manager.migrations       received changeset metadata     {"height": 1362, "numChunks": 0, "chunkSizes": []}
2024-10-01T03:32:05.071Z        debug   kwild.listener-manager.migrations       empty changesets for height     {"height": 1362}
2024-10-01T03:32:10.482Z        debug   kwild.txapp     no events to propose    {"height": 1177}
2024-10-01T03:32:10.509Z        info    kwild.cometbft  finalizing commit of block      {"module": "consensus", "height": 1177, "hash": "380B4339CBFE3CD77212583100C365033BD0803A7758A69E35876C4A7778E407", "root": "68246EBE5EAAC4B9CA178D14581A71027FDEA095898EC0990AB6B57EB46B19FA", "num_txs": 0}
2024-10-01T03:32:10.517Z        debug   kwild.event-broadcaster local node is current block proposer, skipping voteID broadcast
2024-10-01T03:32:10.517Z        debug   kwild.abci      Finalize(start) {"height": 1176, "appHash": "68246ebe5eaac4b9ca178d14581a71027fdea095898ec0990ab6b57eb46b19fa"}
2024-10-01T03:32:10.522Z        info    kwild.cometbft  finalized block {"module": "state", "height": 1177, "num_txs_res": 0, "num_val_updates": 0, "block_app_hash": "731D2E44527B652360AD886053F93E1D74784721E72190B4FA4844F556D18846"}
2024-10-01T03:32:11.925Z        info    kwild.listener-manager.migrations       received changeset metadata     {"height": 1363, "numChunks": 0, "chunkSizes": []}
2024-10-01T03:32:11.932Z        info    kwild.listener-manager.migrations       adding changeset migration event        {"height": 1363, "size": 0, "prevHeight": 0}
2024-10-01T03:32:12.939Z        info    kwild.listener-manager.migrations       changesets have been synchronized with the old chain    {"height": 1364}
2024-10-01T03:32:12.939Z        debug   kwild.listener-manager  Event listener stopped (cleanly)        {"listener": "migrations"}
2024-10-01T03:32:16.51Z info    kwild.txapp     Creating new ValidatorVoteBodies transaction    {"events": 1}
2024-10-01T03:32:16.511Z        info    kwild.abci      PrepareProposal: number of transactions in proposed block has changed!  {"stage": "ABCI PrepareProposal", "height": 1178, "txs": 0, "in": 0, "out": 1}
2024-10-01T03:32:16.537Z        info    kwild.cometbft  finalizing commit of block      {"module": "consensus", "height": 1178, "hash": "7134B35336BD0A5CE3568ACA3E5B3D386AE1E25F8FC8694A68AFC54C27F5B3F2", "root": "731D2E44527B652360AD886053F93E1D74784721E72190B4FA4844F556D18846", "num_txs": 1}
2024-10-01T03:32:16.545Z        debug   kwild.txapp     executing transaction   {"tx": {"signature":{"sig":"mSrmc6zM15aptLxjCvIaoWBO0JYUXfT8RSns8H2Lfw63Y9ypZFUq7XSl6uA/NyARsvKxqV/eUCfhgUgHRiWjCA==","type":"ed25519"},"body":{"desc":"","payload":"AAHh4N+TY2hhbmdlc2V0X21pZ3JhdGlvbooAAceCBVOAAYCA","type":"validator_vote_bodies","fee":"10000","nonce":488,"chain_id":"kwil-chain-19"},"serialization":"concat","sender":"997ccd99f95ae68327609938b62201caad61227bb3c5e04e0bedf529d5e306c9"}}
2024-10-01T03:32:16.549Z        debug   kwild.event-broadcaster local node is current block proposer, skipping voteID broadcast
2024-10-01T03:32:16.549Z        debug   kwild.abci      Finalize(start) {"height": 1177, "appHash": "731d2e44527b652360ad886053f93e1d74784721e72190b4fa4844f556d18846"}
2024-10-01T03:32:16.552Z        debug   kwild.txapp     resolving resolution    {"type": "changeset_migration", "id": "1ec3bcd6-93a1-5134-a0c0-4b5d00d5db2d"}
2024-10-01T03:32:16.552Z        debug   kwild.txapp.changeset_migration insert changeset chunk  {"height": 1363, "chunkIndex": 0}
2024-10-01T03:32:16.556Z        info    kwild.txapp.changesets  Applied changesets      {"height": 1363}
2024-10-01T03:32:16.557Z        info    kwild.txapp.changesets  Migration completed. The migration section in genesis.json and the migrate_from configuration in config.toml are no longer relevant and should be removed.      {"height": 1363}
2024-10-01T03:32:16.561Z        info    kwild.cometbft  finalized block {"module": "state", "height": 1178, "num_txs_res": 1, "num_val_updates": 0, "block_app_hash": "9EBB26F7A9A24A31EE319100243AC49CC7E8D0910C4922AFF3E45E21E7A8705F"}

There were no user transactions in the old chain or the new chain.