search

kurtosis-tech / near-package

A Kurtosis package that spins up a local near testnet
8 stars 2 forks source link

URLs reported by the NEAR module don't work #10

Closed mieubrisse closed 1 year ago

mieubrisse commented 1 year ago

Run:

 13:06:12 /Users/zerix/code/kurtosis-docs/docs/explanations (kevin/explain-docker-networking L)  kurtosis run github.com/kurtosis-tech/near-package
WARN[2022-12-05T13:06:19-03:00] You are running an old version of the Kurtosis CLI; we suggest you to update it to the latest version, '0.57.6'
WARN[2022-12-05T13:06:19-03:00] You can manually upgrade the CLI tool following these instructions: https://docs.kurtosistech.com/installation.html#upgrading-kurtosis-cli
INFO[2022-12-05T13:06:20-03:00] Creating a new enclave for Starlark to run inside...
INFO[2022-12-05T13:06:29-03:00] Enclave 'black-water' created successfully

> print "Starting the near-package with input struct(backend_ip_address = \"127.0.0.1\")"
Starting the near-package with input struct(backend_ip_address = "127.0.0.1")

> print "Launching contract helper postgresql"
Launching contract helper postgresql

> print "Adding contract helper Posgresql DB running on port '5432'"
Adding contract helper Posgresql DB running on port '5432'

> add_service service_id="contract-helper-db"
Service 'contract-helper-db' added with internal ID 'contract-helper-db-1670256390'

> exec service_id="contract-helper-db" command=["sleep", "10"]
Command returned with exit code '0' with no output

> exec service_id="contract-helper-db" command=["psql", "-U", "near", "-c", "\\l"]
Command returned with exit code '0' and the following output:
List of databases
   Name    | Owner | Encoding |  Collate   |   Ctype    | Access privileges
-----------+-------+----------+------------+------------+-------------------
 near      | near  | UTF8     | en_US.utf8 | en_US.utf8 |
 postgres  | near  | UTF8     | en_US.utf8 | en_US.utf8 |
 template0 | near  | UTF8     | en_US.utf8 | en_US.utf8 | =c/near          +
           |       |          |            |            | near=CTc/near
 template1 | near  | UTF8     | en_US.utf8 | en_US.utf8 | =c/near          +
           |       |          |            |            | near=CTc/near
(4 rows)

> exec service_id="contract-helper-db" command=["psql", "-U", "near", "-c", "create database indexer with owner=near"]
Command returned with exit code '0' and the following output: 'CREATE DATABASE'

> exec service_id="contract-helper-db" command=["psql", "-U", "near", "-c", "create database analytics with owner=near"]
Command returned with exit code '0' and the following output: 'CREATE DATABASE'

> exec service_id="contract-helper-db" command=["psql", "-U", "near", "-c", "create database telemetry with owner=near"]
Command returned with exit code '0' and the following output: 'CREATE DATABASE'

> print "Contract helper postgresql db info struct(analytics_db = \"analytics\", db_user_password = \"near\", db_username = \"near\", indexer_db = \"indexer\", private_url = struct(ip_address = \"contract-helper-db\", path = \"\", port_number = 5432, protocol = \"postgres\"), telemetry_db = \"telemetry\")"
Contract helper postgresql db info struct(analytics_db = "analytics", db_user_password = "near", db_username = "near", indexer_db = "indexer", private_url = struct(ip_address = "contract-helper-db", path = "", port_number = 5432, protocol = "postgres"), telemetry_db = "telemetry")

> print "Launching contract helper dynamo db"
Launching contract helper dynamo db

> print "Adding contract helper DynamoDB running on default port '8000'"
Adding contract helper DynamoDB running on default port '8000'

> add_service service_id="contract-helper-dynamo-db"
Service 'contract-helper-dynamo-db' added with internal ID 'contract-helper-dynamo-db-1670256406'

> print "Contract helper dynamodb info struct(private_url = struct(ip_address = \"contract-helper-dynamo-db\", path = \"\", port_number = 8000, protocol = \"TCP\"))"
Contract helper dynamodb info struct(private_url = struct(ip_address = "contract-helper-dynamo-db", path = "", port_number = 8000, protocol = "TCP"))

> print "Launching indexer"
Launching indexer

> print "Adding indexer service..."
Adding indexer service...

> upload_files src="github.com/kurtosis-tech/near-package/static_files/near-configs/localnet" artifact_id="bc9aad0c-5626-4975-8d93-0c952cc7bfb9"
Files uploaded with artifact ID 'bc9aad0c-5626-4975-8d93-0c952cc7bfb9'

> add_service service_id="indexer-node"
Service 'indexer-node' added with internal ID 'indexer-node-1670256410'

> exec service_id="indexer-node" command=["sleep", "10"]
Command returned with exit code '0' with no output

> exec service_id="indexer-node" command=["cat", "/root/.near/validator_key.json"]
Command returned with exit code '0' and the following output:
{
  "account_id": "test.near",
  "public_key": "ed25519:3Kuyi2DUXdoHgoaNEvCxa1m6G8xqc6Xs7WGajaqLhNmW",
  "secret_key": "ed25519:2ykcMLiM7vCmsSECcgfmUzihBtNdBv7v2CxNi94sNt4R8ar4xsrMMYvtsSNGQDfSRhNWXEnZvgx2wzS9ViBiS9jW"
}

> print "Indexer launched with struct(private_rpc_url = struct(ip_address = \"indexer-node\", path = \"\", port_number = 3030, protocol = \"http\"), public_rpc_url = struct(ip_address = \"127.0.0.1\", path = \"\", port_number = 8332, protocol = \"http\"), validator_key = {\"account_id\": \"test.near\", \"public_key\": \"ed25519:3Kuyi2DUXdoHgoaNEvCxa1m6G8xqc6Xs7WGajaqLhNmW\", \"secret_key\": \"ed25519:2ykcMLiM7vCmsSECcgfmUzihBtNdBv7v2CxNi94sNt4R8ar4xsrMMYvtsSNGQDfSRhNWXEnZvgx2wzS9ViBiS9jW\"})"
Indexer launched with struct(private_rpc_url = struct(ip_address = "indexer-node", path = "", port_number = 3030, protocol = "http"), public_rpc_url = struct(ip_address = "127.0.0.1", path = "", port_number = 8332, protocol = "http"), validator_key = {"account_id": "test.near", "public_key": "ed25519:3Kuyi2DUXdoHgoaNEvCxa1m6G8xqc6Xs7WGajaqLhNmW", "secret_key": "ed25519:2ykcMLiM7vCmsSECcgfmUzihBtNdBv7v2CxNi94sNt4R8ar4xsrMMYvtsSNGQDfSRhNWXEnZvgx2wzS9ViBiS9jW"})

> print "Launching contract helper"
Launching contract helper

> print "Adding contract helper service running on port '3000'"
Adding contract helper service running on port '3000'

> add_service service_id="contract-helper-service"
Service 'contract-helper-service' added with internal ID 'contract-helper-service-1670256426'

> print "Contract helper launchded with struct(private_url = struct(ip_address = \"contract-helper-service\", path = \"\", port_number = 3000, protocol = \"http\"), public_url = struct(ip_address = \"127.0.0.1\", path = \"\", port_number = 8330, protocol = \"http\"))"
Contract helper launchded with struct(private_url = struct(ip_address = "contract-helper-service", path = "", port_number = 3000, protocol = "http"), public_url = struct(ip_address = "127.0.0.1", path = "", port_number = 8330, protocol = "http"))

> print "Launching explorer backend"
Launching explorer backend

> print "Adding explorer backend service"
Adding explorer backend service

> add_service service_id="explorer-backend"
Service 'explorer-backend' added with internal ID 'explorer-backend-1670256430'

> print "Explorer backend launchded with struct(private_url = struct(ip_address = \"explorer-backend\", path = \"\", port_number = 8080, protocol = \"http\"), public_url = struct(ip_address = \"127.0.0.1\", path = \"\", port_number = 18080, protocol = \"http\"))"
Explorer backend launchded with struct(private_url = struct(ip_address = "explorer-backend", path = "", port_number = 8080, protocol = "http"), public_url = struct(ip_address = "127.0.0.1", path = "", port_number = 18080, protocol = "http"))

> print "Launching explorer frontend"
Launching explorer frontend

> print "Adding explorer frontend service running on port '3000'"
Adding explorer frontend service running on port '3000'

> add_service service_id="explorer-frontend"
Service 'explorer-frontend' added with internal ID 'explorer-frontend-1670256434'

> print "Explorer frontend launchded with struct(public_url = struct(ip_address = \"127.0.0.1\", path = \"\", port_number = 8331, protocol = \"http\"))"
Explorer frontend launchded with struct(public_url = struct(ip_address = "127.0.0.1", path = "", port_number = 8331, protocol = "http"))

> print "Launching wallet"
Launching wallet

> print "Adding wallet service running on port '3004"
Adding wallet service running on port '3004

> print "Replacing variable 'NODE_URL' to 'http://127.0.0.1:8332' using regexp: '([,{])NODE_URL:[^,]*([,}])'"
Replacing variable 'NODE_URL' to 'http://127.0.0.1:8332' using regexp: '([,{])NODE_URL:[^,]*([,}])'

> print "Replacing variable 'ACCOUNT_HELPER_URL' to 'http://127.0.0.1:8330' using regexp: '([,{])ACCOUNT_HELPER_URL:[^,]*([,}])'"
Replacing variable 'ACCOUNT_HELPER_URL' to 'http://127.0.0.1:8330' using regexp: '([,{])ACCOUNT_HELPER_URL:[^,]*([,}])'

> print "Replacing variable 'EXPLORER_URL' to 'http://127.0.0.1:8331' using regexp: '([,{])EXPLORER_URL:[^,]*([,}])'"
Replacing variable 'EXPLORER_URL' to 'http://127.0.0.1:8331' using regexp: '([,{])EXPLORER_URL:[^,]*([,}])'

> print "Replacing variable 'NETWORK_ID' to 'localnet' using regexp: '([,{])NETWORK_ID:[^,]*([,}])'"
Replacing variable 'NETWORK_ID' to 'localnet' using regexp: '([,{])NETWORK_ID:[^,]*([,}])'

> print "Replacing variable 'ACCOUNT_ID_SUFFIX' to 'test.near' using regexp: '([,{])ACCOUNT_ID_SUFFIX:[^,]*([,}])'"
Replacing variable 'ACCOUNT_ID_SUFFIX' to 'test.near' using regexp: '([,{])ACCOUNT_ID_SUFFIX:[^,]*([,}])'

> print "Replacing variable 'ACCESS_KEY_FUNDING_AMOUNT' to '3000000000000000000000000' using regexp: '([,{])ACCESS_KEY_FUNDING_AMOUNT:[^,]*([,}])'"
Replacing variable 'ACCESS_KEY_FUNDING_AMOUNT' to '3000000000000000000000000' using regexp: '([,{])ACCESS_KEY_FUNDING_AMOUNT:[^,]*([,}])'

> add_service service_id="wallet"
Service 'wallet' added with internal ID 'wallet-1670256444'

> print "Explorer wallet struct(public_url = struct(ip_address = \"127.0.0.1\", path = \"\", port_number = 8334, protocol = \"http\"))"
Explorer wallet struct(public_url = struct(ip_address = "127.0.0.1", path = "", port_number = 8334, protocol = "http"))

Starlark code successfully executed. Output was:
{
    "contract_helper_service_url": "http://127.0.0.1:8330",
    "explorer_url": "http://127.0.0.1:8331",
    "near_node_rpc_url": "http://127.0.0.1:8332",
    "network_name": "localnet",
    "root_validator_key": {
        "account_id": "test.near",
        "public_key": "ed25519:3Kuyi2DUXdoHgoaNEvCxa1m6G8xqc6Xs7WGajaqLhNmW",
        "secret_key": "ed25519:2ykcMLiM7vCmsSECcgfmUzihBtNdBv7v2CxNi94sNt4R8ar4xsrMMYvtsSNGQDfSRhNWXEnZvgx2wzS9ViBiS9jW"
    },
    "wallet_url": "http://127.0.0.1:8334"
}
INFO[2022-12-05T13:07:41-03:00] ====================================================
INFO[2022-12-05T13:07:41-03:00] ||          Created enclave: black-water          ||
INFO[2022-12-05T13:07:41-03:00] ====================================================

Broken URL screenshots:

Screen Shot 2022-12-05 at 1 10 56 PM Screen Shot 2022-12-05 at 1 10 52 PM

Enclave dump output (based on the Docker inspect output, it seems like the services are getting bound to ephemeral ports, rather than the hardcoded ports): black-water.zip

mieubrisse commented 1 year ago

Confirmed that this is because the ports are getting bound to ephemerals, rather than to the public static ports:

 13:12:09 /tmp  kt enclave inspect black-water
WARN[2022-12-05T13:13:24-03:00] You are running an old version of the Kurtosis CLI; we suggest you to update it to the latest version, '0.57.6'
WARN[2022-12-05T13:13:24-03:00] You can manually upgrade the CLI tool following these instructions: https://docs.kurtosistech.com/installation.html#upgrading-kurtosis-cli
Enclave ID:                           black-water
Enclave Status:                       RUNNING
Creation Time:                        Mon, 05 Dec 2022 13:06:20 -03
API Container Status:                 RUNNING
API Container Host GRPC Port:         127.0.0.1:61608
API Container Host GRPC Proxy Port:   127.0.0.1:61607

========================================= Kurtosis Modules =========================================
GUID   ID   Ports

========================================== User Services ==========================================
GUID                                   ID                          Ports                                   Status
contract-helper-db-1670256390          contract-helper-db          postgres: 5432/tcp -> 127.0.0.1:61614   RUNNING
contract-helper-dynamo-db-1670256406   contract-helper-dynamo-db   default: 8000/tcp -> 127.0.0.1:61618    RUNNING
contract-helper-service-1670256426     contract-helper-service     rest: 3000/tcp -> 127.0.0.1:61627       RUNNING
explorer-backend-1670256430            explorer-backend            http: 8080/tcp -> 127.0.0.1:61631       RUNNING
explorer-frontend-1670256434           explorer-frontend           http: 3000/tcp -> 127.0.0.1:61636       RUNNING
indexer-node-1670256410                indexer-node                gossip: 24567/tcp -> 127.0.0.1:61622    RUNNING
                                                                   rpc: 3030/tcp -> 127.0.0.1:61623
wallet-1670256444                      wallet                      http: 3004/tcp -> 127.0.0.1:61641       RUNNING
h4ck3rk3y commented 1 year ago

You might be on a version of Kurtosis that doesn't support public_ports, can you upgrade and try again?

This is my test from the afternoon, https://github.com/kurtosis-tech/near-package/pull/5#issuecomment-1336770193

mieubrisse commented 1 year ago

Confirmed that this happens on the lastest version:

 13:16:29 /tmp  kurtosis enclave inspect cold-shape
WARN[2022-12-05T13:16:57-03:00] The currently-running Kurtosis engine version is '0.57.4' but the latest version is '0.57.6'; you can pull the latest fixes by running 'kurtosis engine restart'
Enclave ID:                           cold-shape
Enclave Status:                       RUNNING
Creation Time:                        Mon, 05 Dec 2022 13:15:31 -03
API Container Status:                 RUNNING
API Container Host GRPC Port:         127.0.0.1:61729
API Container Host GRPC Proxy Port:   127.0.0.1:61730

========================================= Kurtosis Modules =========================================
GUID   ID   Ports

========================================== User Services ==========================================
GUID                                   ID                          Ports                                   Status
contract-helper-db-1670256937          contract-helper-db          postgres: 5432/tcp -> 127.0.0.1:61736   RUNNING
contract-helper-dynamo-db-1670256952   contract-helper-dynamo-db   default: 8000/tcp -> 127.0.0.1:61743    RUNNING
contract-helper-service-1670256972     contract-helper-service     rest: 3000/tcp -> 127.0.0.1:61752       RUNNING
explorer-backend-1670256975            explorer-backend            http: 8080/tcp -> 127.0.0.1:61756       RUNNING
explorer-frontend-1670256980           explorer-frontend           http: 3000/tcp -> 127.0.0.1:61761       RUNNING
indexer-node-1670256955                indexer-node                gossip: 24567/tcp -> 127.0.0.1:61747    RUNNING
                                                                   rpc: 3030/tcp -> 127.0.0.1:61748
wallet-1670256984                      wallet                      http: 3004/tcp -> 127.0.0.1:61766       RUNNING

 13:16:58 /tmp  kurtosis version
0.57.6
h4ck3rk3y commented 1 year ago

I think you are still on engine 0.57.4, try with the new one!

mieubrisse commented 1 year ago

Confirmed - upgrading CLI and restarting the engine did it!