Confusing error message when `ENTROPY_TUI_ENDPOINT` not set correctly, and failure to connect when setting the chain endpoint to `localhost`

ameba23 commented 2 weeks ago

This is maybe an issue for the SDK rather that the CLI, but putting it here since steps to recreate involve the CLI, and i came across it when trying out the new release of the CLI.

It feels like i am using the wrong version of this CLI - but since there is no --version option and the ascii logo still displays core v0.1.0, cant be totally sure. I have npm version 9.5.0 and node version 18.15.0 I did:

$npm i -g @entropyxyz/cli@0.0.4-1
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: 'yocto-spinner@0.1.1',
npm WARN EBADENGINE   required: { node: '>=18.19' },
npm WARN EBADENGINE   current: { node: 'v18.15.0', npm: '9.5.0' }
npm WARN EBADENGINE }
npm WARN deprecated @substrate/connect@0.8.11: versions below 1.x are no longer maintained

changed 162 packages in 21s

23 packages are looking for funding
  run `npm fund` for details

$ whereis entropy
entropy: /home/turnip/.nvm/versions/node/v18.15.0/bin/entropy

$ ls -l /home/turnip/.nvm/versions/node/v18.15.0/bin/entropy
lrwxrwxrwx 1 turnip turnip 47 Oct 24 08:58 /home/turnip/.nvm/versions/node/v18.15.0/bin/entropy -> ../lib/node_modules/@entropyxyz/cli/dist/cli.js

$ head ~/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/package.json
{
  "name": "@entropyxyz/cli",
  "version": "0.0.4-1",
  "description": "cli and tui for interacting with the entropy protocol",
  "type": "module",
  "scripts": {
    "start": "yarn build:global && entropy",
    "start:debug": "DEBUG=@entropyxyz/cli yarn start",
    "build": "tsup",
    "build:global": "tsup && npm install -g",

So it looks like i have the right version installed

I am running a local test network with the docker compose script from entropy-core - NOT the docker script from the SDK. So i dont know if that makes a difference but ideally it shouldn't.

I am running the 0.3.0 release candidate - lets check that:

$ curl 127.0.0.1:3001/version | less
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    35  100    35    0     0  22026      0 --:--:-- --:--:-- --:--:-- 35000
0.3.0-rc.1-VERGEN_IDEMPOTENT_OUTPUT

I have also checked i am able to register using the entropy-test-cli and the //Alice account.

Lets try to register using the TUI:

$ export ENTROPY_ENDPOINT=ws://localhost:9944

$ entropy

Then with the TUI i create an account and attempt to register it. The account isn't funded so i don't expect this to work, but i hope for a nice error message:

 ? Select Action Register
Attempting to register the address: 5Dcfax2yfAtbxsS2xpL6qQArHFp7NDUSqP5oWJrAddNL1sAq
file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/node_modules/@polkadot/types/metadata/decorate/extrinsics/createUnchecked.js:11
            throw new Error(`Extrinsic ${section}.${funcName} expects ${expectedArgs.length} arguments, got ${args.length}.`);
                  ^

Error: Extrinsic registry.register expects 3 arguments, got 2.
    at extrinsicFn (file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/node_modules/@polkadot/types/metadata/decorate/extrinsics/createUnchecked.js:11:19)
    at Object.decorated [as register] (file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/node_modules/@polkadot/api/base/Decorate.js:539:50)
    at RegistrationManager.register (file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/node_modules/@entropyxyz/sdk/dist/index.js:99:51)
    at Entropy.register (file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/node_modules/@entropyxyz/sdk/dist/index.js:831:57)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v18.15.0

This is strange, it seems like is is expecting version v0.2.0 which had 3 arguments to the register extrinsic (before key visibility was removed).

But i can see that in v0.3.0 of the SDK, only 2 arguments are used: https://github.com/entropyxyz/sdk/blob/faa9ddca18618e8d7d59134ee6c9e9fef40047a9/src/registration/index.ts#L90

Which makes me wonder if i am actually running the right version of the entropy CLI.

Lets try registering with the command line:

$ entropy account create foo | less
{"name":"foo","address":"5ENWQkRZcpzYZqKe8yxPJv3aDQUMpRU3t288YL9hEiDTMBkM","verifyingKeys":[]}

$ entropy account register foo
2024-10-24 09:13:37          API-WS: disconnected from ws://localhost:9944: 1006:: Abnormal Closure
connect ECONNREFUSED ::1:9944
file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/dist/cli.js:3566
    throw new Error("Signer keypair is undefined or not properly initialized.");
          ^

Error: Signer keypair is undefined or not properly initialized.
    at loadEntropy (file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/dist/cli.js:3566:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Command.<anonymous> (file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/dist/cli.js:11877:21)
    at async Command.parseAsync (/home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/node_modules/commander/lib/command.js:1092:5)

Node.js v18.15.0

This is also strange, now it seems it cannot connect to the chain endpoint.

Lets double check that the chain endpoint is accessible with the other CLI:

$ export ENTROPY_DEVNET=ws://localhost:9944
turnip ~ $ entropy-test-cli status
There are 1 registered Entropy accounts.

Verifying key:                                                   Visibility:  Programs:
02784b3cf0fb8341c49f44a393030d335e0281b6532d6a065e0da287cba0384ef8 ["0x3289…d9b3"]

There are 2 stored programs

Hash                                                             Stored by:                                       Times used: Size in bytes: Configurable? Has auxiliary?
3289435006003c3c83da5582d779d810c343f177fb1fecddbaca5d326ef9d9b3 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY           1          20971 false         false
0000000000000000000000000000000000000000000000000000000000000000 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY          10         300498 true          true
Success: Got status
That took 73.35926ms

I have the feeling that either i am somehow running the wrong version, or there are issues specific to the entropy-core docker compose setup as opposed to the SDK docker compose script which i assume is what yous are testing against.

Now lets try with zach's testnet, which is the only staging network on v0.3.0-rc.1 i currently know of:

$ export ENTROPY_ENDPOINT=wss://api.dev.testnet.zach.sandbox.entropy.xyz
turnip ~ $ entropy account register foo
broken config:
undefined:1

SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at deserialize (file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/dist/cli.js:3308:17)
    at async EventEmitter.<anonymous> (file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/dist/cli.js:3444:23)

Node.js v18.15.0

not sure what went on there, but i'll try with the TUI:

? Select Action Register
Attempting to register the address: 5ENWQkRZcpzYZqKe8yxPJv3aDQUMpRU3t288YL9hEiDTMBkM
file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/node_modules/@polkadot/types/metadata/decorate/extrinsics/createUnchecked.js:11
            throw new Error(`Extrinsic ${section}.${funcName} expects ${expectedArgs.length} arguments, got ${args.length}.`);
                  ^

Error: Extrinsic registry.register expects 3 arguments, got 2.
    at extrinsicFn (file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/node_modules/@polkadot/types/metadata/decorate/extrinsics/createUnchecked.js:11:19)
    at Object.decorated [as register] (file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/node_modules/@polkadot/api/base/Decorate.js:539:50)
    at RegistrationManager.register (file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/node_modules/@entropyxyz/sdk/dist/index.js:99:51)
    at Entropy.register (file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/node_modules/@entropyxyz/sdk/dist/index.js:831:57)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

hmm same as before.

Finally i try with the CLI again:

$ entropy account register foo
2024-10-24 09:34:25        RPC-CORE: submitAndWatchExtrinsic(extrinsic: Extrinsic): ExtrinsicStatus:: 1010: Invalid Transaction: Inability to pay some fees , e.g. account balance too low
2024-10-24 09:34:25        RPC-CORE: submitAndWatchExtrinsic(extrinsic: Extrinsic): ExtrinsicStatus:: 1010: Invalid Transaction: Inability to pay some fees , e.g. account balance too low
file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/node_modules/@entropyxyz/sdk/dist/index.js:60
        reject(Error(e.message));
               ^

Error: 1010: Invalid Transaction: Inability to pay some fees , e.g. account balance too low
    at file:///home/turnip/.nvm/versions/node/v18.15.0/lib/node_modules/@entropyxyz/cli/node_modules/@entropyxyz/sdk/dist/index.js:60:16
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v18.15.0

Yes! This is the desired behavior since that account isn't funded.

So it seems this is an issue related to the entropy-core docker-compose setup. However there is still the issue that there is different behavior with the TUI vs CLI.

I haven't managed to fund an account on Zach's staging network as for some reason the root account seems to be different than the one i have saved.

mixmix commented 2 weeks ago

Ah, for annoying reasons documented elsewhere for the tui the env is currently ENTROPY_TUI_ENDPOINT

worth nothing that error message is ueless tho

mixmix commented 2 weeks ago

@ameba23 please try the sdk docker scripts, see the instructions on the rc PR (on the sdk)

ameba23 commented 2 weeks ago

I can confim things work fine the SDK docker scripts:

I first jumpstart using the test cli to avoid confusion with different endpoint on the TUI and CLI:

$ entropy-test-cli jumpstart-network -m //Alice

$ export ENTROPY_ENDPOINT=dev
$ rm -rf ~/.config/entropy-cryptography
$ subkey inspect //Alice
Secret Key URI `//Alice` is account:
  Network ID:        substrate
  Secret seed:       0xe5be9a5092b81bca64be81d212e7f2f9eba183bb7a90954f7b76361f6edb5c0a
  Public key (hex):  0xd43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d
  Account ID:        0xd43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d
  Public key (SS58): 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY
  SS58 Address:      5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY
turnip ~/.config $ entropy account import alice 0xe5be9a5092b81bca64be81d212e7f2f9eba183bb7a90954f7b76361f6edb5c0a
turnip ~/.config $ entropy account register alice | less
0x02591659813728d7f77cca66219e2596ab9b5ae7de56e367518034cae86126b414

Now lets check that alice is registered. Im not sure how to list registered accounts so use the test-cli:

turnip ~/.config $ entropy-test-cli status
There are 2 registered Entropy accounts.

Verifying key:                                                   Visibility:  Programs:
022b8bc622bead97afffc332862a624c9e39a287a598a1c46be003d5ec81f90b9c ["0x0000…0000"]
02591659813728d7f77cca66219e2596ab9b5ae7de56e367518034cae86126b414 ["0x0000…0000"]

There are 1 stored programs

Hash                                                             Stored by:                                       Times used: Size in bytes: Configurable? Has auxiliary?
0000000000000000000000000000000000000000000000000000000000000000 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY          12         300498 true          true
Success: Got status
That took 71.433999ms

Great.

Now sign a message:

 $ entropy sign --account alice "some message to sign"
{"verifyingKey":"0x02591659813728d7f77cca66219e2596ab9b5ae7de56e367518034cae86126b414","signature":"0xf4c2cad0c627ec2c0e4ce9cf0f300c9c278fb4ed050aa1ad934f20055fc9e7143f465721ce6d8dd5e5854a035

Excellent.

So now i think the problem might be related to that i needed to export ENTROPY_ENDPOINT=dev. When i tried with ws://localhost:9944 i got the same error as before: API-WS: disconnected from ws://localhost:9944: 1006:: Abnormal Closure.

Does dev have some special meaning or is the chain endpoint hostname literally set to 'dev'?

mixmix commented 2 weeks ago

This is awesome, thanks for the clear write-up @ameba23 :heart:

mixmix commented 2 weeks ago

Does dev have some special meaning or is the chain endpoint hostname literally set to 'dev'?

dev is an alias for the the default dev testnet. The aliases are stored in your config

$cat ~/.config/entropy-cryptography/entropy-cli.json | jq .endpoints.dev

"ws://127.0.0.1:9944"

Sooo... "localhost" might be a problem.

We should try export ENTROPY_ENDPOINT=ws://127.0.0.1:9944 to be sure that's working ...

mixmix commented 2 weeks ago

Im not sure how to list registered accounts

entropy account ls | jq

[
  {
    "name": "mix",
    "address": "5CBCdVRZDw8zTvnVkcrwSriAuJKLTnajLpZkz2BU4aNWgGNs",
    "verifyingKeys": []  # <<< would have keys in here if registered
  }
]

ameba23 commented 2 weeks ago

Sooo... "localhost" might be a problem.

We should try export ENTROPY_ENDPOINT=ws://127.0.0.1:9944 to be sure that's working ...

Yep good catch that was it. With both the entropy-core and sdk docker compose environments, if i set the endpoint to the loopback interface ws://127.0.0.1:9944 it works and if i set it to localhost ws://localhost:9944, it doesn't connect.

So i can confirm i am able to register and sign messages on both setups.

I'm very happy that we have the same behavior on both test setups cos otherwise this would be a fundamental flaw in how we are testing.

mixmix commented 1 week ago

Synthesis

[x] --version has been added to next RC
[ ] ENTROPY_TUI_ENDPOINT is terrible
- It's great to have immediatate validation that that was a bad idea
- doing my best to revert that (use ENTROPY_ENDPOINT everywhere)
- => https://github.com/entropyxyz/cli/issues/271
[ ] multiple docker scripts
- @entropyxyz/sdk:
- entropy-core:
- => https://github.com/entropyxyz/cli/issues/272
[ ] incorrect endpoint
- TUI error messages when endpoint is not live / correct is terrible!
```
Error: Extrinsic registry.register expects 3 arguments, got 2.
```
- this might be it connected to old testnet actually
- perhaps we should have some check in place that an endpoint is the expected version
- => https://github.com/entropyxyz/cli/issues/273
[ ] corrupted config file
```
broken config:
```
- you've got a corrupted config file... this is an issue we need to investigate
- => https://github.com/entropyxyz/cli/issues/274
[ ] ENTROPY_ENDPOINT=ws://localhost:9944 broken?
```
API-WS: disconnected from ws://localhost:9944: 1006:: Abnormal Closure
```
- ws://127.0.0.1:9944 works
- => https://github.com/entropyxyz/cli/issues/275

mixmix commented 1 week ago

What a rich bounty of Issues you've surfaced! :heart: :pray: (I'm serious, these are all the coffee-tables-in-the-dark that I we need to know about and get removed for launch, so genuinely thank you for patiently banging your shins on so many!)

I think I've split each problem you found out into their own issues. Can you mark this as closed if you're satisfied @ameba23 ?

entropyxyz / cli

Confusing error message when `ENTROPY_TUI_ENDPOINT` not set correctly, and failure to connect when setting the chain endpoint to `localhost` #268

Synthesis