Error: http code 400 from: vaultUnseal

[rksharma95]

I was trying to install management hub services on my local system with Ubuntu 20.04.4 and getting this error

$ curl -sSL https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh | sudo bash
------- Checking input and initializing...
Management hub services will listen on http://127.0.0.1
Updating apt-get package index...
Installing prerequisites, this could take a minute...
------- Downloading template files...
Substituting environment variables into template files...
------- Downloading/starting Horizon management hub services...
Downloading management hub docker images...
Pulling openhorizon/amd64_agbot:latest ...
Pulling openhorizon/amd64_exchange-api:latest ...
Pulling openhorizon/amd64_cloud-sync-service:latest ...
Trying to pull postgres:13 ...
Pulling mongo:latest ...
Pulling openhorizon/sdo-owner-services:latest ...
Pulling openhorizon/amd64_vault:latest ...
Starting management hub containers...
mongo is up-to-date
postgres is up-to-date
Recreating exchange-api ... done
vault is up-to-date
Recreating css-api      ... 
Recreating css-api      ... done
agbot is up-to-date
Waiting for the exchange
------- Creating the user org, and the admin user in both orgs...
Creating exchange hub admin user, and the admin user and agbot in the system org...
------- Creating a Vault instance and performing all setup and configuration operations ...
Checking Vault service status, initialization, and seal...
Found a Vault instance.
Vault instance is sealed. Unsealing...
Error: http code 400 from: vaultUnseal

the docker ps result is as:

15d82c56dce3   openhorizon/amd64_cloud-sync-service:latest   "/usr/edge-sync-serv…"   7 minutes ago   Up 7 minutes (unhealthy)          127.0.0.1:9443->8080/tcp                                                       css-api
5adac2a46b82   openhorizon/amd64_exchange-api:latest         "/bin/sh -c '/usr/bi…"   7 minutes ago   Up 7 minutes (healthy)            8083/tcp, 127.0.0.1:3090->8080/tcp                                             exchange-api
05bfcfc2bf28   openhorizon/amd64_agbot:latest                "/bin/sh -c /usr/hor…"   18 hours ago    Up About an hour (healthy)        127.0.0.1:3110->8080/tcp, 127.0.0.1:3111->8083/tcp                             agbot
c64f3a2ddf65   openhorizon/amd64_vault:latest                "entrypoint.sh server"   18 hours ago    Up About an hour (healthy)        127.0.0.1:8200->8200/tcp                                                       vault
2e6fc482069f   openhorizon/sdo-owner-services:latest         "/bin/sh -c $WORKDIR…"   18 hours ago    Up About an hour (healthy)        127.0.0.1:8040->8040/tcp, 127.0.0.1:8042->8042/tcp, 127.0.0.1:9008->9008/tcp   sdo-owner-services
8ad7e964931d   postgres:13                                   "docker-entrypoint.s…"   18 hours ago    Up About an hour (healthy)        5432/tcp                                                                       postgres
8a84cbc52a4a   mongo:latest                                  "docker-entrypoint.s…"   18 hours ago    Restarting (132) 25 seconds ago      

[mounaiban]

I ran into the same problem too trying to create a new all-in-one instance from scratch. It may be due to missing vault keys.

From what I can figure out from vaultUnseal() in deploy-mgmt-hub.sh, the key needed to unseal the vault is missing; $VAULT_UNSEAL_KEY is where it should be but the variable is never set.

The key is supposed to come from another file, /tmp/horizon-all-in-1/vaultkeys.json, which was never created. We might need to add a routine to generate the vaultkeys.json file. I'm personally not sure about the format for vaultkeys.json besides that it needs two values:

{
    "root_token": "???",
    "keys_base64": [ "???", ...]
}

At this point, I am trying to find out the correct lengths of the root token and the keys.

[cluibm]

I also ran into this problem upon an unsuccessful run of deploy-mgmt-hub.sh although I do have the key files (and values) /tmp/horizon-all-in-1/vaultkeys.json

[joewxboy]

The vault errors may be a side effect of MongoDB not starting up properly. By ensuring that you install the same version of Mongo that was used in our test suite, you may avoid this error entirely. Here's what I use:

export MONGO_IMAGE_TAG=4.0.6
export HZN_LISTEN_IP=<put your public IP address here>
curl -sSL https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh -A -R | bash

[joewxboy]

If someone can purge their existing installation and follow the steps above, we can mark this as fixed, update the documentation accordingly, and then close the issue.

@johnwalicki FYI ^^^

[cluibm]

I tried it and it works now. OS VERSION="22.04.1 LTS (Jammy Jellyfish)"

bash deploy-mgmt-hub.sh -PS

...

export MONGO_IMAGE_TAG=4.0.6
export HZN_LISTEN_IP=<put your public IP address here>
curl -sSL https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh -A -R | bash

...

bash test-mgmt-hub.sh 
testMgmtHubServices
testDefaultsFile
testAgent
testEdgeService
testExchangeHubAdmin
testExchangeSystemOrg
testExchangeUserOrg
testMms

Ran 8 tests.

OK

[joewxboy]

Thanks. I'll see where we need to update the docs and then close this issue as fixed.

[naphelps]

Updated the default image tag for mongo. Should no longer be an issue.