open-horizon / devops

Devops processes to build and deploy horizon components
Apache License 2.0
10 stars 43 forks source link

Encounter Error in ./deploy-mgmt-hub.sh Error: exit code 5 #137

Closed ilamprinos closed 1 year ago

ilamprinos commented 1 year ago

Hello, I am facing the following problem while running ./deploy-mgmt-hub.sh or curl -sSL https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh | bash (OS: Ubuntu server 22.04 and 18.04). Any ideas on how to solve it?

deploy-mgmt-hub.sh logs: ------- Checking input and initializing... Management hub services will listen on http://127.0.0.1 Updating apt-get package index... Installing prerequisites, this could take a minute... ------- Downloading template files... Substituting environment variables into template files... ------- Downloading/starting Horizon management hub services... Downloading management hub docker images... Pulling openhorizon/amd64_agbot:latest ... Pulling openhorizon/amd64_exchange-api:latest ... Pulling openhorizon/amd64_cloud-sync-service:latest ... Pulling postgres:13 ... Pulling mongo:latest ... Pulling openhorizon/sdo-owner-services:latest ... Pulling openhorizon/amd64_vault:latest ... Starting management hub containers... [+] Running 13/13 ⠿ Network hzn_horizonnet Created 0.1s ⠿ Volume "hzn_vaultvol" Created 0.0s ⠿ Volume "hzn_agbotmsgkeyvol" Created 0.0s ⠿ Volume "hzn_postgresvol" Created 0.0s ⠿ Volume "hzn_mongovol" Created 0.0s ⠿ Volume "hzn_ocsdb" Created 0.0s ⠿ Container mongo Started 2.6s ⠿ Container postgres Started 2.5s ⠿ Container exchange-api Started 3.2s ⠿ Container css-api Started 5.5s ⠿ Container sdo-owner-services Started 5.9s ⠿ Container vault Started 6.1s ⠿ Container agbot Started 7.5s Waiting for the exchange............ ------- Creating the user org, and the admin user in both orgs... Creating exchange hub admin user, and the admin user and agbot in the system org... ------- Creating a Vault instance and performing all setup and configuration operations ... Checking Vault service status, initialization, and seal... A Vault instance has not been initialized. Initializing... Vault instance is sealed. Unsealing... Creating KV ver.2 secrets engine openhorizon... Registering auth plugin openhorizon-exchange to Vault instance... Generating SHA256 hash of openhorizon-exchange plugin... Enabling auth method openhorizon-exchange for secrets engine openhorizon... Configuring auth method openhorizon-exchange for use with the Exchange... Success! Data written to: auth/openhorizon/config ------- Creating an agbot in the exchange... [+] Running 1/1 ⠿ Container agbot Started 12.0s Creating exchange user org and admin user... ------- Downloading/installing/configuring Horizon agent and CLI... Downloading the Horizon agent and CLI packages... Installing the Horizon agent and CLI packages... Configuring the Horizon agent and CLI... Publishing /tmp/horizon-all-in-1/agent-install.cfg in CSS as public object agent-install.cfg in the IBM org... Digital sign with SHA1 will be performed for data integrity. It will delay the MMS object publish. Start hashing the file... Data hash is generated. Start digital signing with the data hash... Digital sign finished. Error: Encountered HTTP error: Put "http://127.0.0.1:9443/api/v1/objects/IBM/agent_files/agent-install.cfg": read tcp 127.0.0.1:59214->127.0.0.1:9443: read: connection reset by peer calling Model Management Service REST API PUT http://127.0.0.1:9443/api/v1/objects/IBM/agent_files/agent-install.cfg. HTTP status: . Error: exit code 5 from: publishing /tmp/horizon-all-in-1/agent-install.cfg in CSS as a public object

MrHassanMurtaza commented 1 year ago

@ilamprinos are you able to solve the issue? We are facing same issue and looking at logs it seems like edge sync service is not able to connect to Mongo DB:

[cssuser@a8017080f69a ~]$ ./cloud-sync-service
* CSS: 2022/11/15 13:57:50 Loaded configuration:
  NodeType  CSS
  DestinationType  Cloud
  DestinationID  Cloud
  OrgID
  ListeningType  unsecure
  ListeningAddress
  SecureListeningPort  8080
  UnsecureListeningPort  8080
  ServerCertificate  <...>
  ServerKey  <...>
  CSSOnWIoTP  false
  UsingEdgeConnector  false
  LeadershipTimeout  45
  AuthenticationHandler  dummy
  ObjectQueueBufferSize  1000
  VerifyQueueBufferSize  500
  CommunicationProtocol  http
  MQTTClientID
  MQTTUserName
  MQTTPassword
  MQTTUseSSL  true
  MQTTCACertificate  <...>
  MQTTSSLCert
  MQTTSSLKey
  MQTTAllowInvalidCertificates  false
  MQTTBrokerConnectTimeout  300
  MQTTParallelMode
  PersistenceRootPath  /var/edge-sync-service/persist/
  BrokerAddress
  BrokerPort  0
  HTTPPollingInterval  10
  HTTPCSSHost
  HTTPCSSPort  0
  HTTPCSSUseSSL  false
  HTTPCSSCACertificate
  HTTPESSClientTimeout  120
  HTTPESSObjClientTimeout  600
  HTTPCSSObjDownloadConcurrencyMultiplier  1
  LogLevel  INFO
  LogRootPath  /var/edge-sync-service/log
  LogTraceDestination  stdout
  LogFileName  sync-service
  TraceLevel  INFO
  TraceRootPath  /var/edge-sync-service/trace
  TraceFileName  sync-service
  LogTraceFileSizeKB  20000
  MaxCompressedlLogTraceFilesNumber  50
  LogTraceMaintenanceInterval  60
  ResendInterval  10
  ESSCallSPIRetryInterval  2
  ESSSPIMaxRetry  1000
  ESSPingInterval  1
  RemoveESSRegistrationTime  30
  EnableDataChunk  true
  MaxDataChunkSize  5242880
  MaxInflightChunks  1
  MongoAddressCsv  mongo:27017
  MongoAuthDbName  admin
  MongoDbName  d_edge
  MongoUsername
  MongoPassword
  MongoUseSSL  false
  MongoCACertificate
  MongoAllowInvalidCertificates  false
  MongoSessionCacheSize  1
  MongoSleepTimeBetweenRetry  4000
  DatabaseConnectTimeout  300
  StorageMaintenanceInterval  30
  ObjectActivationInterval  30
  StorageProvider  mongo
  ESSConsumedObjectsKept  1000
  MessagingGroupCacheExpiration  60
  ShutdownQuiesceTime  60
  ObjectsDataPath
CSS: 2022/11/15 13:57:50 INFO: Horizon Authenticator starting with exchange authenticated identity
* CSS: 2022/11/15 13:57:50 INFO: Connecting to mongo...
* CSS: 2022/11/15 13:58:12 ERROR: Failed to dial mgo. Error: no reachable servers
* CSS: 2022/11/15 13:58:12 ERROR: Retrying to connect to mongo
ilamprinos commented 1 year ago

@MrHassanMurtaza Unfortunately I have nothing yet. I 'll let you know if I find anything.

ilamprinos commented 1 year ago

@MrHassanMurtaza solved it by changing line 197 of deploy-mgmt-hub.sh to: export MONGO_IMAGE_TAG=${MONGO_IMAGE_TAG:-4.0.18} from: export MONGO_IMAGE_TAG=${MONGO_IMAGE_TAG:-latest} Let me know if this worked for you.

MrHassanMurtaza commented 1 year ago

Super! Thanks @ilamprinos seems to pass through that issue.