manthey
commented
1 year ago Hmm... the error is that the girder container can't resolve the mongo container. Normally, docker-compose creates a docker network for the compose group (probably called dsa_default). A docker inspect of each of the dsa-girder-1 and dsa-mongodb-1 containers should show that they both have a NetworkMode of dsa_default and that they have aliases that include girder and mongodb respectively. If docker inspect shows this, then I don't know how there is a failure in name resolution. If not, then whatever is causing that network change is going to be the problem.
rogertrullo
Hi, I have been trying to deploy using
DSA_USER=$(id -u):$(id -g) docker-compose up. I am in an dind setting where the host docker is ubuntu 22. I keep having this error about mongodb connectivity. Here are the logs:dsa_girder
dsa-girder-1 | id: ‘112’: no such user dsa-girder-1 | The user `ubuntu' is already a member of `root'. dsa-girder-1 | net.ipv4.conf.eth0.route_localnet = 1 dsa-girder-1 | iptables v1.8.4 (legacy): Bad IP address ";;" dsa-girder-1 | dsa-girder-1 | Try `iptables -h' or 'iptables --help' for more information. dsa-girder-1 | ==== Pre-Provisioning === dsa-girder-1 | ==== Provisioning === dsa-girder-1 | Running in mode: production dsa-girder-1 | Connecting to MongoDB: mongodb://mongodb:27017/girder?socketTimeoutMS=3600000 dsa-girder-1 | Traceback (most recent call last): dsa-girder-1 | File "/opt/digital_slide_archive/devops/dsa/provision.py", line 604, indsa_mongodb
dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.344+00:00"},"s":"I", "c":"CONTROL", "id":23285, "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.345+00:00"},"s":"I", "c":"NETWORK", "id":4915701, "ctx":"main","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":17},"outgoing":{"minWireVersion":6,"maxWireVersion":17},"isInternalClient":true}}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.346+00:00"},"s":"I", "c":"NETWORK", "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.347+00:00"},"s":"I", "c":"REPL", "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationDonorService","namespace":"config.tenantMigrationDonors"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.347+00:00"},"s":"I", "c":"REPL", "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationRecipientService","namespace":"config.tenantMigrationRecipients"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.347+00:00"},"s":"I", "c":"REPL", "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"ShardSplitDonorService","namespace":"config.tenantSplitDonors"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.347+00:00"},"s":"I", "c":"CONTROL", "id":5945603, "ctx":"main","msg":"Multi threading initialized"} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.348+00:00"},"s":"I", "c":"CONTROL", "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":1,"port":27017,"dbPath":"/data/db","architecture":"64-bit","host":"6289af38e480"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.348+00:00"},"s":"I", "c":"CONTROL", "id":23403, "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"6.0.6","gitVersion":"26b4851a412cc8b9b4a18cdb6cd0f9f642e06aa7","openSSLVersion":"OpenSSL 3.0.2 15 Mar 2022","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2204","distarch":"x86_64","target_arch":"x86_64"}}}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.348+00:00"},"s":"I", "c":"CONTROL", "id":51765, "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"22.04"}}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.348+00:00"},"s":"I", "c":"CONTROL", "id":21951, "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"net":{"bindIp":"*","maxIncomingConnections":1000},"storage":{"journal":{"enabled":false}}}}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.348+00:00"},"s":"W", "c":"STORAGE", "id":22271, "ctx":"initandlisten","msg":"Detected unclean shutdown - Lock file is not empty","attr":{"lockFile":"/data/db/mongod.lock"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.348+00:00"},"s":"I", "c":"STORAGE", "id":22270, "ctx":"initandlisten","msg":"Storage engine to use detected by data files","attr":{"dbpath":"/data/db","storageEngine":"wiredTiger"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.348+00:00"},"s":"W", "c":"STORAGE", "id":22302, "ctx":"initandlisten","msg":"Recovering data from the last clean checkpoint."} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.348+00:00"},"s":"I", "c":"STORAGE", "id":22315, "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=63219M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,remove=true,path=journal,compressor=snappy),builtin_extension_config=(zstd=(compression_level=6)),file_manager=(close_idle_time=600,close_scan_interval=10,close_handle_minimum=2000),statistics_log=(wait=0),json_output=(error,message),verbose=[recovery_progress:1,checkpoint_progress:1,compact_progress:1,backup:0,checkpoint:0,compact:0,evict:0,history_store:0,recovery:0,rts:0,salvage:0,tiered:0,timestamp:0,transaction:0,verify:0,log:0],,log=(enabled=false),"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.933+00:00"},"s":"I", "c":"STORAGE", "id":4795906, "ctx":"initandlisten","msg":"WiredTiger opened","attr":{"durationMillis":585}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.933+00:00"},"s":"I", "c":"RECOVERY", "id":23987, "ctx":"initandlisten","msg":"WiredTiger recoveryTimestamp","attr":{"recoveryTimestamp":{"$timestamp":{"t":0,"i":0}}}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.939+00:00"},"s":"W", "c":"CONTROL", "id":22120, "ctx":"initandlisten","msg":"Access control is not enabled for the database. Read and write access to data and configuration is unrestricted","tags":["startupWarnings"]} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:36:27.942+00:00"},"s":"I", "c":"NETWORK", "id":4915702, "ctx":"initandlisten","msg":"Updated wire specification","attr":{"oldSpec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":17},"outgoing":{"minWireVersion":6,"maxWireV (mil) root@aida-ceacam5-mil-353632:/home/digital_slide_archive/devops/dsa# docker-compose logs mongodb dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:32.739+00:00"},"s":"I", "c":"CONTROL", "id":23285, "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:32.741+00:00"},"s":"I", "c":"NETWORK", "id":4915701, "ctx":"main","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":17},"outgoing":{"minWireVersion":6,"maxWireVersion":17},"isInternalClient":true}}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:32.741+00:00"},"s":"I", "c":"NETWORK", "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:32.742+00:00"},"s":"I", "c":"REPL", "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationDonorService","namespace":"config.tenantMigrationDonors"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:32.742+00:00"},"s":"I", "c":"REPL", "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationRecipientService","namespace":"config.tenantMigrationRecipients"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:32.742+00:00"},"s":"I", "c":"REPL", "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"ShardSplitDonorService","namespace":"config.tenantSplitDonors"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:32.742+00:00"},"s":"I", "c":"CONTROL", "id":5945603, "ctx":"main","msg":"Multi threading initialized"} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:32.743+00:00"},"s":"I", "c":"CONTROL", "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":1,"port":27017,"dbPath":"/data/db","architecture":"64-bit","host":"62f37aa2c0fa"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:32.743+00:00"},"s":"I", "c":"CONTROL", "id":23403, "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"6.0.6","gitVersion":"26b4851a412cc8b9b4a18cdb6cd0f9f642e06aa7","openSSLVersion":"OpenSSL 3.0.2 15 Mar 2022","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2204","distarch":"x86_64","target_arch":"x86_64"}}}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:32.743+00:00"},"s":"I", "c":"CONTROL", "id":51765, "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"22.04"}}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:32.743+00:00"},"s":"I", "c":"CONTROL", "id":21951, "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"net":{"bindIp":"*","maxIncomingConnections":1000},"storage":{"journal":{"enabled":false}}}}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:32.744+00:00"},"s":"I", "c":"STORAGE", "id":22315, "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=63219M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,remove=true,path=journal,compressor=snappy),builtin_extension_config=(zstd=(compression_level=6)),file_manager=(close_idle_time=600,close_scan_interval=10,close_handle_minimum=2000),statistics_log=(wait=0),json_output=(error,message),verbose=[recovery_progress:1,checkpoint_progress:1,compact_progress:1,backup:0,checkpoint:0,compact:0,evict:0,history_store:0,recovery:0,rts:0,salvage:0,tiered:0,timestamp:0,transaction:0,verify:0,log:0],,log=(enabled=false),"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.399+00:00"},"s":"I", "c":"STORAGE", "id":4795906, "ctx":"initandlisten","msg":"WiredTiger opened","attr":{"durationMillis":655}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.399+00:00"},"s":"I", "c":"RECOVERY", "id":23987, "ctx":"initandlisten","msg":"WiredTiger recoveryTimestamp","attr":{"recoveryTimestamp":{"$timestamp":{"t":0,"i":0}}}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.465+00:00"},"s":"W", "c":"CONTROL", "id":22120, "ctx":"initandlisten","msg":"Access control is not enabled for the database. Read and write access to data and configuration is unrestricted","tags":["startupWarnings"]} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.465+00:00"},"s":"I", "c":"STORAGE", "id":20320, "ctx":"initandlisten","msg":"createCollection","attr":{"namespace":"admin.system.version","uuidDisposition":"provided","uuid":{"uuid":{"$uuid":"a61fe108-dc79-42c4-b3d5-b0ec4083a60f"}},"options":{"uuid":{"$uuid":"a61fe108-dc79-42c4-b3d5-b0ec4083a60f"}}}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.511+00:00"},"s":"I", "c":"INDEX", "id":20345, "ctx":"initandlisten","msg":"Index build: done building","attr":{"buildUUID":null,"collectionUUID":{"uuid":{"$uuid":"a61fe108-dc79-42c4-b3d5-b0ec4083a60f"}},"namespace":"admin.system.version","index":"_id_","ident":"index-1--8030490562494532539","collectionIdent":"collection-0--8030490562494532539","commitTimestamp":null}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.511+00:00"},"s":"I", "c":"REPL", "id":20459, "ctx":"initandlisten","msg":"Setting featureCompatibilityVersion","attr":{"newVersion":"6.0"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.511+00:00"},"s":"I", "c":"REPL", "id":5853300, "ctx":"initandlisten","msg":"current featureCompatibilityVersion value","attr":{"featureCompatibilityVersion":"6.0","context":"setFCV"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.511+00:00"},"s":"I", "c":"NETWORK", "id":4915702, "ctx":"initandlisten","msg":"Updated wire specification","attr":{"oldSpec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":17},"outgoing":{"minWireVersion":6,"maxWireVersion":17},"isInternalClient":true},"newSpec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":17,"maxWireVersion":17},"outgoing":{"minWireVersion":17,"maxWireVersion":17},"isInternalClient":true}}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.511+00:00"},"s":"I", "c":"NETWORK", "id":4915702, "ctx":"initandlisten","msg":"Updated wire specification","attr":{"oldSpec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":17,"maxWireVersion":17},"outgoing":{"minWireVersion":17,"maxWireVersion":17},"isInternalClient":true},"newSpec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":17,"maxWireVersion":17},"outgoing":{"minWireVersion":17,"maxWireVersion":17},"isInternalClient":true}}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.511+00:00"},"s":"I", "c":"REPL", "id":5853300, "ctx":"initandlisten","msg":"current featureCompatibilityVersion value","attr":{"featureCompatibilityVersion":"6.0","context":"startup"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.511+00:00"},"s":"I", "c":"STORAGE", "id":5071100, "ctx":"initandlisten","msg":"Clearing temp directory"} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.511+00:00"},"s":"I", "c":"CONTROL", "id":20536, "ctx":"initandlisten","msg":"Flow Control is enabled on this deployment"} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.511+00:00"},"s":"I", "c":"FTDC", "id":20625, "ctx":"initandlisten","msg":"Initializing full-time diagnostic data capture","attr":{"dataDirectory":"/data/db/diagnostic.data"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.514+00:00"},"s":"I", "c":"STORAGE", "id":20320, "ctx":"initandlisten","msg":"createCollection","attr":{"namespace":"local.startup_log","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"ad633a7c-380a-4b08-9a26-1a3d8008e274"}},"options":{"capped":true,"size":10485760}}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.560+00:00"},"s":"I", "c":"INDEX", "id":20345, "ctx":"initandlisten","msg":"Index build: done building","attr":{"buildUUID":null,"collectionUUID":{"uuid":{"$uuid":"ad633a7c-380a-4b08-9a26-1a3d8008e274"}},"namespace":"local.startup_log","index":"_id_","ident":"index-3--8030490562494532539","collectionIdent":"collection-2--8030490562494532539","commitTimestamp":null}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.560+00:00"},"s":"I", "c":"REPL", "id":6015317, "ctx":"initandlisten","msg":"Setting new configuration state","attr":{"newState":"ConfigReplicationDisabled","oldState":"ConfigPreStart"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.560+00:00"},"s":"I", "c":"STORAGE", "id":22262, "ctx":"initandlisten","msg":"Timestamp monitor starting"} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.561+00:00"},"s":"I", "c":"CONTROL", "id":20712, "ctx":"LogicalSessionCacheReap","msg":"Sessions collection is not set up; waiting until next sessions reap interval","attr":{"error":"NamespaceNotFound: config.system.sessions does not exist"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.561+00:00"},"s":"I", "c":"STORAGE", "id":20320, "ctx":"LogicalSessionCacheRefresh","msg":"createCollection","attr":{"namespace":"config.system.sessions","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"0375342f-9f31-4e83-8990-8aac3b341fc8"}},"options":{}}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.561+00:00"},"s":"I", "c":"NETWORK", "id":23015, "ctx":"listener","msg":"Listening on","attr":{"address":"/tmp/mongodb-27017.sock"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.561+00:00"},"s":"I", "c":"NETWORK", "id":23015, "ctx":"listener","msg":"Listening on","attr":{"address":"0.0.0.0"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.561+00:00"},"s":"I", "c":"NETWORK", "id":23016, "ctx":"listener","msg":"Waiting for connections","attr":{"port":27017,"ssl":"off"}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.633+00:00"},"s":"I", "c":"INDEX", "id":20345, "ctx":"LogicalSessionCacheRefresh","msg":"Index build: done building","attr":{"buildUUID":null,"collectionUUID":{"uuid":{"$uuid":"0375342f-9f31-4e83-8990-8aac3b341fc8"}},"namespace":"config.system.sessions","index":"_id_","ident":"index-5--8030490562494532539","collectionIdent":"collection-4--8030490562494532539","commitTimestamp":null}} dsa-mongodb-1 | {"t":{"$date":"2023-06-01T19:39:33.633+00:00"},"s":"I", "c":"INDEX", "id":20345, "ctx":"LogicalSessionCacheRefresh","msg":"Index build: done building","attr":{"buildUUID":null,"collectionUUID":{"uuid":{"$uuid":"0375342f-9f31-4e83-8990-8aac3b341fc8"}},"namespace":"config.system.sessions","index":"lsidTTLIndex","ident":"index-6--8030490562494532539","collectionIdent":"collection-4--8030490562494532539","commitTimestamp":null}}Do you have any idea what can it be? Thanks!