search

dokku / dokku-mongo

a mongo plugin for dokku
MIT License
174 stars 32 forks source link

mongo:create - MongoServerError: Authentication failed #181

Closed nemanjam closed 5 months ago

nemanjam commented 1 year ago

Description of problem

Fresh Dokku install in Docker container. Just try to create new app and new service. It gives this:

ubuntu@arm1:~/traefik-proxy/apps/dokku$ dokku mongo:create mern-boilerplate-database
       Waiting for container to be ready
MongoServerError: Authentication failed.

# container logs
t={"$date":"2023-03-05T08:27:58.138+00:00"} s=I c=ACCESS id=20249 ctx=conn3 msg=Authentication failed attr={"mechanism":"SCRAM-SHA-256","speculative":true,"principalName":"admin","authenticationDatabase":"admin","remote":"127.0.0.1:45700","extraInfo":{},"error":"AuthenticationFailed: SCRAM authentication failed, storedKey mismatch"}

Later I get exact same error when app tries to connect to mongo service.

How reproducible

Steps to Reproduce

dokku network:set --global initial-network dokku-external

dokku apps:create mern-boilerplate

dokku mongo:create mern-boilerplate-database
       Waiting for container to be ready
MongoServerError: Authentication failed.

Actual Results

Auth error.

Expected Results

Should connect to database. No error output on service creation.

Environment Information

dokku report APP_NAME output

App is not needed, it happens on service creation even without any apps.

ubuntu@arm1:~/traefik-proxy/apps/dokku$ dokku report mern-boilerplate
-----> uname: Linux fc7fea5c9a9d 5.15.0-1029-oracle #35-Ubuntu SMP Tue Jan 24 15:21:05 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
-----> memory: 
                     total        used        free      shared  buff/cache   available
       Mem:          11940         515        4692           4        6732       11193
       Swap:             0           0           0
-----> docker version: 
       Client: Docker Engine - Community
        Version:           23.0.1
        API version:       1.42
        Go version:        go1.19.5
        Git commit:        a5ee5b1
        Built:             Thu Feb  9 19:46:32 2023
        OS/Arch:           linux/arm64
        Context:           default

       Server: Docker Engine - Community
        Engine:
         Version:          23.0.1
         API version:      1.42 (minimum version 1.12)
         Go version:       go1.19.5
         Git commit:       bc3805a
         Built:            Thu Feb  9 19:46:30 2023
         OS/Arch:          linux/arm64
         Experimental:     false
        containerd:
         Version:          1.6.18
         GitCommit:        2456e983eb9e37e47538f59ea18f2043c9a73640
        runc:
         Version:          1.1.4
         GitCommit:        v1.1.4-0-g5fd4c4d
        docker-init:
         Version:          0.19.0
         GitCommit:        de40ad0
-----> docker daemon info: 
       Client:
        Context:    default
        Debug Mode: true
        Plugins:
         compose: Docker Compose (Docker Inc.)
           Version:  v2.16.0
           Path:     /usr/libexec/docker/cli-plugins/docker-compose

       Server:
        Containers: 6
         Running: 4
         Paused: 0
         Stopped: 2
        Images: 17
        Server Version: 23.0.1
        Storage Driver: overlay2
         Backing Filesystem: extfs
         Supports d_type: true
         Using metacopy: false
         Native Overlay Diff: true
         userxattr: false
        Logging Driver: json-file
        Cgroup Driver: systemd
        Cgroup Version: 2
        Plugins:
         Volume: local
         Network: bridge host ipvlan macvlan null overlay
         Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
        Swarm: inactive
        Runtimes: io.containerd.runc.v2 runc
        Default Runtime: runc
        Init Binary: docker-init
        containerd version: 2456e983eb9e37e47538f59ea18f2043c9a73640
        runc version: v1.1.4-0-g5fd4c4d
        init version: de40ad0
        Security Options:
         apparmor
         seccomp
          Profile: builtin
         cgroupns
        Kernel Version: 5.15.0-1029-oracle
        Operating System: Ubuntu 22.04.2 LTS
        OSType: linux
        Architecture: aarch64
        CPUs: 2
        Total Memory: 11.66GiB
        Name: arm1
        ID: 91a59387-ea2d-472e-888c-e9327bf5feba
        Docker Root Dir: /var/lib/docker
        Debug Mode: false
        Registry: https://index.docker.io/v1/
        Experimental: false
        Insecure Registries:
         127.0.0.0/8
        Live Restore Enabled: false

-----> git version: git version 2.25.1
-----> sigil version: 0.9.0build+bc921b7
-----> herokuish version: 
 !     herokuish not supported on arm64 architecture
-----> dokku version: dokku version 0.30.1
-----> plugn version: plugn: 0.12.0build+3a27594
-----> dokku plugins: 
         00_dokku-standard    0.30.1 enabled    dokku core standard plugin
         20_events            0.30.1 enabled    dokku core events logging plugin
         app-json             0.30.1 enabled    dokku core app-json plugin
         apps                 0.30.1 enabled    dokku core apps plugin
         builder              0.30.1 enabled    dokku core builder plugin
         builder-dockerfile   0.30.1 enabled    dokku core builder-dockerfile plugin
         builder-herokuish    0.30.1 enabled    dokku core builder-herokuish plugin
         builder-lambda       0.30.1 enabled    dokku core builder-lambda plugin
         builder-null         0.30.1 enabled    dokku core builder-null plugin
         builder-pack         0.30.1 enabled    dokku core builder-pack plugin
         buildpacks           0.30.1 enabled    dokku core buildpacks plugin
         caddy-vhosts         0.30.1 enabled    dokku core caddy-vhosts plugin
         certs                0.30.1 enabled    dokku core certificate management plugin
         checks               0.30.1 enabled    dokku core checks plugin
         common               0.30.1 enabled    dokku core common plugin
         config               0.30.1 enabled    dokku core config plugin
         cron                 0.30.1 enabled    dokku core cron plugin
         docker-options       0.30.1 enabled    dokku core docker-options plugin
         domains              0.30.1 enabled    dokku core domains plugin
         enter                0.30.1 enabled    dokku core enter plugin
         git                  0.30.1 enabled    dokku core git plugin
         haproxy-vhosts       0.30.1 enabled    dokku core haproxy-vhosts plugin
         letsencrypt          0.20.0 enabled    Automated installation of let's encrypt TLS certificates
         logs                 0.30.1 enabled    dokku core logs plugin
         mongo                1.32.0 enabled    dokku mongo service plugin
         network              0.30.1 enabled    dokku core network plugin
         nginx-vhosts         0.30.1 enabled    dokku core nginx-vhosts plugin
         plugin               0.30.1 enabled    dokku core plugin plugin
         postgres             1.32.0 enabled    dokku postgres service plugin
         proxy                0.30.1 enabled    dokku core proxy plugin
         ps                   0.30.1 enabled    dokku core ps plugin
         registry             0.30.1 enabled    dokku core registry plugin
         repo                 0.30.1 enabled    dokku core repo plugin
         resource             0.30.1 enabled    dokku core resource plugin
         run                  0.30.1 enabled    dokku core run plugin
         scheduler            0.30.1 enabled    dokku core scheduler plugin
         scheduler-docker-local 0.30.1 enabled    dokku core scheduler-docker-local plugin
         scheduler-null       0.30.1 enabled    dokku core scheduler-null plugin
         shell                0.30.1 enabled    dokku core shell plugin
         ssh-keys             0.30.1 enabled    dokku core ssh-keys plugin
         storage              0.30.1 enabled    dokku core storage plugin
         trace                0.30.1 enabled    dokku core trace plugin
         traefik-vhosts       0.30.1 enabled    dokku core traefik-vhosts plugin
=====> mern-boilerplate app-json information
       App json computed selected:    app.json
       App json global selected:      app.json
       App json selected:             
=====> mern-boilerplate app information
       App created at:                1678004858
       App deploy source:             
       App deploy source metadata:    
       App dir:                       /home/dokku/mern-boilerplate
       App locked:                    false
=====> mern-boilerplate builder information
       Builder build dir:             
       Builder computed build dir:    
       Builder computed selected:     
       Builder global build dir:      
       Builder global selected:       
       Builder selected:              
=====> mern-boilerplate builder-dockerfile information
       Builder dockerfile computed dockerfile path: Dockerfile               
       Builder dockerfile global dockerfile path: Dockerfile               
       Builder dockerfile dockerfile path:                          
=====> mern-boilerplate builder-lambda information
       Builder lambda computed lambdayml path: lambda.yml               
       Builder lambda global lambdayml path: lambda.yml               
       Builder lambda lambdayml path:                          
=====> mern-boilerplate builder-pack information
       Builder pack computed projecttoml path: project.toml             
       Builder pack global projecttoml path: project.toml             
       Builder pack projecttoml path:                          
=====> mern-boilerplate buildpacks information
       Buildpacks computed stack:     gliderlabs/herokuish:latest-20
       Buildpacks global stack:       
       Buildpacks list:               
       Buildpacks stack:              
=====> mern-boilerplate ssl information
       Ssl dir:                       /home/dokku/mern-boilerplate/tls
       Ssl enabled:                   false                    
       Ssl hostnames:                                          
       Ssl expires at:                                         
       Ssl issuer:                                             
       Ssl starts at:                                          
       Ssl subject:                                            
       Ssl verified:                                           
=====> mern-boilerplate checks information
       Checks disabled list:          none                     
       Checks skipped list:           none                     
       Checks computed wait to retire: 60                       
       Checks global wait to retire:  60                       
       Checks wait to retire:                                  
=====> mern-boilerplate cron information
       Cron task count:               0
=====> mern-boilerplate docker options information
       Docker options build:                                   
       Docker options deploy:         --restart=on-failure:10  
       Docker options run:                                     
=====> mern-boilerplate domains information
       Domains app enabled:           true                     
       Domains app vhosts:            mern-boilerplate.dokku.arm1.localhost3002.live
       Domains global enabled:        true                     
       Domains global vhosts:         dokku.arm1.localhost3002.live
=====> mern-boilerplate git information
       Git deploy branch:             master                   
       Git global deploy branch:      master                   
       Git keep git dir:              false                    
       Git rev env var:               GIT_REV                  
       Git sha:                                                
       Git source image:                                       
       Git last updated at:                                    
Can't open /home/dokku/mern-boilerplate/tls/server.crt for reading, No such file or directory
281473808300512:error:02001002:system library:fopen:No such file or directory:../crypto/bio/bss_file.c:69:fopen('/home/dokku/mern-boilerplate/tls/server.crt','r')
281473808300512:error:2006D080:BIO routines:BIO_new_file:no such file:../crypto/bio/bss_file.c:76:
unable to load certificate
=====> mern-boilerplate letsencrypt information
       Letsencrypt active:            false                    
       Letsencrypt autorenew:         false                    
       Letsencrypt computed dns provider:                          
       Letsencrypt global dns provider:                          
       Letsencrypt dns provider:                               
       Letsencrypt computed email:                             
       Letsencrypt global email:                               
       Letsencrypt email:                                      
       Letsencrypt expiration:        1677974400               
       Letsencrypt computed graceperiod: 2592000                  
       Letsencrypt global graceperiod:                          
       Letsencrypt graceperiod:                                
       Letsencrypt computed lego docker args:                          
       Letsencrypt global lego docker args:                          
       Letsencrypt lego docker args:                           
       Letsencrypt computed server:   https://acme-v02.api.letsencrypt.org/directory
       Letsencrypt global server:                              
       Letsencrypt server:                                     
=====> mern-boilerplate logs information
       Logs computed max size:        10m
       Logs global max size:          10m
       Logs global vector sink:       
       Logs max size:                 
       Logs vector sink:              
=====> mern-boilerplate network information
       Network attach post create:           
       Network attach post deploy:           
       Network bind all interfaces:          false
       Network computed attach post create:  
       Network computed attach post deploy:  
       Network computed bind all interfaces: false
       Network computed initial network:     dokku-external
       Network computed tld:                 
       Network global attach post create:    
       Network global attach post deploy:    
       Network global bind all interfaces:   false
       Network global initial network:       dokku-external
       Network global tld:                   
       Network initial network:              
       Network static web listener:          
       Network tld:                          
       Network web listeners:                
=====> mern-boilerplate nginx information
       Nginx access log format:                                
       Nginx access log path:         /var/log/nginx/mern-boilerplate-access.log
       Nginx bind address ipv4:                                
       Nginx bind address ipv6:       ::                       
       Nginx client max body size:                             
       Nginx disable custom config:   false                    
       Nginx error log path:          /var/log/nginx/mern-boilerplate-error.log
       Nginx global hsts:             true                     
       Nginx computed hsts:           true                     
       Nginx hsts:                                             
       Nginx hsts include subdomains: true                     
       Nginx hsts max age:            15724800                 
       Nginx hsts preload:            false                    
       Nginx computed nginx conf sigil path: nginx.conf.sigil         
       Nginx global nginx conf sigil path: nginx.conf.sigil         
       Nginx nginx conf sigil path:                            
       Nginx proxy buffer size:       4096                     
       Nginx proxy buffering:         on                       
       Nginx proxy buffers:           8 4096                   
       Nginx proxy busy buffers size: 8192                     
       Nginx proxy read timeout:      60s                      
       Nginx last visited at:                                  
       Nginx x forwarded for value:   $remote_addr             
       Nginx x forwarded port value:  $server_port             
       Nginx x forwarded proto value: $scheme                  
       Nginx x forwarded ssl:                                  
=====> mern-boilerplate proxy information
       Proxy enabled:                 true
       Proxy port map:                
       Proxy type:                    nginx
=====> mern-boilerplate ps information
       Deployed:                      false
       Processes:                     0
       Ps can scale:                  true
       Ps computed procfile path:     Procfile
       Ps global procfile path:       Procfile
       Ps procfile path:              
       Ps restart policy:             on-failure:10
       Restore:                       true
       Running:                       false
=====> mern-boilerplate registry information
       Registry computed image repo:      dokku/mern-boilerplate
       Registry computed push on release: false
       Registry computed server:          
       Registry global push on release:   
       Registry global server:            
       Registry image repo:               
       Registry push on release:          
       Registry server:                   
       Registry tag version:              
=====> mern-boilerplate resource information
=====> mern-boilerplate scheduler information
       Scheduler computed selected:   docker-local
       Scheduler global selected:     docker-local
       Scheduler selected:            
=====> mern-boilerplate scheduler-docker-local information
       Scheduler docker local disable chown:                          
       Scheduler docker local init process: true                     
       Scheduler docker local parallel schedule count:                          
=====> mern-boilerplate storage information
       Storage build mounts:                                   
       Storage deploy mounts:                                  
       Storage run mounts:                                     
ubuntu@arm1:~/traefik-proxy/apps/dokku$

How (deb/make) and where (AWS, VirtualBox, physical, etc.) was Dokku installed?:

Dokku in Docker container.

Additional information

ubuntu@arm1:~/traefik-proxy/apps/dokku$ dokku nginx:show-config mern-boilerplate
 !     No nginx.conf exists for mern-boilerplate
josegonzalez commented 1 year ago

This works fine on intel machines, so I'm wondering if there is something weird about how the mongodb 5 image works on arm64.

nemanjam commented 1 year ago

Setting image version doesn't seem to work.

ubuntu@arm1:~$ export MONGO_IMAGE_VERSION="4.0"
ubuntu@arm1:~$ echo $MONGO_IMAGE_VERSION
4.0
ubuntu@arm1:~$ dokku mongo:create mbd
       Waiting for container to be ready
MongoServerError: Authentication failed.
ubuntu@arm1:~$ dokku mongo:info mbd
=====> mbd mongo service information
       Config dir:          /var/lib/dokku/services/mongo/mbd/config
       Config options:       --storageEngine wiredTiger --auth 
       Data dir:            /var/lib/dokku/services/mongo/mbd/data
       Dsn:                 mongodb://mbd:970edc7faaf5e6c7c30ec115547aee67@dokku-mongo-mbd:27017/mbd
       Exposed ports:       -                        
       Id:                  829af68ab4e80a1ff2bfbf64c9d1a8df37be85aa0988327a0e9343d697336921
       Internal ip:         172.17.0.2               
       Initial network:                              
       Links:               -                        
       Post create network:                          
       Post start network:                           
       Service root:        /var/lib/dokku/services/mongo/mbd
       Status:              running                  
       Version:             mongo:5.0.14
josegonzalez commented 1 year ago

I wonder if this is an issue with mongo versions lower than 6. I just released a plugin update that allows mongo 6 usage, mind updating your plugin and checking to see if that fixes it?

nemanjam commented 1 year ago

Same thing with 6.0.4.

ubuntu@arm1:~$ echo $MONGO_IMAGE_VERSION

ubuntu@arm1:~$ dokku mongo:create mbd
       Waiting for container to be ready
MongoServerError: Authentication failed.
ubuntu@arm1:~$ dokku mongo:info mbd
=====> mbd mongo service information
       Config dir:          /var/lib/dokku/services/mongo/mbd/config
       Config options:       --storageEngine wiredTiger --auth 
       Data dir:            /var/lib/dokku/services/mongo/mbd/data
       Dsn:                 mongodb://mbd:be27c884b8091c96afda3b8af7356273@dokku-mongo-mbd:27017/mbd
       Exposed ports:       -                        
       Id:                  053c5aa23ef927c3f17a91d6a9fb8612781345435a6ec1ecaf55b23c6e6294c2
       Internal ip:         172.17.0.2               
       Initial network:                              
       Links:               -                        
       Post create network:                          
       Post start network:                           
       Service root:        /var/lib/dokku/services/mongo/mbd
       Status:              running                  
       Version:             mongo:6.0.4              
ubuntu@arm1:~$ 
ubuntu@arm1:~$ dokku plugin:list
  ...
  mongo                1.33.0 enabled    dokku mongo service plugin
josegonzalez commented 1 year ago

Can you run the create command with trace mode on and show the output?

This definitely seems like a weird result for arm64, so I'll need some time to get access to such a host for dokku and see if I can replicate/debug.

nemanjam commented 1 year ago

I can give you ssh keys of my arm server and Dokku container.

ubuntu@arm1:~$ dokku trace:on
-----> Enabling trace mode
ubuntu@arm1:~$ dokku mongo:create mbd
+ export DOKKU_HOST_ROOT=/home/ubuntu/traefik-proxy/apps/dokku/dokku-data/home/dokku
+ DOKKU_HOST_ROOT=/home/ubuntu/traefik-proxy/apps/dokku/dokku-data/home/dokku
+ export DOKKU_DISTRO
++ . /etc/os-release
++ echo ubuntu
+ DOKKU_DISTRO=ubuntu
+ export DOCKER_BIN=/usr/local/bin/docker
+ DOCKER_BIN=/usr/local/bin/docker
+ export DOKKU_IMAGE=gliderlabs/herokuish:latest-20
+ DOKKU_IMAGE=gliderlabs/herokuish:latest-20
+ export DOKKU_CNB_BUILDER=heroku/buildpacks
+ DOKKU_CNB_BUILDER=heroku/buildpacks
+ export DOKKU_LIB_ROOT=/var/lib/dokku
+ DOKKU_LIB_ROOT=/var/lib/dokku
+ export PLUGIN_PATH=/var/lib/dokku/plugins
+ PLUGIN_PATH=/var/lib/dokku/plugins
+ export PLUGIN_AVAILABLE_PATH=/var/lib/dokku/plugins/available
+ PLUGIN_AVAILABLE_PATH=/var/lib/dokku/plugins/available
+ export PLUGIN_ENABLED_PATH=/var/lib/dokku/plugins/enabled
+ PLUGIN_ENABLED_PATH=/var/lib/dokku/plugins/enabled
+ export PLUGIN_CORE_PATH=/var/lib/dokku/core-plugins
+ PLUGIN_CORE_PATH=/var/lib/dokku/core-plugins
+ export PLUGIN_CORE_AVAILABLE_PATH=/var/lib/dokku/core-plugins/available
+ PLUGIN_CORE_AVAILABLE_PATH=/var/lib/dokku/core-plugins/available
+ export PLUGIN_CORE_ENABLED_PATH=/var/lib/dokku/core-plugins/enabled
+ PLUGIN_CORE_ENABLED_PATH=/var/lib/dokku/core-plugins/enabled
+ export DOKKU_SYSTEM_GROUP=dokku
+ DOKKU_SYSTEM_GROUP=dokku
+ export DOKKU_SYSTEM_USER=dokku
+ DOKKU_SYSTEM_USER=dokku
+ export DOKKU_API_VERSION=1
+ DOKKU_API_VERSION=1
+ export DOKKU_NOT_IMPLEMENTED_EXIT=10
+ DOKKU_NOT_IMPLEMENTED_EXIT=10
+ export DOKKU_VALID_EXIT=0
+ DOKKU_VALID_EXIT=0
+ export DOKKU_PID=19134
+ DOKKU_PID=19134
+ export DOKKU_LOGS_DIR=/var/log/dokku
+ DOKKU_LOGS_DIR=/var/log/dokku
+ export DOKKU_LOGS_HOST_DIR=/var/log/dokku
+ DOKKU_LOGS_HOST_DIR=/var/log/dokku
+ export DOKKU_EVENTS_LOGFILE=/var/log/dokku/events.log
+ DOKKU_EVENTS_LOGFILE=/var/log/dokku/events.log
+ export DOKKU_CONTAINER_LABEL=dokku
+ DOKKU_CONTAINER_LABEL=dokku
+ export 'DOKKU_GLOBAL_BUILD_ARGS=--label=org.label-schema.schema-version=1.0 --label=org.label-schema.vendor=dokku --label=dokku'
+ DOKKU_GLOBAL_BUILD_ARGS='--label=org.label-schema.schema-version=1.0 --label=org.label-schema.vendor=dokku --label=dokku'
+ export 'DOKKU_GLOBAL_RUN_ARGS=--label=org.label-schema.schema-version=1.0 --label=org.label-schema.vendor=dokku --label=dokku'
+ DOKKU_GLOBAL_RUN_ARGS='--label=org.label-schema.schema-version=1.0 --label=org.label-schema.vendor=dokku --label=dokku'
+ source /var/lib/dokku/core-plugins/available/common/functions
++ set -eo pipefail
++ [[ -n 1 ]]
++ set -x
+ parse_args mongo:create mbd
+ declare 'desc=top-level cli arg parser'
+ local next_index=1
+ local skip=false
+ args=("$@")
+ local args
+ local flags
+ for arg in "$@"
+ [[ false == \t\r\u\e ]]
+ case "$arg" in
+ [[ false == \t\r\u\e ]]
+ [[ mongo:create == \-\-\a\p\p ]]
+ [[ mongo:create =~ ^--.* ]]
+ next_index=2
+ for arg in "$@"
+ [[ false == \t\r\u\e ]]
+ case "$arg" in
+ [[ false == \t\r\u\e ]]
+ [[ mbd == \-\-\a\p\p ]]
+ [[ mbd =~ ^--.* ]]
+ next_index=3
+ [[ -z '' ]]
++ echo -e ''
++ sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//'
+ export DOKKU_GLOBAL_FLAGS=
+ DOKKU_GLOBAL_FLAGS=
+ return 0
+ args=("$@")
+ skip_arg=false
+ [[ mongo:create =~ ^--.* ]]
+ has_tty
+ declare 'desc=return 0 if we have a tty'
+ [[ '' == \t\r\u\e ]]
++ LC_ALL=C
++ /usr/bin/tty
+ [[ /dev/pts/0 == \n\o\t\ \a\ \t\t\y ]]
+ return 0
++ id -un
+ [[ root != \d\o\k\k\u ]]
+ unset TMP TMPDIR TEMP TEMPDIR
+ [[ ! mongo:create =~ plugin:* ]]
+ [[ mongo:create != \s\s\h\-\k\e\y\s\:\a\d\d ]]
+ [[ mongo:create != \s\s\h\-\k\e\y\s\:\r\e\m\o\v\e ]]
++ id -un
+ export SSH_USER=root
+ SSH_USER=root
+ sudo -u dokku -E -H dokku mongo:create mbd
+ export DOKKU_HOST_ROOT=/home/ubuntu/traefik-proxy/apps/dokku/dokku-data/home/dokku
+ DOKKU_HOST_ROOT=/home/ubuntu/traefik-proxy/apps/dokku/dokku-data/home/dokku
+ export DOKKU_DISTRO
++ . /etc/os-release
++ echo ubuntu
+ DOKKU_DISTRO=ubuntu
+ export DOCKER_BIN=/usr/local/bin/docker
+ DOCKER_BIN=/usr/local/bin/docker
+ export DOKKU_IMAGE=gliderlabs/herokuish:latest-20
+ DOKKU_IMAGE=gliderlabs/herokuish:latest-20
+ export DOKKU_CNB_BUILDER=heroku/buildpacks
+ DOKKU_CNB_BUILDER=heroku/buildpacks
+ export DOKKU_LIB_ROOT=/var/lib/dokku
+ DOKKU_LIB_ROOT=/var/lib/dokku
+ export PLUGIN_PATH=/var/lib/dokku/plugins
+ PLUGIN_PATH=/var/lib/dokku/plugins
+ export PLUGIN_AVAILABLE_PATH=/var/lib/dokku/plugins/available
+ PLUGIN_AVAILABLE_PATH=/var/lib/dokku/plugins/available
+ export PLUGIN_ENABLED_PATH=/var/lib/dokku/plugins/enabled
+ PLUGIN_ENABLED_PATH=/var/lib/dokku/plugins/enabled
+ export PLUGIN_CORE_PATH=/var/lib/dokku/core-plugins
+ PLUGIN_CORE_PATH=/var/lib/dokku/core-plugins
+ export PLUGIN_CORE_AVAILABLE_PATH=/var/lib/dokku/core-plugins/available
+ PLUGIN_CORE_AVAILABLE_PATH=/var/lib/dokku/core-plugins/available
+ export PLUGIN_CORE_ENABLED_PATH=/var/lib/dokku/core-plugins/enabled
+ PLUGIN_CORE_ENABLED_PATH=/var/lib/dokku/core-plugins/enabled
+ export DOKKU_SYSTEM_GROUP=dokku
+ DOKKU_SYSTEM_GROUP=dokku
+ export DOKKU_SYSTEM_USER=dokku
+ DOKKU_SYSTEM_USER=dokku
+ export DOKKU_API_VERSION=1
+ DOKKU_API_VERSION=1
+ export DOKKU_NOT_IMPLEMENTED_EXIT=10
+ DOKKU_NOT_IMPLEMENTED_EXIT=10
+ export DOKKU_VALID_EXIT=0
+ DOKKU_VALID_EXIT=0
+ export DOKKU_PID=19149
+ DOKKU_PID=19149
+ export DOKKU_LOGS_DIR=/var/log/dokku
+ DOKKU_LOGS_DIR=/var/log/dokku
+ export DOKKU_LOGS_HOST_DIR=/var/log/dokku
+ DOKKU_LOGS_HOST_DIR=/var/log/dokku
+ export DOKKU_EVENTS_LOGFILE=/var/log/dokku/events.log
+ DOKKU_EVENTS_LOGFILE=/var/log/dokku/events.log
+ export DOKKU_CONTAINER_LABEL=dokku
+ DOKKU_CONTAINER_LABEL=dokku
+ export 'DOKKU_GLOBAL_BUILD_ARGS=--label=org.label-schema.schema-version=1.0 --label=org.label-schema.vendor=dokku --label=dokku'
+ DOKKU_GLOBAL_BUILD_ARGS='--label=org.label-schema.schema-version=1.0 --label=org.label-schema.vendor=dokku --label=dokku'
+ export 'DOKKU_GLOBAL_RUN_ARGS=--label=org.label-schema.schema-version=1.0 --label=org.label-schema.vendor=dokku --label=dokku'
+ DOKKU_GLOBAL_RUN_ARGS='--label=org.label-schema.schema-version=1.0 --label=org.label-schema.vendor=dokku --label=dokku'
+ source /var/lib/dokku/core-plugins/available/common/functions
++ set -eo pipefail
++ [[ -n 1 ]]
++ set -x
+ parse_args mongo:create mbd
+ declare 'desc=top-level cli arg parser'
+ local next_index=1
+ local skip=false
+ args=("$@")
+ local args
+ local flags
+ for arg in "$@"
+ [[ false == \t\r\u\e ]]
+ case "$arg" in
+ [[ false == \t\r\u\e ]]
+ [[ mongo:create == \-\-\a\p\p ]]
+ [[ mongo:create =~ ^--.* ]]
+ next_index=2
+ for arg in "$@"
+ [[ false == \t\r\u\e ]]
+ case "$arg" in
+ [[ false == \t\r\u\e ]]
+ [[ mbd == \-\-\a\p\p ]]
+ [[ mbd =~ ^--.* ]]
+ next_index=3
+ [[ -z '' ]]
++ echo -e ''
++ sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//'
+ export DOKKU_GLOBAL_FLAGS=
+ DOKKU_GLOBAL_FLAGS=
+ return 0
+ args=("$@")
+ skip_arg=false
+ [[ mongo:create =~ ^--.* ]]
+ has_tty
+ declare 'desc=return 0 if we have a tty'
+ [[ '' == \t\r\u\e ]]
++ LC_ALL=C
++ /usr/bin/tty
+ [[ /dev/pts/0 == \n\o\t\ \a\ \t\t\y ]]
+ return 0
++ id -un
+ [[ dokku != \d\o\k\k\u ]]
+ [[ mongo:create =~ ^plugin:.* ]]
+ [[ mongo:create == \s\s\h\-\k\e\y\s\:\a\d\d ]]
+ [[ mongo:create == \s\s\h\-\k\e\y\s\:\r\e\m\o\v\e ]]
+ [[ -n '' ]]
+ dokku_auth mongo:create mbd
+ declare 'desc=calls user-auth plugin trigger'
+ export SSH_USER=root
+ SSH_USER=root
+ export SSH_NAME=default
+ SSH_NAME=default
+ export DOKKU_COMMAND=mongo:create
+ DOKKU_COMMAND=mongo:create
++ wc -l
++ find /var/lib/dokku/plugins/enabled/20_events/user-auth
+ local user_auth_count=1
+ [[ 1 == 0 ]]
+ [[ 1 == 1 ]]
+ [[ -f /var/lib/dokku/plugins/enabled/20_events/user-auth ]]
+ return 0
+ case "$1" in
+ execute_dokku_cmd mongo:create mbd
+ declare 'desc=executes dokku sub-commands'
+ local PLUGIN_NAME=mongo:create
+ local PLUGIN_CMD=mongo:create
+ local implemented=0
+ local script
+ argv=("$@")
+ local argv
+ case "$PLUGIN_NAME" in
++ readlink -f /var/lib/dokku/plugins/enabled/mongo
+ [[ /var/lib/dokku/plugins/available/mongo == *core-plugins* ]]
+ [[ -x /var/lib/dokku/plugins/enabled/mongo:create/subcommands/default ]]
+ [[ -x /var/lib/dokku/plugins/enabled/mongo:create/subcommands/mongo:create ]]
+ [[ -x /var/lib/dokku/plugins/enabled/mongo/subcommands/create ]]
+ [[ -n create ]]
+ /var/lib/dokku/plugins/enabled/mongo/subcommands/create mongo:create mbd
+ source /var/lib/dokku/core-plugins/available/common/functions
++ set -eo pipefail
++ [[ -n 1 ]]
++ set -x
++++ dirname /var/lib/dokku/plugins/enabled/mongo/subcommands/create
+++ cd /var/lib/dokku/plugins/enabled/mongo/subcommands
+++ pwd
++ dirname /var/lib/dokku/plugins/enabled/mongo/subcommands
+ source /var/lib/dokku/plugins/enabled/mongo/functions
++++ dirname /var/lib/dokku/plugins/enabled/mongo/functions
+++ cd /var/lib/dokku/plugins/enabled/mongo
+++ pwd
++ source /var/lib/dokku/plugins/enabled/mongo/config
+++++ dirname /var/lib/dokku/plugins/enabled/mongo/config
++++ cd /var/lib/dokku/plugins/enabled/mongo
++++ pwd
+++ _DIR=/var/lib/dokku/plugins/enabled/mongo
+++ export MONGO_IMAGE=mongo
+++ MONGO_IMAGE=mongo
+++ export MONGO_IMAGE_VERSION=6.0.4
+++ MONGO_IMAGE_VERSION=6.0.4
+++ export MONGO_ROOT=/var/lib/dokku/services/mongo
+++ MONGO_ROOT=/var/lib/dokku/services/mongo
+++ export DOKKU_LIB_HOST_ROOT=/var/lib/dokku
+++ DOKKU_LIB_HOST_ROOT=/var/lib/dokku
+++ export MONGO_HOST_ROOT=/var/lib/dokku/services/mongo
+++ MONGO_HOST_ROOT=/var/lib/dokku/services/mongo
+++ PLUGIN_UNIMPLEMENTED_SUBCOMMANDS=()
+++ export PLUGIN_UNIMPLEMENTED_SUBCOMMANDS
+++ export PLUGIN_COMMAND_PREFIX=mongo
+++ PLUGIN_COMMAND_PREFIX=mongo
+++ export PLUGIN_CONFIG_ROOT=/var/lib/dokku/config/mongo
+++ PLUGIN_CONFIG_ROOT=/var/lib/dokku/config/mongo
+++ export PLUGIN_DATA_ROOT=/var/lib/dokku/services/mongo
+++ PLUGIN_DATA_ROOT=/var/lib/dokku/services/mongo
+++ export PLUGIN_DATA_HOST_ROOT=/var/lib/dokku/services/mongo
+++ PLUGIN_DATA_HOST_ROOT=/var/lib/dokku/services/mongo
+++ PLUGIN_DATASTORE_PORTS=(27017 27018 27019 28017)
+++ export PLUGIN_DATASTORE_PORTS
+++ export PLUGIN_DATASTORE_WAIT_PORT=27017
+++ PLUGIN_DATASTORE_WAIT_PORT=27017
+++ export PLUGIN_DEFAULT_ALIAS=MONGO
+++ PLUGIN_DEFAULT_ALIAS=MONGO
+++ export PLUGIN_DISABLE_PULL=
+++ PLUGIN_DISABLE_PULL=
+++ export PLUGIN_DISABLE_PULL_VARIABLE=MONGO_DISABLE_PULL
+++ PLUGIN_DISABLE_PULL_VARIABLE=MONGO_DISABLE_PULL
+++ export PLUGIN_ALT_ALIAS=DOKKU_MONGO
+++ PLUGIN_ALT_ALIAS=DOKKU_MONGO
+++ export PLUGIN_IMAGE=mongo
+++ PLUGIN_IMAGE=mongo
+++ export PLUGIN_IMAGE_VERSION=6.0.4
+++ PLUGIN_IMAGE_VERSION=6.0.4
+++ export PLUGIN_SCHEME=mongodb
+++ PLUGIN_SCHEME=mongodb
+++ export PLUGIN_SERVICE=MongoDB
+++ PLUGIN_SERVICE=MongoDB
+++ export PLUGIN_VARIABLE=MONGO
+++ PLUGIN_VARIABLE=MONGO
+++ export PLUGIN_BASE_PATH=/var/lib/dokku/plugins
+++ PLUGIN_BASE_PATH=/var/lib/dokku/plugins
+++ export PLUGIN_CONFIG_SUFFIX=config
+++ PLUGIN_CONFIG_SUFFIX=config
+++ [[ -n 1 ]]
+++ export PLUGIN_BASE_PATH=/var/lib/dokku/plugins/enabled
+++ PLUGIN_BASE_PATH=/var/lib/dokku/plugins/enabled
+++ export PLUGIN_BUSYBOX_IMAGE=busybox:1.34.1-uclibc
+++ PLUGIN_BUSYBOX_IMAGE=busybox:1.34.1-uclibc
+++ export PLUGIN_AMBASSADOR_IMAGE=dokku/ambassador:0.5.0
+++ PLUGIN_AMBASSADOR_IMAGE=dokku/ambassador:0.5.0
+++ export PLUGIN_S3BACKUP_IMAGE=dokku/s3backup:0.14.0
+++ PLUGIN_S3BACKUP_IMAGE=dokku/s3backup:0.14.0
+++ export PLUGIN_WAIT_IMAGE=dokku/wait:0.6.0
+++ PLUGIN_WAIT_IMAGE=dokku/wait:0.6.0
+++ export 'MONGO_CONFIG_OPTIONS= --storageEngine wiredTiger --auth '
+++ MONGO_CONFIG_OPTIONS=' --storageEngine wiredTiger --auth '
++ set -eo pipefail
++ [[ -n 1 ]]
++ set -x
++++ dirname /var/lib/dokku/plugins/enabled/mongo/functions
+++ cd /var/lib/dokku/plugins/enabled/mongo
+++ pwd
++ source /var/lib/dokku/plugins/enabled/mongo/common-functions
+++++ dirname /var/lib/dokku/plugins/enabled/mongo/common-functions
++++ cd /var/lib/dokku/plugins/enabled/mongo
++++ pwd
+++ source /var/lib/dokku/plugins/enabled/mongo/config
++++++ dirname /var/lib/dokku/plugins/enabled/mongo/config
+++++ cd /var/lib/dokku/plugins/enabled/mongo
+++++ pwd
++++ _DIR=/var/lib/dokku/plugins/enabled/mongo
++++ export MONGO_IMAGE=mongo
++++ MONGO_IMAGE=mongo
++++ export MONGO_IMAGE_VERSION=6.0.4
++++ MONGO_IMAGE_VERSION=6.0.4
++++ export MONGO_ROOT=/var/lib/dokku/services/mongo
++++ MONGO_ROOT=/var/lib/dokku/services/mongo
++++ export DOKKU_LIB_HOST_ROOT=/var/lib/dokku
++++ DOKKU_LIB_HOST_ROOT=/var/lib/dokku
++++ export MONGO_HOST_ROOT=/var/lib/dokku/services/mongo
++++ MONGO_HOST_ROOT=/var/lib/dokku/services/mongo
++++ PLUGIN_UNIMPLEMENTED_SUBCOMMANDS=()
++++ export PLUGIN_UNIMPLEMENTED_SUBCOMMANDS
++++ export PLUGIN_COMMAND_PREFIX=mongo
++++ PLUGIN_COMMAND_PREFIX=mongo
++++ export PLUGIN_CONFIG_ROOT=/var/lib/dokku/config/mongo
++++ PLUGIN_CONFIG_ROOT=/var/lib/dokku/config/mongo
++++ export PLUGIN_DATA_ROOT=/var/lib/dokku/services/mongo
++++ PLUGIN_DATA_ROOT=/var/lib/dokku/services/mongo
++++ export PLUGIN_DATA_HOST_ROOT=/var/lib/dokku/services/mongo
++++ PLUGIN_DATA_HOST_ROOT=/var/lib/dokku/services/mongo
++++ PLUGIN_DATASTORE_PORTS=(27017 27018 27019 28017)
++++ export PLUGIN_DATASTORE_PORTS
++++ export PLUGIN_DATASTORE_WAIT_PORT=27017
++++ PLUGIN_DATASTORE_WAIT_PORT=27017
++++ export PLUGIN_DEFAULT_ALIAS=MONGO
++++ PLUGIN_DEFAULT_ALIAS=MONGO
++++ export PLUGIN_DISABLE_PULL=
++++ PLUGIN_DISABLE_PULL=
++++ export PLUGIN_DISABLE_PULL_VARIABLE=MONGO_DISABLE_PULL
++++ PLUGIN_DISABLE_PULL_VARIABLE=MONGO_DISABLE_PULL
++++ export PLUGIN_ALT_ALIAS=DOKKU_MONGO
++++ PLUGIN_ALT_ALIAS=DOKKU_MONGO
++++ export PLUGIN_IMAGE=mongo
++++ PLUGIN_IMAGE=mongo
++++ export PLUGIN_IMAGE_VERSION=6.0.4
++++ PLUGIN_IMAGE_VERSION=6.0.4
++++ export PLUGIN_SCHEME=mongodb
++++ PLUGIN_SCHEME=mongodb
++++ export PLUGIN_SERVICE=MongoDB
++++ PLUGIN_SERVICE=MongoDB
++++ export PLUGIN_VARIABLE=MONGO
++++ PLUGIN_VARIABLE=MONGO
++++ export PLUGIN_BASE_PATH=/var/lib/dokku/plugins
++++ PLUGIN_BASE_PATH=/var/lib/dokku/plugins
++++ export PLUGIN_CONFIG_SUFFIX=config
++++ PLUGIN_CONFIG_SUFFIX=config
++++ [[ -n 1 ]]
++++ export PLUGIN_BASE_PATH=/var/lib/dokku/plugins/enabled
++++ PLUGIN_BASE_PATH=/var/lib/dokku/plugins/enabled
++++ export PLUGIN_BUSYBOX_IMAGE=busybox:1.34.1-uclibc
++++ PLUGIN_BUSYBOX_IMAGE=busybox:1.34.1-uclibc
++++ export PLUGIN_AMBASSADOR_IMAGE=dokku/ambassador:0.5.0
++++ PLUGIN_AMBASSADOR_IMAGE=dokku/ambassador:0.5.0
++++ export PLUGIN_S3BACKUP_IMAGE=dokku/s3backup:0.14.0
++++ PLUGIN_S3BACKUP_IMAGE=dokku/s3backup:0.14.0
++++ export PLUGIN_WAIT_IMAGE=dokku/wait:0.6.0
++++ PLUGIN_WAIT_IMAGE=dokku/wait:0.6.0
++++ export 'MONGO_CONFIG_OPTIONS= --storageEngine wiredTiger --auth '
++++ MONGO_CONFIG_OPTIONS=' --storageEngine wiredTiger --auth '
+++ set -eo pipefail
+++ [[ -n 1 ]]
+++ set -x
+++ source /var/lib/dokku/core-plugins/available/common/property-functions
++++ set -eo pipefail
++++ [[ -n 1 ]]
++++ set -x
+++ source /var/lib/dokku/plugins/available/config/functions
++++ set -eo pipefail
++++ [[ -n 1 ]]
++++ set -x
++++ source /var/lib/dokku/core-plugins/available/common/functions
+++++ set -eo pipefail
+++++ [[ -n 1 ]]
+++++ set -x
++ source /var/lib/dokku/core-plugins/available/common/functions
+++ set -eo pipefail
+++ [[ -n 1 ]]
+++ set -x
++ source /var/lib/dokku/core-plugins/available/common/property-functions
+++ set -eo pipefail
+++ [[ -n 1 ]]
+++ set -x
++ source /var/lib/dokku/plugins/available/config/functions
+++ set -eo pipefail
+++ [[ -n 1 ]]
+++ set -x
+++ source /var/lib/dokku/core-plugins/available/common/functions
++++ set -eo pipefail
++++ [[ -n 1 ]]
++++ set -x
++ [[ -f /var/lib/dokku/plugins/available/docker-options/functions ]]
++ source /var/lib/dokku/plugins/available/docker-options/functions
+++ set -eo pipefail
+++ [[ -n 1 ]]
+++ set -x
+++ source /var/lib/dokku/core-plugins/available/common/functions
++++ set -eo pipefail
++++ [[ -n 1 ]]
++++ set -x
+++ AVAILABLE_PHASES=(build deploy run)
+ service-create-cmd mongo:create mbd
+ declare 'desc=create a MongoDB service'
+ argv=("$@")
+ local cmd=mongo:create argv
+ [[ mongo:create == \m\o\n\g\o\:\c\r\e\a\t\e ]]
+ shift 1
+ CREATE_FLAGS_LIST=("${@:2}")
+ declare SERVICE=mbd CREATE_FLAGS_LIST
+ service_create mbd
+ local SERVICE=mbd
+ is_valid_service_name mbd
+ declare 'desc=validate a service name'
+ declare SERVICE=mbd
+ [[ -z mbd ]]
+ [[ mbd =~ ^[A-Za-z0-9_-]+$ ]]
+ return 0
+ [[ -z mbd ]]
+ [[ ! -d /var/lib/dokku/services/mongo/mbd ]]
+ SERVICE_ROOT=/var/lib/dokku/services/mongo/mbd
+ LINKS_FILE=/var/lib/dokku/services/mongo/mbd/LINKS
+ service_parse_args
+ declare 'desc=cli arg parser'
+ local next_index=1
+ local skip=false
+ args=("$@")
+ local args
+ OPTIND=1
+ getopts na:c:C:d:i:I:m:n:N:p:P:q:R:r:s:S:u: opt
+ shift 0
+ service_image_exists mbd
+ declare 'desc=check if the current image exists'
+ declare SERVICE=mbd PLUGIN_IMAGE=mongo PLUGIN_IMAGE_VERSION=6.0.4
+ local plugin_image=mongo
+ local plugin_image_version=6.0.4
+ [[ -z mongo ]]
+ [[ -z 6.0.4 ]]
+ local IMAGE=mongo:6.0.4
++ /usr/local/bin/docker image ls -q mongo:6.0.4
+ [[ bd066eec4864 == '' ]]
+ return 0
+ plugn trigger service-action pre-create mongo mbd
+ mkdir -p /var/lib/dokku/services/mongo/mbd
+ mkdir -p /var/lib/dokku/services/mongo/mbd/data
+ mkdir -p /var/lib/dokku/services/mongo/mbd/config
+ touch /var/lib/dokku/services/mongo/mbd/LINKS
++ openssl rand -hex 16
+ ROOTPASSWORD=6d787d3bde975114d3cfdeebdac711a4
++ openssl rand -hex 16
+ PASSWORD=6df28897722299b2208e0fcb1cc6ae74
+ [[ -n '' ]]
+ [[ -n '' ]]
+ echo 6d787d3bde975114d3cfdeebdac711a4
+ echo 6df28897722299b2208e0fcb1cc6ae74
+ chmod 640 /var/lib/dokku/services/mongo/mbd/ROOTPASSWORD /var/lib/dokku/services/mongo/mbd/PASSWORD
+ service_commit_config mbd
+ declare SERVICE=mbd
+ local SERVICE_ROOT=/var/lib/dokku/services/mongo/mbd
+ local CONFIG_VARIABLE=MONGO_CONFIG_OPTIONS
+ local ENV_VARIABLE=MONGO_CUSTOM_ENV
+ custom_env=
+ [[ -n '' ]]
+ [[ -n '' ]]
+ echo ''
+ config_options=' --storageEngine wiredTiger --auth '
+ [[ -n '' ]]
+ [[ -n  --storageEngine wiredTiger --auth  ]]
+ echo ' --storageEngine wiredTiger --auth '
+ [[ -n '' ]]
+ [[ -n '' ]]
+ [[ -n mongo ]]
+ echo mongo
+ [[ -n 6.0.4 ]]
+ echo 6.0.4
+ [[ -n '' ]]
+ [[ -n '' ]]
+ [[ -n '' ]]
+ write_database_name mbd
+ declare 'desc=write a sanitized database name'
+ declare SERVICE=mbd
+ local SERVICE_ROOT=/var/lib/dokku/services/mongo/mbd
+ echo mbd
+ tr .- _
+ plugn trigger service-action post-create mongo mbd
+ service_create_container mbd
+ local SERVICE=mbd
+ local SERVICE_ROOT=/var/lib/dokku/services/mongo/mbd
+ local SERVICE_HOST_ROOT=/var/lib/dokku/services/mongo/mbd
++ get_service_name mbd
++ declare 'desc=retrieve a docker service label'
++ declare SERVICE=mbd
++ echo dokku.mongo.mbd
+ local SERVICE_NAME=dokku.mongo.mbd
++ service_root_password mbd
++ declare 'desc=fetch the root password for a service'
++ declare SERVICE=mbd
++ local SERVICE_ROOT=/var/lib/dokku/services/mongo/mbd
++ local PASSWORD_FILE=/var/lib/dokku/services/mongo/mbd/ROOTPASSWORD
++ [[ -f /var/lib/dokku/services/mongo/mbd/ROOTPASSWORD ]]
++ cat /var/lib/dokku/services/mongo/mbd/ROOTPASSWORD
+ local ROOTPASSWORD=6d787d3bde975114d3cfdeebdac711a4
++ service_password mbd
++ declare 'desc=fetch the password for a service'
++ declare SERVICE=mbd
++ local SERVICE_ROOT=/var/lib/dokku/services/mongo/mbd
++ local PASSWORD_FILE=/var/lib/dokku/services/mongo/mbd/PASSWORD
++ [[ -f /var/lib/dokku/services/mongo/mbd/PASSWORD ]]
++ cat /var/lib/dokku/services/mongo/mbd/PASSWORD
+ local PASSWORD=6df28897722299b2208e0fcb1cc6ae74
++ get_database_name mbd
++ declare 'desc=retrieve a sanitized database name'
++ declare SERVICE=mbd
++ local SERVICE_ROOT=/var/lib/dokku/services/mongo/mbd
++ [[ ! -f /var/lib/dokku/services/mongo/mbd/DATABASE_NAME ]]
++ cat /var/lib/dokku/services/mongo/mbd/DATABASE_NAME
+ local DATABASE_NAME=mbd
+ [[ -f /var/lib/dokku/services/mongo/mbd/CONFIG_OPTIONS ]]
++ cat /var/lib/dokku/services/mongo/mbd/CONFIG_OPTIONS
+ export 'CONFIG_OPTIONS= --storageEngine wiredTiger --auth '
+ CONFIG_OPTIONS=' --storageEngine wiredTiger --auth '
++ service_dns_hostname mbd
++ declare 'desc=retrieve the alias of a service'
++ declare SERVICE=mbd
+++ get_service_name mbd
+++ declare 'desc=retrieve a docker service label'
+++ declare SERVICE=mbd
+++ echo dokku.mongo.mbd
++ local SERVICE_NAME=dokku.mongo.mbd
++ echo dokku.mongo.mbd
++ tr ._ -
+ local network_alias=dokku-mongo-mbd
+ rm -f /var/lib/dokku/services/mongo/mbd/ID
+ declare -a DOCKER_ARGS
+ DOCKER_ARGS=()
+ DOCKER_ARGS+=("--cidfile=$SERVICE_ROOT/ID")
+ DOCKER_ARGS+=("--env-file=$SERVICE_ROOT/ENV")
+ DOCKER_ARGS+=("--env=MONGO_INITDB_DATABASE=$DATABASE_NAME")
+ DOCKER_ARGS+=("--env=MONGO_INITDB_ROOT_USERNAME=admin")
+ DOCKER_ARGS+=("--env=MONGO_INITDB_ROOT_PASSWORD=$ROOTPASSWORD")
+ DOCKER_ARGS+=("--hostname=$SERVICE_NAME")
+ DOCKER_ARGS+=("--label=dokku.service=$PLUGIN_COMMAND_PREFIX")
+ DOCKER_ARGS+=("--label=dokku=service")
+ DOCKER_ARGS+=("--name=$SERVICE_NAME")
+ DOCKER_ARGS+=("--restart=always")
+ DOCKER_ARGS+=("--volume=$SERVICE_HOST_ROOT/data:/data/db")
+ declare -a LINK_CONTAINER_DOCKER_ARGS
+ LINK_CONTAINER_DOCKER_ARGS=()
+ LINK_CONTAINER_DOCKER_ARGS+=("--rm")
+ LINK_CONTAINER_DOCKER_ARGS+=("--link")
+ LINK_CONTAINER_DOCKER_ARGS+=("$SERVICE_NAME:$network_alias")
+ [[ -f /var/lib/dokku/services/mongo/mbd/SERVICE_MEMORY ]]
+ [[ -n '' ]]
+ [[ -f /var/lib/dokku/services/mongo/mbd/SHM_SIZE ]]
+ [[ -n '' ]]
+ [[ -f /var/lib/dokku/services/mongo/mbd/IMAGE ]]
++ cat /var/lib/dokku/services/mongo/mbd/IMAGE
+ PLUGIN_IMAGE=mongo
+ [[ -f /var/lib/dokku/services/mongo/mbd/IMAGE_VERSION ]]
++ cat /var/lib/dokku/services/mongo/mbd/IMAGE_VERSION
+ PLUGIN_IMAGE_VERSION=6.0.4
++ fn-plugin-property-get mongo mbd initial-network
++ declare 'desc=returns the value for a given property'
++ declare PLUGIN=mongo APP=mbd KEY=initial-network DEFAULT=
++ fn-plugin-property-get-default mongo mbd initial-network ''
++ declare 'desc=returns the value for a given property with a specified default value'
++ declare PLUGIN=mongo APP=mbd KEY=initial-network DEFAULT=
++ /var/lib/dokku/core-plugins/available/common/prop get-with-default mongo mbd initial-network ''
+ local network=
+ [[ -n '' ]]
+ suppress_output /usr/local/bin/docker container create --cidfile=/var/lib/dokku/services/mongo/mbd/ID --env-file=/var/lib/dokku/services/mongo/mbd/ENV --env=MONGO_INITDB_DATABASE=mbd --env=MONGO_INITDB_ROOT_USERNAME=admin --env=MONGO_INITDB_ROOT_PASSWORD=6d787d3bde975114d3cfdeebdac711a4 --hostname=dokku.mongo.mbd --label=dokku.service=mongo --label=dokku=service --name=dokku.mongo.mbd --restart=always --volume=/var/lib/dokku/services/mongo/mbd/data:/data/db mongo:6.0.4 mongod --storageEngine wiredTiger --auth
+ declare 'desc=suppress all output from a given command unless there is an error'
+ local TMP_COMMAND_OUTPUT
++ mktemp /tmp/dokku-19149-suppress_output.XXXXXX
+ TMP_COMMAND_OUTPUT=/tmp/dokku-19149-suppress_output.LNzBzU
+ trap 'rm -rf '\''/tmp/dokku-19149-suppress_output.LNzBzU'\'' >/dev/null' RETURN
+ /usr/local/bin/docker container create --cidfile=/var/lib/dokku/services/mongo/mbd/ID --env-file=/var/lib/dokku/services/mongo/mbd/ENV --env=MONGO_INITDB_DATABASE=mbd --env=MONGO_INITDB_ROOT_USERNAME=admin --env=MONGO_INITDB_ROOT_PASSWORD=6d787d3bde975114d3cfdeebdac711a4 --hostname=dokku.mongo.mbd --label=dokku.service=mongo --label=dokku=service --name=dokku.mongo.mbd --restart=always --volume=/var/lib/dokku/services/mongo/mbd/data:/data/db mongo:6.0.4 mongod --storageEngine wiredTiger --auth
+ return 0
++ rm -rf /tmp/dokku-19149-suppress_output.LNzBzU
++ fn-plugin-property-get mongo mbd post-create-network
++ declare 'desc=returns the value for a given property'
++ declare PLUGIN=mongo APP=mbd KEY=post-create-network DEFAULT=
++ fn-plugin-property-get-default mongo mbd post-create-network ''
++ declare 'desc=returns the value for a given property with a specified default value'
++ declare PLUGIN=mongo APP=mbd KEY=post-create-network DEFAULT=
++ /var/lib/dokku/core-plugins/available/common/prop get-with-default mongo mbd post-create-network ''
+ [[ -n '' ]]
++ cat /var/lib/dokku/services/mongo/mbd/ID
+ suppress_output /usr/local/bin/docker container start 21c251fff0ee1cf5c52d1b61173c9278c401a8f4f7e7d0351650ebe1764aab69
+ declare 'desc=suppress all output from a given command unless there is an error'
+ local TMP_COMMAND_OUTPUT
++ mktemp /tmp/dokku-19149-suppress_output.XXXXXX
+ TMP_COMMAND_OUTPUT=/tmp/dokku-19149-suppress_output.5btLzA
+ trap 'rm -rf '\''/tmp/dokku-19149-suppress_output.5btLzA'\'' >/dev/null' RETURN
+ /usr/local/bin/docker container start 21c251fff0ee1cf5c52d1b61173c9278c401a8f4f7e7d0351650ebe1764aab69
+ return 0
++ rm -rf /tmp/dokku-19149-suppress_output.5btLzA
+ service_port_reconcile_status mbd
+ declare SERVICE=mbd
+ local SERVICE_ROOT=/var/lib/dokku/services/mongo/mbd
+ local PORT_FILE=/var/lib/dokku/services/mongo/mbd/PORT
++ get_service_name mbd
++ declare 'desc=retrieve a docker service label'
++ declare SERVICE=mbd
++ echo dokku.mongo.mbd
+ local SERVICE_NAME=dokku.mongo.mbd
+ local EXPOSED_NAME=dokku.mongo.mbd.ambassador
+ [[ ! -s /var/lib/dokku/services/mongo/mbd/PORT ]]
+ /usr/local/bin/docker container inspect dokku.mongo.mbd.ambassador
+ return
++ fn-plugin-property-get mongo mbd post-start-network
++ declare 'desc=returns the value for a given property'
++ declare PLUGIN=mongo APP=mbd KEY=post-start-network DEFAULT=
++ fn-plugin-property-get-default mongo mbd post-start-network ''
++ declare 'desc=returns the value for a given property with a specified default value'
++ declare PLUGIN=mongo APP=mbd KEY=post-start-network DEFAULT=
++ /var/lib/dokku/core-plugins/available/common/prop get-with-default mongo mbd post-start-network ''
+ [[ -n '' ]]
+ dokku_log_verbose_quiet 'Waiting for container to be ready'
+ declare 'desc=log verbose formatter (with quiet option)'
+ [[ -z '' ]]
+ echo '       Waiting for container to be ready'
       Waiting for container to be ready
+ suppress_output /usr/local/bin/docker container run --rm --link dokku.mongo.mbd:dokku-mongo-mbd dokku/wait:0.6.0 -c dokku-mongo-mbd:27017
+ declare 'desc=suppress all output from a given command unless there is an error'
+ local TMP_COMMAND_OUTPUT
++ mktemp /tmp/dokku-19149-suppress_output.XXXXXX
+ TMP_COMMAND_OUTPUT=/tmp/dokku-19149-suppress_output.R8pQ8J
+ trap 'rm -rf '\''/tmp/dokku-19149-suppress_output.R8pQ8J'\'' >/dev/null' RETURN
+ /usr/local/bin/docker container run --rm --link dokku.mongo.mbd:dokku-mongo-mbd dokku/wait:0.6.0 -c dokku-mongo-mbd:27017
+ return 0
++ rm -rf /tmp/dokku-19149-suppress_output.R8pQ8J
+ /usr/local/bin/docker container exec -i dokku.mongo.mbd mongosh -u admin -p 6d787d3bde975114d3cfdeebdac711a4 --authenticationDatabase admin mbd --eval 'if (db.getUser('\''mbd'\'') == null) { db.createUser({user:'\''mbd'\'',pwd:'\''6df28897722299b2208e0fcb1cc6ae74'\'',roles:[{role:'\''readWrite'\'',db:'\''mbd'\''}]}) }'
MongoServerError: Authentication failed.
ubuntu@arm1:~$
josegonzalez commented 1 year ago

Can you post the full logs of the mongodb container?

nemanjam commented 1 year ago 
t={"$date":"2023-03-05T13:08:29.558+00:00"} s=I c=NETWORK id=4915701 ctx=- msg=Initialized wire specification attr={"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":17},"outgoing":{"minWireVersion":6,"maxWireVersion":17},"isInternalClient":true}}

t={"$date":"2023-03-05T13:08:29.561+00:00"} s=I c=CONTROL id=23285 ctx=main msg=Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'

t={"$date":"2023-03-05T13:08:29.561+00:00"} s=I c=NETWORK id=4648601 ctx=main msg=Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize.

t={"$date":"2023-03-05T13:08:29.562+00:00"} s=I c=REPL id=5123008 ctx=main msg=Successfully registered PrimaryOnlyService attr={"service":"TenantMigrationDonorService","namespace":"config.tenantMigrationDonors"}

t={"$date":"2023-03-05T13:08:29.563+00:00"} s=I c=REPL id=5123008 ctx=main msg=Successfully registered PrimaryOnlyService attr={"service":"TenantMigrationRecipientService","namespace":"config.tenantMigrationRecipients"}

t={"$date":"2023-03-05T13:08:29.563+00:00"} s=I c=REPL id=5123008 ctx=main msg=Successfully registered PrimaryOnlyService attr={"service":"ShardSplitDonorService","namespace":"config.tenantSplitDonors"}

t={"$date":"2023-03-05T13:08:29.563+00:00"} s=I c=CONTROL id=5945603 ctx=main msg=Multi threading initialized

t={"$date":"2023-03-05T13:08:29.563+00:00"} s=I c=CONTROL id=4615611 ctx=initandlisten msg=MongoDB starting attr={"pid":1,"port":27017,"dbPath":"/data/db","architecture":"64-bit","host":"dokku.mongo.mbd"}

t={"$date":"2023-03-05T13:08:29.563+00:00"} s=I c=CONTROL id=23403 ctx=initandlisten msg=Build Info attr={"buildInfo":{"version":"6.0.4","gitVersion":"44ff59461c1353638a71e710f385a566bcd2f547","openSSLVersion":"OpenSSL 3.0.2 15 Mar 2022","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2204","distarch":"aarch64","target_arch":"aarch64"}}}

t={"$date":"2023-03-05T13:08:29.563+00:00"} s=I c=CONTROL id=51765 ctx=initandlisten msg=Operating System attr={"os":{"name":"Ubuntu","version":"22.04"}}

t={"$date":"2023-03-05T13:08:29.563+00:00"} s=I c=CONTROL id=21951 ctx=initandlisten msg=Options set by command line attr={"options":{"net":{"bindIp":"*"},"security":{"authorization":"enabled"},"storage":{"engine":"wiredTiger"}}}

t={"$date":"2023-03-05T13:08:29.564+00:00"} s=I c=STORAGE id=22297 ctx=initandlisten msg=Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem tags=["startupWarnings"]

t={"$date":"2023-03-05T13:08:29.565+00:00"} s=I c=STORAGE id=22315 ctx=initandlisten msg=Opening WiredTiger attr={"config":"create,cache_size=5458M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,remove=true,path=journal,compressor=snappy),builtin_extension_config=(zstd=(compression_level=6)),file_manager=(close_idle_time=600,close_scan_interval=10,close_handle_minimum=2000),statistics_log=(wait=0),json_output=(error,message),verbose=[recovery_progress:1,checkpoint_progress:1,compact_progress:1,backup:0,checkpoint:0,compact:0,evict:0,history_store:0,recovery:0,rts:0,salvage:0,tiered:0,timestamp:0,transaction:0,verify:0,log:0],"}

t={"$date":"2023-03-05T13:08:30.625+00:00"} s=I c=STORAGE id=4795906 ctx=initandlisten msg=WiredTiger opened attr={"durationMillis":1060}

t={"$date":"2023-03-05T13:08:30.625+00:00"} s=I c=RECOVERY id=23987 ctx=initandlisten msg=WiredTiger recoveryTimestamp attr={"recoveryTimestamp":{"$timestamp":{"t":0,"i":0}}}

t={"$date":"2023-03-05T13:08:30.637+00:00"} s=W c=CONTROL id=5123300 ctx=initandlisten msg=vm.max_map_count is too low attr={"currentValue":65530,"recommendedMinimum":1677720,"maxConns":838860} tags=["startupWarnings"]

t={"$date":"2023-03-05T13:08:30.640+00:00"} s=I c=NETWORK id=4915702 ctx=initandlisten msg=Updated wire specification attr={"oldSpec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":17},"outgoing":{"minWireVersion":6,"maxWireVersion":17},"isInternalClient":true},"newSpec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":13,"maxWireVersion":17},"outgoing":{"minWireVersion":13,"maxWireVersion":17},"isInternalClient":true}}

t={"$date":"2023-03-05T13:08:30.640+00:00"} s=I c=REPL id=5853300 ctx=initandlisten msg=current featureCompatibilityVersion value attr={"featureCompatibilityVersion":"5.0","context":"startup"}

t={"$date":"2023-03-05T13:08:30.640+00:00"} s=I c=STORAGE id=5071100 ctx=initandlisten msg=Clearing temp directory

t={"$date":"2023-03-05T13:08:30.641+00:00"} s=I c=CONTROL id=20536 ctx=initandlisten msg=Flow Control is enabled on this deployment

t={"$date":"2023-03-05T13:08:30.642+00:00"} s=I c=FTDC id=20625 ctx=initandlisten msg=Initializing full-time diagnostic data capture attr={"dataDirectory":"/data/db/diagnostic.data"}

t={"$date":"2023-03-05T13:08:30.646+00:00"} s=I c=REPL id=6015317 ctx=initandlisten msg=Setting new configuration state attr={"newState":"ConfigReplicationDisabled","oldState":"ConfigPreStart"}

t={"$date":"2023-03-05T13:08:30.646+00:00"} s=I c=STORAGE id=22262 ctx=initandlisten msg=Timestamp monitor starting

t={"$date":"2023-03-05T13:08:30.647+00:00"} s=I c=NETWORK id=23015 ctx=listener msg=Listening on attr={"address":"/tmp/mongodb-27017.sock"}

t={"$date":"2023-03-05T13:08:30.647+00:00"} s=I c=NETWORK id=23015 ctx=listener msg=Listening on attr={"address":"0.0.0.0"}

t={"$date":"2023-03-05T13:08:30.648+00:00"} s=I c=NETWORK id=23016 ctx=listener msg=Waiting for connections attr={"port":27017,"ssl":"off"}

t={"$date":"2023-03-05T13:08:31.004+00:00"} s=I c=NETWORK id=22943 ctx=listener msg=Connection accepted attr={"remote":"172.17.0.3:33862","uuid":"1d3cde55-45da-4023-a9a3-8ac7d944bb17","connectionId":1,"connectionCount":1}

t={"$date":"2023-03-05T13:08:31.007+00:00"} s=I c=NETWORK id=22944 ctx=conn1 msg=Connection ended attr={"remote":"172.17.0.3:33862","uuid":"1d3cde55-45da-4023-a9a3-8ac7d944bb17","connectionId":1,"connectionCount":0}

t={"$date":"2023-03-05T13:08:31.789+00:00"} s=I c=NETWORK id=22943 ctx=listener msg=Connection accepted attr={"remote":"127.0.0.1:51410","uuid":"004d7cad-e19e-4ecb-8d22-1d80ed3b29cd","connectionId":2,"connectionCount":1}

t={"$date":"2023-03-05T13:08:31.796+00:00"} s=I c=NETWORK id=51800 ctx=conn2 msg=client metadata attr={"remote":"127.0.0.1:51410","client":"conn2","doc":{"driver":{"name":"nodejs|mongosh","version":"5.1.0"},"os":{"type":"Linux","name":"linux","architecture":"arm64","version":"5.15.0-1029-oracle"},"platform":"Node.js v16.19.1, LE (unified)","version":"5.1.0|1.8.0","application":{"name":"mongosh 1.8.0"}}}

t={"$date":"2023-03-05T13:08:31.802+00:00"} s=I c=NETWORK id=22943 ctx=listener msg=Connection accepted attr={"remote":"127.0.0.1:51424","uuid":"cbecd187-8774-4dbd-a534-26774764a335","connectionId":3,"connectionCount":2}

t={"$date":"2023-03-05T13:08:31.805+00:00"} s=I c=NETWORK id=51800 ctx=conn3 msg=client metadata attr={"remote":"127.0.0.1:51424","client":"conn3","doc":{"driver":{"name":"nodejs|mongosh","version":"5.1.0"},"os":{"type":"Linux","name":"linux","architecture":"arm64","version":"5.15.0-1029-oracle"},"platform":"Node.js v16.19.1, LE (unified)","version":"5.1.0|1.8.0","application":{"name":"mongosh 1.8.0"}}}

t={"$date":"2023-03-05T13:08:31.813+00:00"} s=I c=ACCESS id=20249 ctx=conn3 msg=Authentication failed attr={"mechanism":"SCRAM-SHA-256","speculative":true,"principalName":"admin","authenticationDatabase":"admin","remote":"127.0.0.1:51424","extraInfo":{},"error":"AuthenticationFailed: SCRAM authentication failed, storedKey mismatch"}

t={"$date":"2023-03-05T13:08:31.817+00:00"} s=I c=NETWORK id=22944 ctx=conn2 msg=Connection ended attr={"remote":"127.0.0.1:51410","uuid":"004d7cad-e19e-4ecb-8d22-1d80ed3b29cd","connectionId":2,"connectionCount":1}

t={"$date":"2023-03-05T13:08:31.826+00:00"} s=I c=NETWORK id=22944 ctx=conn3 msg=Connection ended attr={"remote":"127.0.0.1:51424","uuid":"cbecd187-8774-4dbd-a534-26774764a335","connectionId":3,"connectionCount":0}
josegonzalez commented 1 year ago

Okay interesting. This at least doesn't work on arm (not arm64), which is what I'm running on my raspberry pi. It doesn't work there because there is no linux/arm/v7 version of mongo 😂

I got access to oracle cloud, so I might have a bit of time later this week to test it out there.

nemanjam commented 1 year ago

Is there a chance for this to be resolved, it's blocking the project I intended to do?

josegonzalez commented 1 year ago

There is, but since this is open source and not a paid request, I can only look at it when I have time/energy.

Of note, I haven't been able to provision an OCI Ampere instance, so last night I requested an upgrade to "pay as you go" and am still awaiting verification. I can't really do anything about not having access to a machine that emulates your issue.

nemanjam commented 1 year ago

I can give you ssh keys from my server and Dokku container. It's demo server with no private data.

lebalz commented 1 year ago

@nemanjam run in the same problem today on a x8664 ubuntu machine. In my case the workaround was easy: i had to replace all - to underscores `` in my mongodb name. So i had to use

dokku mongo:create mern_boilerplate_database instead of dokku mongo:create mern-boilerplate-database.

nemanjam commented 1 year ago

@lebalz useful, will try it, thanks.

nemanjam commented 1 year ago

@nemanjam run in the same problem today on a x8664 ubuntu machine. In my case the workaround was easy: i had to replace all - to underscores `` in my mongodb name. So i had to use

dokku mongo:create mern_boilerplate_database instead of dokku mongo:create mern-boilerplate-database.

ubuntu@arm1:~/traefik-proxy/apps/dokku$ dokku mongo:create mern_boilerplate_database
       Waiting for container to be ready
MongoServerError: Authentication failed.
ubuntu@arm1:~/traefik-proxy/apps/dokku$
lebalz commented 1 year ago

@nemanjam run in the same problem today on a x8664 ubuntu machine. In my case the workaround was easy: i had to replace all - to underscores `in my mongodb name. So i had to use dokku mongo:create mern_boilerplate_databaseinstead ofdokku mongo:create mern-boilerplate-database`.

ubuntu@arm1:~/traefik-proxy/apps/dokku$ dokku mongo:create mern_boilerplate_database
       Waiting for container to be ready
MongoServerError: Authentication failed.
ubuntu@arm1:~/traefik-proxy/apps/dokku$

Is the old dokku instance with the old naming still running? Could you try another name for testing?

nemanjam commented 1 year ago

I don't plan to waste any more time on this.