search

emqxarchive / emqx-docker

This repository is no longer maintained, please go to https://github.com/emqx/emqx-rel/tree/master/deploy/docker
Apache License 2.0
235 stars 162 forks source link

which directories should i persist in emqx #89

Open nerdial opened 5 years ago

nerdial commented 5 years ago

Question

which directories should i persist in emqx ?

Description

I need to persist each application credential i create in emqx in case of backup or something goes wrong.if s.th goes worng with my docker container, my software could still be available and therefore i don't have to create another app id and app secret. i already tried to persist like below :

but unfortunately it does not work as it expected. when i docker down and docker up again my app credentials will be lost but i believe the logs and data are there. overall which directories should i persist ? especially if i need to activate emqx_retainer and save messages to disk

Rory-Z commented 5 years ago
nerdial commented 5 years ago

i added these volumes in my docker-compose file and created an application called "johndoe" in emqx dashboard, but unfortunately this just does not work. i ran the following command :

docker-compose up -d --force-recreate emqx

but when i visited dashboard, application was not there and it seemed it was deleted. i searched in these directories and i could find this file in the directory

i ran this command in my host machine and i am pretty sure this file exists in both host and container.

grep -nri johndoe . Binary file ./data/mnesia/10b0bd32802d@192.168.0.6/LATEST.LOG matches

deepaksood619 commented 5 years ago

service for emqx for persisting data across container recreation

services:
  emqx:
    stdin_open: true
    tty: true
    restart: always
    image: emqx/emqx:v3.0-rc.5
    container_name: emqx
    entrypoint: []
    networks:
      - confluent
    ports:
      - "18083:18083"
      - "1883:1883"
      - "8080:8080"
    volumes:
      - ./emqx/log:/opt/emqx/log #working
      - ./emqx/data/mnesia:/opt/emqx/data/mnesia #working
      - ./emqx/lib:/opt/emqx/lib #not working
      - ./emqx/etc:/opt/emqx/etc/plugins/ #not working

Using the above configuration i was not able to start the docker container

Logs:

Creating emqx ... done
Attaching to emqx
emqx                  | node.max_ports=1048576
emqx                  | cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory
emqx                  | cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory
emqx                  | listener.tcp.external.acceptors=64
emqx                  | cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory
emqx                  | cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory
emqx                  | cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory
emqx                  | listener.ssl.external.acceptors=32
emqx                  | cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory
emqx                  | node.process_limit=2097152
emqx                  | cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory
emqx                  | node.max_ets_tables=2097152
emqx                  | cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory
emqx                  | listener.ws.external.acceptors=16
emqx                  | cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory
emqx                  | node.name=74a0c98149e9@172.19.0.2
emqx                  | cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory
emqx                  | cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory
emqx                  | cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory
emqx                  | cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory
emqx                  | cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory
emqx                  | cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory
emqx                  | cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory
emqx                  | run_erl[510]: Erlang closed the connection.
emqx                  | 2018-12-11T09:37:06.443197+00:00 [error] Translation for 'emqx_auth_pgsql.server' expected to find setting 'auth.pgsql.database' but was missing
emqx                  | 2018-12-11T09:37:06.443349+00:00 [error] Translation for 'emqx_auth_mongo.auth_query' expected to find setting 'auth.mongo.auth_query.password_hash' but was missing
emqx                  | 2018-12-11T09:37:06.443503+00:00 [error] Translation for 'emqx_auth_http.auth_req' expected to find setting 'auth.http.auth_req.params' but was missing
emqx                  |
emqx                  | =====
emqx                  | ===== LOGGING STARTED Tue Dec 11 09:41:03 UTC 2018
emqx                  | =====
emqx                  | init terminating in do_boot ({load_failed,[supervisor,proc_lib,lists,logger_simple_h,logger_config,logger_backend,logger_server,logger_filters,logger,kernel,heart,gen_server,gen_event,gen,file_io_serv
emqx                  |
emqx                  | Crash dump is being written to: erl_crash.dump...done
emqx                  | init terminating in do_boot ({load_failed,[supervisor,proc_lib,lists,logger_simple_h,logger_config,logger_backend,logger_server,logger_filters,logger,kernel,heart,gen_server,gen_event,gen,file_io_serv
emqx                  |
emqx                  | Crash dump is being written to: erl_crash.dump...done
emqx                  | ['2018-12-11T09:41:21Z']:waiting emqx
emqx                  | init terminating in do_boot ({load_failed,[supervisor,proc_lib,lists,logger_simple_h,logger_config,logger_backend,logger_server,logger_filters,logger,kernel,heart,gen_server,gen_event,gen,file_io_serv
emqx                  |
emqx                  | Crash dump is being written to: erl_crash.dump...done
emqx                  | ['2018-12-11T09:41:22Z']:waiting emqx
emqx                  | init terminating in do_boot ({load_failed,[supervisor,proc_lib,lists,logger_simple_h,logger_config,logger_backend,logger_server,logger_filters,logger,kernel,heart,gen_server,gen_event,gen,file_io_serv
emqx                  |
emqx                  | Crash dump is being written to: erl_crash.dump...done
emqx                  | ['2018-12-11T09:41:24Z']:waiting emqx
emqx                  | init terminating in do_boot ({load_failed,[supervisor,proc_lib,lists,logger_simple_h,logger_config,logger_backend,logger_server,logger_filters,logger,kernel,heart,gen_server,gen_event,gen,file_io_serv
emqx                  |
emqx                  | Crash dump is being written to: erl_crash.dump...done
emqx                  | ['2018-12-11T09:41:25Z']:waiting emqx
emqx                  | init terminating in do_boot ({load_failed,[supervisor,proc_lib,lists,logger_simple_h,logger_config,logger_backend,logger_server,logger_filters,logger,kernel,heart,gen_server,gen_event,file_io_server,g
emqx                  |
emqx                  | Crash dump is being written to: erl_crash.dump...done
emqx                  | ['2018-12-11T09:41:26Z']:waiting emqx
emqx                  | init terminating in do_boot ({load_failed,[supervisor,proc_lib,lists,logger_simple_h,logger_config,logger_backend,logger_server,logger_filters,logger,kernel,heart,gen_server,gen_event,gen,file_io_serv
emqx                  |
emqx                  | Crash dump is being written to: erl_crash.dump...done
emqx                  | ['2018-12-11T09:41:28Z']:waiting emqx
emqx                  | ['2018-12-11T09:41:28Z']:timeout error
emqx exited with code 1

I have created the emqx/log, emqx/data, emqx/lib, emqx/etc in my local directory where docker-compose.yml resides.

Using only the log and mnesia seems to start the container.

I want to persist my app configuration in case of container restarts (i.e. when container is recreated)

What is the right way to achieve this?

Thanks!

RaymondMouthaan commented 5 years ago

@deepaksood619, I faced the same issue as you regarding persistence (https://github.com/emqx/emqx-docker/pull/91#issuecomment-445826906) ... unfortunately no final solution yet.

I haven't tried, but a workaround could be:

  1. start a temporary docker container without given persitstence directories, but instead persist /tmp docker run -it -v /hostdir:/tmp --name=temp-emqx emqx/emqx:v3.0-rc.5
  2. login to the container docker exec -it temp-emq /bin/sh
  3. within the container copy the required dirs (/opt/emqx/log,/opt/emqx/data,/opt/emqx/lib and /opt/emqx/etc/plugins/) to /tmp. Maybe the emqx process needs to be stopped before copying. cp -Rv /opt/emqx/log /opt/emqx/data /opt/emqx/lib /opt/emqx/etc/plugins /tmp
  4. Stop and remove temporary container. At this point all dirs should be available on your host
  5. Start your docker service / container pointing to the host dirs which where copied in previous step.
deepaksood619 commented 5 years ago

@RaymondMouthaan, i tried this approach but was not able to copy the files as both emqx and root user with both emqx started and stopped.

deepaksood@Deepaks-MacBook-Pro  ~  docker exec -it --user root temp-emqx /bin/sh
/opt/emqx #
/opt/emqx # ./bin/emqx stop
ok
/opt/emqx # cp -Rv /opt/emqx/log /opt/emqx/data /opt/emqx/lib /opt/
emqx/etc/plugins /tmp
cp: can't create directory '/tmp/log': Permission denied
cp: can't create directory '/tmp/data': Permission denied
cp: can't create directory '/tmp/lib': Permission denied
cp: can't create directory '/tmp/plugins': Permission denied
/opt/emqx # whoami
root

Same i tried as emqx user too, but with no success.

RaymondMouthaan commented 5 years ago

@deepaksood619 can you do:

ls -al / within the container? ... just to check the permissions on the temp dir

deepaksood619 commented 5 years ago

@RaymondMouthaan 

/ # ls -al
total 60
drwxr-xr-x    1 root     root          4096 Dec 11 15:40 .
drwxr-xr-x    1 root     root          4096 Dec 11 15:40 ..
-rwxr-xr-x    1 root     root             0 Dec 11 15:40 .dockerenv
drwxr-xr-x    1 root     root          4096 Dec  1 10:16 bin
drwxr-xr-x    5 root     root           360 Dec 11 15:40 dev
drwxr-xr-x    1 root     root          4096 Dec 11 15:40 etc
drwxr-xr-x    1 root     root          4096 Dec  1 10:17 home
drwxr-xr-x    1 root     root          4096 Dec  1 10:16 lib
drwxr-xr-x    5 root     root          4096 Sep 11 20:23 media
drwxr-xr-x    2 root     root          4096 Sep 11 20:23 mnt
drwxr-xr-x    1 root     root          4096 Dec  1 10:17 opt
dr-xr-xr-x  205 root     root             0 Dec 11 15:40 proc
drwx------    1 root     root          4096 Dec 11 15:40 root
drwxr-xr-x    2 root     root          4096 Sep 11 20:23 run
drwxr-xr-x    2 root     root          4096 Sep 11 20:23 sbin
drwxr-xr-x    2 root     root          4096 Sep 11 20:23 srv
dr-xr-xr-x   13 root     root             0 Dec 11 15:40 sys
drwxr-xr-x    2 root     root            64 Dec 11 14:57 tmp
drwxr-xr-x    1 root     root          4096 Dec  1 10:16 usr
drwxr-xr-x    1 root     root          4096 Dec  1 10:16 var
/ # whoami
root
/ #
RaymondMouthaan commented 5 years ago

hmm, weird .. have you tried, creating a dir within /tmp like this? mkdir -p /tmp/test

deepaksood619 commented 5 years ago

already done but with no success

/ # whoami
root
/ # cd tmp/
/tmp # mkdir -p test
mkdir: can't create directory 'test': Permission denied
/tmp #
deepaksood619 commented 5 years ago

Also tried less /etc/passwd

root:x:0:0:root:/root:/bin/ash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
news:x:9:13:news:/usr/lib/news:/sbin/nologin
uucp:x:10:14:uucp:/var/spool/uucppublic:/sbin/nologin
operator:x:11:0:operator:/root:/bin/sh
man:x:13:15:man:/usr/man:/sbin/nologin
postmaster:x:14:12:postmaster:/var/spool/mail:/sbin/nologin
cron:x:16:16:cron:/var/spool/cron:/sbin/nologin
ftp:x:21:21::/var/lib/ftp:/sbin/nologin
sshd:x:22:22:sshd:/dev/null:/sbin/nologin
at:x:25:25:at:/var/spool/cron/atjobs:/sbin/nologin
squid:x:31:31:Squid:/var/cache/squid:/sbin/nologin
xfs:x:33:33:X Font Server:/etc/X11/fs:/sbin/nologin
games:x:35:35:games:/usr/games:/sbin/nologin
postgres:x:70:70::/var/lib/postgresql:/bin/sh
cyrus:x:85:12::/usr/cyrus:/sbin/nologin
vpopmail:x:89:89::/var/vpopmail:/sbin/nologin
ntp:x:123:123:NTP:/var/empty:/sbin/nologin
smmsp:x:209:209:smmsp:/var/spool/mqueue:/sbin/nologin
guest:x:405:100:guest:/dev/null:/sbin/nologin
nobody:x:65534:65534:nobody:/:/sbin/nologin
emqx:x:1000:1000:Linux User,,,:/home/emqx:

But can't seem to find the problem

deepaksood619 commented 5 years ago

@RaymondMouthaan, here is the DockerFile emqtt is using for building it's images. Please have a look. https://github.com/emqx/emqx-docker/blob/emqx30/Dockerfile

RaymondMouthaan commented 5 years ago

Like you, i am confused as well .. can you try without persist ? (assuming you have used -v in your docker command)?

deepaksood619 commented 5 years ago

Yes i am able to create the directory now inside /tmp without the volume mount.

But we used -v /hostdir:/tmp to persist the data between docker recreates... There must be some way to persist data using docker without native deployment.

RaymondMouthaan commented 5 years ago

Great, then there could be at least two reasons I can think of why it doesn't work with /tmp

What you could test is .. mount volume as before but add a file to the host dir and check if this file is visible inside the container.

Another test/solution would be, mount /opt/emqx/log to your host file dir, then copy all other dirs to that. So basically instead of /tmp use /opt/emqx/log instead. Maybe remove created logs before copying...

deepaksood619 commented 5 years ago

For the first part, mounting files using docker run -it -v /hostdir:/tmp --name=temp-emqx emqx/emqx:v3.0-rc.5 makes all the file available inside the hostdir available inside docker container, and creating a new file is reflected inside docker container. But I was not able to create file inside docker container using both users emqx and root.

For second I got permission denied for the volume mounts.

 master ●  docker run -it -v /hostdir:/opt/emqx/log --name=temp-emqx emqx/emqx:v3.0-rc.5
node.max_ports=1048576
listener.tcp.external.acceptors=64
listener.ssl.external.acceptors=32
node.process_limit=2097152
node.max_ets_tables=2097152
listener.ws.external.acceptors=16
node.name=5e70861b5f49@172.17.0.2
run_erl[517]: errno=13 'Permission denied'
Can't open log file '/opt/emqx/log/erlang.log.1'.
tail: can't open '/opt/emqx/log/erlang.log.1': No such file or directory
tail: no files
Node '5e70861b5f49@172.17.0.2' not responding to pings.
['2018-12-11T17:01:40Z']:waiting emqx
Node '5e70861b5f49@172.17.0.2' not responding to pings.
['2018-12-11T17:01:42Z']:waiting emqx
Node '5e70861b5f49@172.17.0.2' not responding to pings.
['2018-12-11T17:01:43Z']:waiting emqx
Node '5e70861b5f49@172.17.0.2' not responding to pings.
['2018-12-11T17:01:45Z']:waiting emqx
Node '5e70861b5f49@172.17.0.2' not responding to pings.
['2018-12-11T17:01:47Z']:waiting emqx
Node '5e70861b5f49@172.17.0.2' not responding to pings.
['2018-12-11T17:01:48Z']:waiting emqx
['2018-12-11T17:01:48Z']:timeout error

But i was able to do this using docker-compose

services:
  emqx:
    stdin_open: true
    tty: true
    restart: always
    image: emqx/emqx:v3.0-rc.5
    container_name: emqx
    entrypoint: []
    environment:
      EMQX_ALLOW_ANONYMOUS: "false"
      EMQX_LOADED_PLUGINS: "emqx_recon,emqx_retainer,emqx_management,emqx_dashboard"
    networks:
      - confluent
    ports:
      - "18083:18083"
      - "1883:1883"
      - "8080:8080"
    volumes:
      - ./emqx/log:/opt/emqx/log #able to mount but 
#      - ./emqx/data:/opt/emqx/data #not working
#      - ./emqx/lib:/opt/emqx/lib #not working
#      - ./emqx/etc:/opt/emqx/etc #not working

Now doing the above i was able to create file both sides i.e. on host machine and inside docker container and both reflected back to other side.

RaymondMouthaan commented 5 years ago

Try ./hostdir instead /hostdir and make sure the dir exists.

deepaksood619 commented 5 years ago

Yes! it worked, data is copied to hostdir and emqtt is starting. But the initial problem persists, that an app created from management console in localhost:18083 doesn't persist after new container recreate.

This is what i did-

docker run -it -v /Users/deepaksood/hostdir/data:/opt/emqx/data -v /Users/deepaksood/hostdir/lib:/opt/emqx/lib -v /Users/deepaksood/hostdir/plugins:/opt/emqx/etc/plugins -p 18083:18083 -p 8080:8080 --name=temp-emqx emqx/emqx:v3.0-rc.5

All the previous session data is there in the /Users/deepaksood/hostdir/data folder but after new docker container new data folders are created.

 deepaksood@Deepaks-MacBook-Pro  ~/hostdir  ls -al data/mnesia/
total 0
drwxr-xr-x  4 deepaksood  staff   128B Dec 12 00:06 ./
drwxr-xr-x  6 deepaksood  staff   192B Dec 12 00:01 ../
drwxr-xr-x  7 deepaksood  staff   224B Dec 12 00:06 092f51295a19@172.17.0.2/
drwxr-xr-x  7 deepaksood  staff   224B Dec 12 00:01 112add4a7925@172.17.0.2/
 deepaksood@Deepaks-MacBook-Pro  ~/hostdir 
 deepaksood@Deepaks-MacBook-Pro  ~/hostdir  ls
data/    lib/     plugins/
 deepaksood@Deepaks-MacBook-Pro  ~/hostdir 
 deepaksood@Deepaks-MacBook-Pro  ~/hostdir 
 deepaksood@Deepaks-MacBook-Pro  ~/hostdir  ls data
configs/        emqx_erl_pipes/ loaded_plugins  mnesia/
 deepaksood@Deepaks-MacBook-Pro  ~/hostdir  ls -al data
total 8
drwxr-xr-x  6 deepaksood  staff   192B Dec 12 00:01 ./
drwxr-xr-x  5 deepaksood  staff   160B Dec 12 00:01 ../
drwxr-xr-x  8 deepaksood  staff   256B Dec 12 00:06 configs/
drwxr-xr-x  4 deepaksood  staff   128B Dec 12 00:06 emqx_erl_pipes/
-rw-r--r--  1 deepaksood  staff    60B Dec 12 00:01 loaded_plugins
drwxr-xr-x  4 deepaksood  staff   128B Dec 12 00:06 mnesia/
 deepaksood@Deepaks-MacBook-Pro  ~/hostdir  ls -al data/emqx_erl_pipes
total 0
drwxr-xr-x  4 deepaksood  staff   128B Dec 12 00:06 ./
drwxr-xr-x  6 deepaksood  staff   192B Dec 12 00:01 ../
drwxr-xr-x  4 deepaksood  staff   128B Dec 12 00:06 092f51295a19@172.17.0.2/
drwxr-xr-x  4 deepaksood  staff   128B Dec 12 00:01 112add4a7925@172.17.0.2/
 deepaksood@Deepaks-MacBook-Pro  ~/hostdir  ls -al data/configs
total 400
drwxr-xr-x  8 deepaksood  staff   256B Dec 12 00:06 ./
drwxr-xr-x  6 deepaksood  staff   192B Dec 12 00:01 ../
-rw-r--r--  1 deepaksood  staff    77K Dec 12 00:01 app.2018.12.11.18.29.22.conf
-rw-r--r--  1 deepaksood  staff    14K Dec 12 00:01 app.2018.12.11.18.29.23.config
-rw-r--r--  1 deepaksood  staff    77K Dec 12 00:06 app.2018.12.11.18.36.47.conf
-rw-r--r--  1 deepaksood  staff    14K Dec 12 00:06 app.2018.12.11.18.36.48.config
-rw-r--r--  1 deepaksood  staff   241B Dec 12 00:01 vm.2018.12.11.18.29.23.args
-rw-r--r--  1 deepaksood  staff   241B Dec 12 00:06 vm.2018.12.11.18.36.48.args
 deepaksood@Deepaks-MacBook-Pro  ~/hostdir 

How can i make emqtt use old data folders instead of creating new ones.

RaymondMouthaan commented 5 years ago

Okee, this confuses me a lot, basically when the dirs / files exists on the host, then emqx creates new ones.. If they don't exist, emqx won't even start 🤔

deepaksood619 commented 5 years ago

Yes that sums it all... 😐

On Wed 12 Dec, 2018, 12:27 AM Raymond Mouthaan <notifications@github.com wrote:

Okee, this confuses me a lot, basically when the dirs / files exists on the host, then emqx creates new ones.. If they don't exist, emqx won't even start 🤔

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/emqx/emqx-docker/issues/89#issuecomment-446319252, or mute the thread https://github.com/notifications/unsubscribe-auth/AMmEVEv6tK19L7mFyEjGMdlTRRRBEYw5ks5u4AAegaJpZM4YxbLJ .

deepaksood619 commented 5 years ago

Any update on this issue?

WhyKickAmooCow commented 5 years ago

As far as I can tell emqx is creating folders to contain config files based on the id of the node. This is probably in cases where there are multiple nodes running on a single physical machine without the use of containers in order to avoid conflicts when reading and writing to these files; When a new container is created, the hostname and possibly IP address also change, changing the name of the emqx node and therefore the folders that the node writes to.

The solution that worked for me was to manually set the emqx node name via an environment variable to ensure that the node name and therefore configuration folder being used was consistent across container destruction and recreation. The environment variable for controlling this is EMQX_NODE__NAME and should contain data in the format of EMQX_NAME@EMQX_HOST, in my case this was "ash-mqtt@mqtt.local" and user login, and applications are now persistent.

RaymondMouthaan commented 5 years ago

@WhyKickAmooCow thanks for your reply! Could you supply an example of you full docker command / docker compose file, how you manage to get persistence to work?

WhyKickAmooCow commented 5 years ago

Here is a sample of my docker-compose file. For me it really was as simple as keeping the suggested folders above in volumes, and setting an environment variable.

    mqtt:
        image: emqx-docker-v3.0.0
        environment:
            - EMQX_NODE__NAME=ash-mqtt@mqtt.local
        ports:
            - 1883:1883
            - 8083:8083
            - 8084:8084
            - 8080:8080
            - 18083:18083
        volumes:
            - ./emqx-config:/opt/emqx/etc
            - ./emqx-data:/opt/emqx/data
            - ./emqx-lib:/opt/emqx/lib
        networks: 
            ash:
                aliases:
                    - mqtt.local
RaymondMouthaan commented 5 years ago

Probably my mistake but I still can't get it to work with:

services:
  emqx-master:
    image: raymondmm/emqx
    hostname: emqx-master
    environment:
      - TZ=Europe/Amsterdam
      - EMQX_NAME=emq
      - EMQX_HOST=master.mq.tt
      - EMQX_NODE__COOKIE=ef16498f66804df1cc6172f6996d5492
      - EMQX_WAIT_TIME=60
      - EMQX_NODE__NAME=emq@master.mq.tt
    networks:
      indonesia-net:
        aliases:
          - master.mq.tt
      proxy_indonesia-net:
    volumes:
      - /shares/rpi-cluster-1/emqx-master/log:/opt/emqx/log
      - /shares/rpi-cluster-1/emqx-master/data:/opt/emqx/data
      - /shares/rpi-cluster-1/emqx-master/lib:/opt/emqx/lib
      - /shares/rpi-cluster-1/emqx-master/etc:/opt/emqx/etc

These are the errors I am getting:

cat: can't open '/opt/emqx/etc/emqx.conf': No such file or directory

cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory

cat: can't open '/opt/emqx/etc/emqx.conf': No such file or directory

cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory

cat: can't open '/opt/emqx/etc/emqx.conf': No such file or directory

cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory

cat: can't open '/opt/emqx/etc/emqx.conf': No such file or directory

cat: can't open '/opt/emqx/etc/plugins/*': No such file or directory

vm.args needs to have a -name parameter.

  -sname is not supported.

=====

===== LOGGING STARTED Thu Jan  3 02:16:07 CET 2019

=====

init terminating in do_boot ({load_failed,[supervisor,proc_lib,lists,logger_simple_h,logger_config,logger_backend,logger_server,logger_filters,logger,kernel,heart,gen_server,gen_event,gen,file_io_serv

Crash dump is being written to: erl_crash.dump...done

/opt/emqx/bin/emqx_ctl: line 40: echoerr: not found

['2019-01-03T01:18:54Z']:waiting emqx

/opt/emqx/bin/emqx_ctl: line 40: echoerr: not found

['2019-01-03T01:18:55Z']:waiting emqx

/opt/emqx/bin/emqx_ctl: line 40: echoerr: not found

['2019-01-03T01:18:56Z']:waiting emqx

/opt/emqx/bin/emqx_ctl: line 40: echoerr: not found

All host dirs are created and are empty before creating the service.

WhyKickAmooCow commented 5 years ago

On first setup I let the container run without the lib and etc volumes mounted so not to overwrite the existing folders in the container with empty ones. Then I used docker cp container_id:folder src_dir to copy the lib and etc dirs from the running container into my volume. I could then get the docker-compose run without any errors, although you might be alright to only copy the etc folder given your errors.

RaymondMouthaan commented 5 years ago

Thanks, that got me a little further... Still got some issues, but I'll try again later. Either way it's not very user friendly to persist emqx. Hopefully emqx team will improve this, also when running in clustered setup. A full example with a master and a worker including persistence for both would very helpful 👍🏽

aisven commented 5 years ago

This here is really an issue when deploying emqx. It does cost time to figure it all out. I finally got really far but now it keeps telling me this remaining bit and then exists abnormally:

Error opening file: /opt/emqx/data/configs/app.2019.04.10.10.10.33.conf no such file or directory
neo-feng commented 5 years ago

The logic should be:

  1. use container default files as default if host folders are empty
  2. host changes map to container (by default)
gxsshallot commented 4 years ago

I have resolved persist with emqx/emqx:v4.0.0, but I think it is also valid for previous version.

First, use yml file without volume. You must set NODE_NAME and NODE_HOST to make node unique. Following is docker-compose.yml:

version: "3"
services:
    emqx:
        container_name: emqx
        image: emqx/emqx:v4.0.0
        networks:
            cloud:
                aliases:
                    - emqx
        environment:
            - EMQX_NAME=mqtt
            - EMQX_HOST=10.20.30.40
        ports:
            - "1883:1883"
networks:
    cloud:
        external: true

After startup the container successfully, you should copy the folder in container to your persist location:

docker cp emqx:/opt/emqx/etc ~/emqx/etc
docker cp emqx:/opt/emqx/lib ~/emqx/lib
docker cp emqx:/opt/emqx/data ~/emqx/data
docker cp emqx:/opt/emqx/log ~/emqx/log

Then change the owner, group of these folders:

cd ~/emqx
chown -R 1000:1000 data etc lib log
chmod -R 775 data etc lib log

Modify the docker-compose.yml and add the volume mapping:

volumes:
    - $HOME/emqx/lib:/opt/emqx/lib
    - $HOME/emqx/etc:/opt/emqx/etc
    - $HOME/emqx/data:/opt/emqx/data
    - $HOME/emqx/log:/opt/emqx/log

Remove the container and recreate one. That is OK.