MQTT not working after I restart my docker container

[seaverd]

Schollz,

I pulled the latest update of find3 as I was having the issued that you documented and closed in #90 and #91. I am running in docker and all is well until I restart my find3 container. Below is my log...I copied and pasted a section that includes it running as it supposed to for the first few lines...then at 18:01 I restarted the container...you can see it startup at 18:01:22 and it could not update the mosquitto config.

So when the container restarts I cannot connect to the MQTT server. If I delete the container and my find3 data folder then set up the container from scratch it will work (until the next restart).

My logs contain the following

`2018-05-29 18:01:00 [37m[DEBUG][0m [PID-17] mqtt.go messageReceived:210 Got valid MQTT request for group MYFAMILYNAME, user mycell

2018-05-29 18:01:00 [37m[DEBUG][0m [PID-17] db.go AddSensor:365 [MYFAMILYNAME] inserted sensor data, 21.002443ms 2018-05-29 18:01:00 [ERROR] [PID-17] analysis.go AnalyzeSensorData:222 unable to analyze: could not find '/data/data/zTuP5y1K.find3.ai'

2018-05-29 18:01:00 [DEBUG] [PID-17] mqtt.go sendOutData:247 [MYFAMILYNAME] sending data over mqtt (mycell)

2018-05-29 18:01:00 [DEBUG] [PID-17] analysis.go func2:126 [MYFAMILYNAME] nb1 classified 3.059742ms

2018-05-29 18:01:22 [DEBUG] [PID-15] mqtt.go Setup:49 setting up

2018-05-29 18:01:22 [DEBUG] [PID-15] mqtt.go Setup:56 using current setup

2018-05-29 18:01:22 [DEBUG] [PID-15] mqtt.go updateMosquittoConfig:85 opening mosquitto database

2018-05-29 18:01:22 [DEBUG] [PID-15] mqtt.go updateMosquittoConfig:94 starting database

2018-05-29 18:01:22 [DEBUG] [PID-15] mqtt.go updateMosquittoConfig:98 making acl

2018-05-29 18:01:22 [DEBUG] [PID-15] mqtt.go updateMosquittoConfig:103 making passwd

2018-05-29 18:01:22 [DEBUG] [PID-15] mqtt.go updateMosquittoConfig:108 making conf

2018-05-29 18:01:22 [WARN] [PID-15] server.go Run:38 could not update mosquitto config: problem giving HUP: exit status 1

2018-05-29 18:01:22 [DEBUG] [PID-15] server.go Run:38 setup mqtt

2018-05-29 18:01:22 [DEBUG] [PID-15] server.go Run:44 current families: [MYFAMILYNAME]

2018-05-29 18:01:22 [INFO] [PID-15] server.go Run:355 Running on 0.0.0.0:8003`

[yajrendrag]

What i found was that if the pid file exists in the mosquitto_config directory (i haven;t updated find3 since i installed it a few months ago, so not sure if the directory locations have changed), that mosquitto doesn't start. I am not running docker, but what i do when i restart the machine on which find3 is installed is test to see if the pid file exists. if it does, i delete it. then when find starts up, all works as expected. Possibly you're experiencing the same thing when restarting the find3 container.

[seaverd]

That is exactly the issue. If I delete the file named pid in the /data/find3/mosquitto_config directory the docker will restart the way it does on a brand new clean install. I am running in docker. Does anyone have a suggestion on how to delete this file as part of the docker restart?

[schollz]

Just pushed a change that might fix this. I was having problems getting MQTT to work at all, as it wasn't starting it properly. Now the Docker startup script will start it on its own and the find3 server succesfully gives a HUP signal to reload parameters. Its working now again for me.

[seaverd]

Schollz,

Thanks for looking into this. I deleted my container and all data then pulled the current docker and it is not working for me. First run of the docker everything is fine. Restart container and mqtt does not load correctly, however if I manually delete the /data/find3/mosquitto_conifg/pid file then it will start correctly.

Let me know if you need anything from my system...or would like me to try something.

Dan

[schollz]

Let's make sure we are on the same page. Can you try doing the following:

docker pull schollz/find3
sudo rm -rf ~/FIND_DATA && mkdir ~/FIND_DATA
docker run -p 11883:1883 -p 8015:8003 -v /home/$USER/FIND_DATA:/data -e MQTT_ADMIN=ADMIN -e MQTT_PASS=PASSWORD -e MQTT_SERVER='localhost:1883' --name find3test -d -t schollz/find3
go get github.com/schollz/find3
cd $GOPATH/src/github.com/schollz/find3/server/main/testing
python3 submit_jsons.py http://localhost:8015 testdb.learn.1439597065993.jsons

Now to test it:

In one terminal:

http GET localhost:8015/api/v1/mqtt/testdb
mosquitto_sub -h 127.0.0.1 -p 11883 -u testdb -P XXX -t '#'

And then in another terminal:

cd $GOPATH/src/github.com/schollz/find3/server/main/testing
python3 submit_jsons.py http://localhost:8015 testdb.learn.1439597065993.jsons

That should work. Now lets shut down and start up:

docker stop find3test
docker start find3test

Now do another python3 submit_jsons.py ... while listening to mosquitto and you should see more messages.

[seaverd]

Sorry for the delayed response, I have been away and unable to test. One problem I have is I am running Unraid and using their docker...which is template based. I was able to manually copy the testing data from github to my setup...but am unable to run http GET localhost:8015/api/v1/mqtt/testdb it does not recognize the get command. I now see there are new issues related to docker compose....could that be my issue as well?

[schollz]

Are you using docker compose?