search

schollz / find

High-precision indoor positioning framework for most wifi-enabled devices.
https://www.internalpositioning.com/
GNU Affero General Public License v3.0
5.05k stars 368 forks source link

mosquitto does not create pid file #140

Open tavalin opened 7 years ago

tavalin commented 7 years ago

I'm trying to test the new file-based runtime config using a Docker image. On the whole the config file is read and the config options are loaded.

The issue at the moment is that the sample conf file (findserver.conf) specifies that the mosquitto.pid should be read from /var/run/mosquitto.pid which seems logical and according to the mosquitto documentation.

However /var/run/mosquitto.pid is never created in the Docker image and causes findserver to terminate. In fact I can't seem to find it anywhere within the image.

At a guess I'd say there's an issue with mosquitto within the image or maybe it doesn't have correct permissions to create the pid file. Any ideas what's going on?

schollz commented 7 years ago

I ran into this exact issue on my home server - Ubuntu 14.04.5 LTS, mosquitto build 25 Aug 2016. mosquitto will never make a pid file (run in normal mode, daemon mode, or from init script). However, on my DO droplet (also Ubuntu 14.04.5 LTS, same mosquitto build), it does.

Maybe its a permission issue? I will have to look into it and let you know.

tavalin commented 7 years ago

Glad to know it's not just me. Having manually run mosquitto in the Docker image I seem to recall it ran with a mosquitto user, rather than root user when I looked at the ps -ef output. I think I'll try manually creating the /var/run/mosquitto.pid and giving whichever user mosquitto is being run with the correction permissions and see what happens.

Just found this, might be useful:

pid_file file path Write a pid file to the file specified. If not given (the default), no pid file will be written. If the pid file cannot be written, mosquitto will exit. This option only has an effect is mosquitto is run in daemon mode. If mosquitto is being automatically started by an init script it will usually be required to write a pid file. This should then be configured as e.g. /var/run/mosquitto.pid Not reloaded on reload signal.

I'll try adding the daemon mode flag to the supervisord.conf

tavalin commented 7 years ago

So I did a bit more research and found this:

Programs meant to be run under supervisor should not daemonize themselves.

Mosquitto only seems to create the pid when running as a daemon, but supervisord doesn't like running programs as daemons. Maybe we need a new way to determine the pid.

schollz commented 7 years ago

Yeah, its annoying supervisord won't allow you to create a new pid file.

What about just using ps aux in the findserver itself? I didn't really want to do this, but it could resort to that as a last resort.

tavalin commented 7 years ago

Do you mean ps aux in the .conf file? Or a code change in the src code?

schollz commented 7 years ago

Code change in the source code. I.e. if the PID is not provided but mosquitto is specified, then the findserver should try to obtain the PID itself.

tavalin commented 7 years ago

To be honest, if it were me I'd remove the mosquitto integration, replacing it simply with a go MQTT client library (to be able to publish to the location results). This way users could continue to install and run mosquitto if they wish by pointing the Find config to their install, equally users (like myself) who have an external MQTT server aren't required to have mosquitto installed on their Find box.

I'm aware that it was a quick and easy way to integrate MQTT into Find but it seems to be causing a bit of unnecessary headache.

schollz commented 7 years ago

Yeah, sorry about this headache. I think I'll just go with trying ps aux within the code itself.

I realize that the mosquitto built-in is quite a work-around some issues, but right now it is the simplest way for me to deploy FIND with MQTT server builtin with security, i.e. password protection on channels. Having FIND just use an arbitrary external MQTT server would be great, but I'm not sure how to also include password protection on subscribed channels, as that would depend on the external MQTT server.

I'd be very open to ideas about this. Like maybe it would be possible to publish only to a specific channel that can't be subscribed to via "#"?

tavalin commented 7 years ago

Can you explain a bit about the use cases for needing those features like password protection for specific channels? My current thinking(maybe naively):

  1. You publish to a private MQTT server as you're conscious about privacy and security, which you configure to require a username and password in order to be able to connect.
  2. You publish to a public MQTT server as you don't care about the privacy and security of the data that's coming out the Find server.

Something else that has just come to mind...Go runs on both Linux, Windows and Mac, right? Aren't you effectively killing the ability for a user to run a Find server with MQTT on Windows with this method of mosquitto integration?

schollz commented 7 years ago

The way MQTT works seems to be that you can subscribe to any channel you want if they aren't password protected. For the public FIND server, this is bad, because you could just subscribe to "#" and track all of the users. The only way around this in Mosquitto, I determined, was to use password protection on the channels to prevent subscriptions to "#" on the public server (except for the admin).

Of course, on your own computer/server you are free to do whatever you want (and not use Mosquitto). However I think a lot of people, maybe most, just use FIND as a service via the app I don't want them to worry about their MQTT data being insecure. This is my main use-case: providing the FIND service and this is facilitated by integrating Mosquitto specific things to streamline.

Yes, the FIND server definitely works on Linux/Windows/Mac. Windows/Mac will definetly not work with Mosquitto at all, because the implementation in FIND relies on sending a SIGHUP which I only implemented in Linux. However, if Windows/Mac machines have their own MQTT server and they just wish to publish things to it from FIND, that should work just fine with the recent change: https://github.com/schollz/find/commit/bc2533031d9206a0249a3cbe671ff625cb875667.

I think https://github.com/schollz/find/commit/bc2533031d9206a0249a3cbe671ff625cb875667 is a very good change to FIND, and I think implementing something more general, like you are getting at, is also a good idea - so Mosquitto is not so integrated in a obtrusive way. Unfortunately I can't really expand this into FIND because I don't use any MQTT software except Mosquitto on Linux, so I don't really have a good way of going about developing and testing this kind of change. I am more than happy to assist in providing this kind of functionality though!

tavalin commented 7 years ago

As an interim solution I've forked the repo and managed to track down the offending code. The issue (for me) is updateMosquittoConfig() in mqtt.go so I've commented the body of that function out and it seems to be working fine now under Docker.

As a compromise could we make that function optional depending on some flag?

schollz commented 7 years ago

Definitely! If you'd like to submit a PR I'd be happy to merge. Otherwise I can get to that later this weekend probably.

tavalin commented 7 years ago

Given that I've never programmed in Go before, I feel you may have done it before I figure out all the command line parsing but if I get some free time tomorrow I'll have a look into it :)

gonjumixproject commented 6 years ago

I am using "/usr/bin/pgrep -u mosquitto > /var/run/mosquitto.pid" to manually create the "mosquitto.pid" file.