Use FIND3 with own MQTT server

[pservais]

Is it possible to use my own MQTT server for FIND3 server? I already have a Mosquitto server running for my home automation system. With FIND I could use the Zanzito app for tracking, which is already present on our smartphones.

[yajrendrag]

If you are running find3 server locally via native install (ie, not docker), yes. I just set this up. When you launch the main server component, you can simply specify mqtt-server (e.g, localhost:1883), mqtt-admin, mqtt-pass, where mqtt-admin is defined in your mqtt acl config to have access to your find3 family (ie, family/#). It does seem that find3 and mqtt have to run on same host or at least I couldn’t figure out how to make them run on different hosts.

to see main server command options, start main like ./main --help

Not sure about zanzito. I was using room+ on old find system and that doesnt work with find3. You will likely need the new find3 android app.

[pschmitt]

I submitted a PR on the original find repo that added support for external MQTT servers. It should not be too hard to transpose this to this repo.

[schollz]

It does seem that find3 and mqtt have to run on same host.

This is correct. FIND and MQTT must be on the same server. The reason is that mosquitto MQTT requires a PID signal in order to reload the new config files. The config files need to be reloaded anytime a new user is added.

[yajrendrag]

@schollz - in preior version of find there was an option (-mosquitto) to pass the PID # to find when starting find. when i run ./main --help i don't see that option any longer. is it still there? or is that not needed to do what you're talking about here?

[schollz]

@yajrendrag forgot about that flag. I will add that back!

[schollz]

@yajrendrag @pservais I think I need to understand more about this scenario. Adding a PID will not be sufficient because FIND3 wouldn't find the configuration folder. mosquitto is tricky with its authentication and my workarounds have FIND3 rewrite config files each time a new user asks to use MQTT. Thus, if you use FIND3 with your previous mosquitto configs then you are at risk of having your previous configurations rewritten.

To figure out how this will work I need some more information.

Are you planning on running the FIND3 server without Docker (none of the below will work with Docker)?

Do you have a configuration file for mosquitto that you wish to preserve?

Are you using an acl and a passwd file that you also want to preserve?

If you aren't using acl/passwd files, are you comfortable editing them to include them for FIND3?

[yajrendrag]

i never really understood the use of the PID previously as it seemed like it was intended for use with a separate mosquitto instance associated with FIND. but the only way i could get FIND v2 to work was to use that option. in the FIND working directory, some mosquitto config files were written. but my mosquitto instance has it's config in /etc/mosquitto.

I dont use docker.

so mosquitto runs & all of my home assistant use cases use that instance of mosquitto. i have a local acl file and define various topics that are allowed to be read/written by various mosquitto users.

when i started using FIND, i simply added another mosquitto user and associated topic. my home assistant config simply parses the FIND json for the location info.

now with find3, its the same thing. so when i start main, i give it the mqtt-user and password that is in my current passwd file in /etc/mosquitto/.

If i change mosquitto config info, i simply restart mosquitto. so unsure why find and mosquitto have to be on the same server. ie, i'm not expecting any find behavior or changes to automatically get picked up by mosquitto.

not sure if im explaining this well or if i've done something wrong but only reason i mentioned that -mosquitto option was i was wondering if the PID file you mentioned needed it.

i also don't really have a need to run find and mosquitto on different servers. but when i started w/ find3, i wasn't sure if i was going to run into any issues with the installation of find 3 causing any issues with find v2, so i tried to install it on another server and that was how i discovered it wouldn't work with moquitto on a different machine. (i'm not much of a developer, but i fumbled my way through the go installation & once i understood the find3 install wouldn't overwrite anything in find v2, i simply installed it on the same server on which i have everything else running..).

[schollz]

I'm not really equipped to solve this, because I don't run an additional mosquitto broker for anything and not sure how to test these cases that I don't experience. I would recommend that people use Docker to circumvent this issue (so mosquitto is completely contained).

I'm happy to accept a PR that is a reasonable solution!

[yajrendrag]

let me try and make it simpler.

i start main with the -mqtt-pass, -mqtt-admin, and -mqtt-server options. my local mosquitto broker is at the mqtt-server location (e.g., localhost:1883). I simply added mqtt-user and mqtt-pass to my acl file at /etc/mosquitto/conf.d allowing mqtt-user readwrite access to topic FAMILY/location/#. this all works fine.

i don't know go language, but in looking through mqtt.go, it seems that you do some mqtt setup and that mqtt config info gets written to main/mosquitto_config. it seems that if my local mqtt instance is already running and configured on localhost:1883, find is content to use my existing config from /etc/mosquitto/conf.d (since i added the user info as described above)

not sure if that is pure luck or as intended. but i am content to manually start/stop mqtt as needed if i make any changes to the config.

If i understand this correctly and you don't see any issue, i'm good to go. But above, if you meant that find3 will potentially rewrite my configs at /etc/mosquitto then i think i have a problem.

[yajrendrag]

i looked through it some more - i see it's more integrated than simply altering mqtt.go. so instead, i run the combination of find3/mosquitto on another machine - or you can do same by running find3 w/ docker. then i add a bridge configuration (to find3's mosquitto instance) to my main mosquitto broker that homeassistant uses - homeassistant can only use 1 active broker. works like a champ. i rsync'd my existing database from one find3 install to another & i'm good.

[tyldum]

I solved this by federating with my main mqtt-server.

[schollz]

@tyldum Could you elaborate on your solution? I think it would be really useful, I could add it to the docs too.

[heikomat]

if #101 gets solved, one could configure the mosquitto-instance used by find3 to be an mqtt-bridge. With that, find3 could keep full control of its mqtt-server AND people could use their own mqtt-server at the same time

[yajrendrag]

using an mqtt bridge is exactly how i do it. & i put the bridge configuration in my other mqtt instance - ie, not the find3 mqtt bridge. with mqtt bridging in mosquitto the config only has to be done in one msoquitto instance - not both - so it works out well. you can read about it man mosquitto-conf and see the section entitled "CONFIGURING BRIDGES". it only requires a few additional config lines to your other mosquitto instance..

[heikomat]

@yajrendrag i know that only one of them needs to be the bridge, but i'd prefer the find3-mqtt-broker to be the bridge, not my main mosquitto instance, as i have better control over it and the users that can connect to it

[heikomat]

hmm.. looks like i was misunderstanding something, as it probably doesn't make any difference what broker acts as the bridge.

[sjcliffe]

I'm running this with my own existing Mosquitto server. I put my server details in the var in mqtt.go, set the Existing flag to true and it almost worked. Because I use authentication I had to change:

opts.AddBroker(server).SetClientID(utils.RandomString(5)).SetCleanSession(true) to opts.AddBroker(server).SetClientID(utils.RandomString(5)).SetUsername(AdminUser).SetPassword(AdminPassword).SetCleanSession(true)

Of course, this only works when building from scratch but it wouldn't take much work to add a command line argument to set the Existing flag - the code is already there to use it.

[rsteckler]

@schollz I recommend closing this. The bridge configuration from https://github.com/schollz/find3/issues/52#issuecomment-435612895 is the correct solution.