Too many MQTT connections and disconnections

dakipro commented 5 years ago

exact camera hardware: Xiaomi Xiaofang 1S
the Xiaomi-Dafang-Hacks commitid (from git log) you are experiencing the issue with: Commit #994 1904a3e4c0f7dcc530b094fdab0b3aa19c7740e0

Description

I have a mosquitto mqtt broker which works fine with 5-6 clients, but since I started using the xiaomi hack I get a bunch of connection entries per second in mosquitto MQTT broker from camera like:

1555875521: New connection from 192.168.1.21 on port 1883. 1555875521: New client connected from 192.168.1.21 as mosqpub|3294-DAFANG (c1, k60, u'mosquser'). 1555875521: Client mosqpub|3294-DAFANG disconnected. 1555875521: New connection from 192.168.1.21 on port 1883. 1555875521: New client connected from 192.168.1.21 as mosqpub|3298-DAFANG (c1, k60, u'mosquser'). 1555875521: Client mosqpub|3298-DAFANG disconnected. 1555875521: New connection from 192.168.1.21 on port 1883. 1555875521: New client connected from 192.168.1.21 as mosqpub|3302-DAFANG (c1, k60, u'mosquser'). 1555875521: Client mosqpub|3302-DAFANG disconnected. 1555875521: New connection from 192.168.1.21 on port 1883. 1555875521: New client connected from 192.168.1.21 as mosqpub|3305-DAFANG (c1, k60, u'mosquser'). 1555875521: Client mosqpub|3305-DAFANG disconnected. 1555875521: New connection from 192.168.1.21 on port 1883. 1555875521: New client connected from 192.168.1.21 as mosqpub|3311-DAFANG (c1, k60, u'mosquser'). 1555875521: Client mosqpub|3311-DAFANG disconnected. 1555875521: New connection from 192.168.1.21 on port 1883. 1555875521: New client connected from 192.168.1.21 as mosqpub|3316-DAFANG (c1, k60, u'mosquser'). 1555875521: Client mosqpub|3316-DAFANG disconnected. 1555875522: New connection from 192.168.1.21 on port 1883. 1555875522: New client connected from 192.168.1.21 as mosqpub|3321-DAFANG (c1, k60, u'mosquser'). 1555875522: Client mosqpub|3321-DAFANG disconnected. 1555875522: New connection from 192.168.1.21 on port 1883. 1555875522: New client connected from 192.168.1.21 as mosqpub|3324-DAFANG (c1, k60, u'mosquser'). 1555875522: Client mosqpub|3324-DAFANG disconnected. 1555875522: New connection from 192.168.1.21 on port 1883. 1555875522: New client connected from 192.168.1.21 as mosqpub|3326-DAFANG (c1, k60, u'mosquser'). 1555875522: Client mosqpub|3326-DAFANG disconnected. 1555875522: New connection from 192.168.1.21 on port 1883. 1555875522: New client connected from 192.168.1.21 as mosqpub|3329-DAFANG (c1, k60, u'mosquser'). 1555875522: Client mosqpub|3329-DAFANG disconnected. 1555875522: New connection from 192.168.1.21 on port 1883. 1555875522: New client connected from 192.168.1.21 as mosqpub|3334-DAFANG (c1, k60, u'mosquser'). 1555875522: Client mosqpub|3334-DAFANG disconnected. 1555875522: New connection from 192.168.1.21 on port 1883. 1555875522: New client connected from 192.168.1.21 as mosqpub|3339-DAFANG (c1, k60, u'mosquser'). 1555875522: Client mosqpub|3339-DAFANG disconnected. 1555875522: New connection from 192.168.1.21 on port 1883. 1555875522: New client connected from 192.168.1.21 as mosqpub|3342-DAFANG (c1, k60, u'mosquser'). 1555875522: Client mosqpub|3342-DAFANG disconnected. 1555875522: New connection from 192.168.1.21 on port 1883. 1555875522: New client connected from 192.168.1.21 as mosqpub|3350-DAFANG (c1, k60, u'mosquser'). 1555875522: Client mosqpub|3350-DAFANG disconnected. 1555875522: New connection from 192.168.1.21 on port 1883. 1555875522: New client connected from 192.168.1.21 as mosqpub|3358-DAFANG (c1, k60, u'mosquser'). 1555875522: Client mosqpub|3358-DAFANG disconnected. 1555875523: New connection from 192.168.1.21 on port 1883.

Evidence

I found nothing in all other log files on the camera itself mentioning mqtt, all logs look normal. Services starting and running, not a single mention of mqtt. Any tips where to look for it?

CaptainHook8 commented 3 years ago

I apologize to everyone for being disappeared for a while, I will post a new revised package and make the code available for everybody who wants to improve it and I will try to do it soon.

If the latest package doesn't connect, I guess it might be because the firmware has been updated in the meantime.

2esq commented 3 years ago

@CaptainHook8 did you manage to update your code? Thanks

exetico commented 3 years ago

@CaptainHook8 - Are there any updates? I would like to have this fixed :-D Guess many other would appreciate that, too.

jgkawell commented 3 years ago

Just pinging here to see if there's any update! Love the hard work from everyone and I'm able to test things out if need be.

2esq commented 3 years ago

This issue is still present to this day. Any word from the dev.?

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

exetico commented 3 years ago

No no (cat) bot! 🤖😾

It's NOT stable, if I'm right, but I would like the fix to be pushed to the main-repo, if that's stable.

kevincaradant commented 3 years ago

Hi

I just found this topic today, is there somewhere a fix? Or the project has been abandoned to fix this issue?

Thank you in advance

Regards

firestrife23 commented 3 years ago

Abandon? more like a stalemate due to lack of motivation, loss of interest, or no time to fix it. I pretty much moved on to different cameras that offer true native RTSP and have not looked back since. Unless someone is brave enough to fork this project then I'll be able to give another lease on life for Wyze Cameras that have been collecting dust for months.

alfem commented 3 years ago

So... you are using this awesome Dafang-Hack in your camera to have the freedom of an opensource solution, BUT you install a binary fix without sources?

Mmmm, I think it is not very coherent :-)

EliasKotlyar commented 2 years ago

@firestrife23 @alfem I suppose @CaptainHook8 had a solution for that problem? If he will provide a PR, i can merge it. I must tell that i dont have a Dafang anymore to do it myself, mines broke(after several dumb experiments) and i havent been able to purchase a new one because they stopped producing them

exetico commented 2 years ago

To solve this issue I took comments above and wrote some code to create two new binaries:

mosquitto_pub_client -> it is a daemon which has to be called once, it opens the connection with mosquitto server and then waits for messages to send over to server. This is its help usage:
paco@debian:~/eclipse-workspace/mosquitto_pub_client/Release$ ./mosquitto_pub_client --help
Simple client to mosquitto server
Usage:
  mosquitto_pub_client [OPTION...]

  -H, --host arg      Mosquitto server hostname [required]
  -p, --port arg      Mosquitto server port (default: 1883)
  -U, --username arg  Username
  -P, --password arg  Password
  -h, --help          Print usage
mosquitto_pub_queue -> it is a binary that just enqueues messages for mosquitto_pub_client to send over to mosquito server. This is its help usage:
paco@debian:~/eclipse-workspace/mosquitto_pub_queue/Release$ ./mosquitto_pub_queue --help
Enqueue messages to be sent to mosquitto server
Usage:
  mosquitto_pub_queue [OPTION...]

  -t, --topic arg    MQTT topic [required]
  -m, --message arg  MQTT message [required]
  -r, --retain       Retain MQTT message
  -h, --help         Print usage
I have run some tests on my local machine with a local mosquitto broker (v3.1.1) and everything seems to work fine as expected. I have NOT tested it on the camera because I don't have it available right now (and I don't expect to have it available for the next 2 weeks).

If someone wants to give it a try I can provide the two binaries already cross-compiled for the camera (with some debug prints enabled to clean up eventually) and a slightly modified version of the script "mqtt-status-interval.sh". Of course once tested it on the camera I will make a pull request with my code.

Let me know please if someone would be interested in running some tests. Thanks, Pasquale

@CaptainHook8 - Please share the source in a repo :) That would be great.

CaptainHook8 commented 2 years ago

You guys are all right, I am really sorry for being vanished for so long. This weekend I will try to put everything I wrote in a repo a share with you to make mainstream.

Thank you!

Gastello007 commented 2 years ago

С вами, ребята, все в порядке, мне действительно жаль, что я так долго отсутствовал. в эти выходные я постараюсь поместить все, что я написал, в репо, чтобы поделиться с вами, чтобы сделать мейнстрим.

Спасибо!

We are all here and waiting)

exetico commented 2 years ago

You guys are all right, I am really sorry for being vanished for so long. This weekend I will try to put everything I wrote in a repo a share with you to make mainstream.

Thank you!

How's your progress? I hope you're able to do it soon 👍.

exetico commented 2 years ago

@Gastello007 ?

exetico commented 1 year ago

UPDATE - Wrong tag. Sorry.

Correct tag below:

@CaptainHook8 Did you find the source?

kevincaradant commented 1 year ago

@CaptainHook8 :D

domoticz-italia commented 1 year ago

hi people, may be there is a solution? this is log from my dafang thanks so much 2023-05-19 17:14:44: New connection from 192.168.0.15:41353 on port 1883. 2023-05-19 17:14:44: Client mosqpub|2235-DAFANG disconnected, not authorised. 2023-05-19 17:14:44: New connection from 192.168.0.15:41354 on port 1883. 2023-05-19 17:14:44: Client mosqpub|2241-DAFANG disconnected, not authorised. 2023-05-19 17:14:44: New connection from 192.168.0.15:41355 on port 1883. 2023-05-19 17:14:44: Client mosqpub|2248-DAFANG disconnected, not authorised. 2023-05-19 17:14:44: New connection from 192.168.0.15:41356 on port 1883. 2023-05-19 17:14:44: Client mosqpub|2254-DAFANG disconnected, not authorised. 2023-05-19 17:14:44: New connection from 192.168.0.15:41357 on port 1883. 2023-05-19 17:14:44: Client mosqpub|2260-DAFANG disconnected, not authorised. 2023-05-19 17:14:44: New connection from 192.168.0.15:41358 on port 1883. 2023-05-19 17:14:44: Client mosqpub|2266-DAFANG disconnected, not authorised. 2023-05-19 17:14:45: New connection from 192.168.0.15:41359 on port 1883. 2023-05-19 17:14:45: Client mosqpub|2269-DAFANG disconnected, not authorised. 2023-05-19 17:14:45: New connection from 192.168.0.15:41360 on port 1883. 2023-05-19 17:14:45: Client mosqpub|2277-DAFANG disconnected, not authorised. 2023-05-19 17:14:45: New connection from 192.168.0.15:41361 on port 1883. 2023-05-19 17:14:45: Client mosqsub|457-DAFANG disconnected, not authorised. 2023-05-19 17:14:45: New connection from 192.168.0.15:41362 on port 1883. 2023-05-19 17:14:45: Client mosqpub|2282-DAFANG disconnected, not authorised. 2023-05-19 17:14:45: New connection from 192.168.0.15:41363 on port 1883. 2023-05-19 17:14:45: Client mosqpub|2294-DAFANG disconnected, not authorised. 2023-05-19 17:14:45: New connection from 192.168.0.15:41364 on port 1883. 2023-05-19 17:14:45: Client mosqpub|2306-DAFANG disconnected, not authorised. 2023-05-19 17:14:46: New connection from 192.168.0.15:41365 on port 1883. 2023-05-19 17:14:46: Client mosqpub|2338-DAFANG disconnected, not authorised.

derekoharrow commented 1 year ago

Not so much a solution, but a way of easing the problem - turn off connection logging for MQTT (if you trust your connections) by putting this in the mqtt configuration file:

connection_messages false

radinsky commented 1 year ago

Not so much a solution, but a way of easing the problem - turn off connection logging for MQTT (if you trust your connections) by putting this in the mqtt configuration file:

connection_messages false

Where should you put it?

radinsky commented 1 year ago

ok, found this: https://github.com/EliasKotlyar/Xiaomi-Dafang-Hacks/issues/1016#issuecomment-575929807

GeorgBraun commented 9 months ago

Hi all, I'm not sure if this is of any help, but today I learned how to run the mosquitto_pub Linux binary in sort of a "demon mode" or as kind of a "publisher service":

Create a named pipe and start publisher service

$ mkfifo pubPipe
$ tail -f pubPipe | mosquitto_pub -h 127.0.0.1 -t namedPipeTest --stdin-line

Notes:

The topic has to be fixed, unfortunately. For different topics, the above could be repeated with different pipe names.
The tail -f is important, as it ignores EOF by design (mosquitto_pub doesn't)

Use it in a second terminal

cd into the folder where pubPipe resides, then:

$ echo hello mqtt 1 >pubPipe
$ echo hello mqtt 2 >pubPipe

If you want to see some CPU load on your system (be careful):

$ for i in {0..1000}; do echo Test_$i >pubPipe; done

You can also pipe the stdout of a program into pubPipe. The delimiter for new messages would be the NL. Use some encoding in case of binary data.

I've tested this on an RPi 3B+ with Raspberry Pi OS based on Debian 11.8 and mosquitto 2.0.11.

EliasKotlyar / Xiaomi-Dafang-Hacks

Too many MQTT connections and disconnections #1016

Description

Evidence