Sensitivity drop with latest version used in OpenMQTTGateway

[Dattel]

Hi there, i just updated to the latest development version of openmqttgateway, which references rtl_433_ESP in version 0.1.3.

I have some issue with my Nexus-T temperature sensor, which has been captured fine before the update but now the signal seems to be ignored. If i put the sensor closer to the receiver, the signal is captured. i there some build flag i can tweek? I'm a bit confused about the new RSSI_THRESHOLD and AVERAGE_RSSI (which seems not to be used anywhere from rtl_433_esp)

That's what the sensor normally sends out:

{ "model": "Nexus-T", "id": 167, "channel": 1, "battery_ok": 1, "temperature_C": 22.1, "protocol": "Nexus, FreeTec NC-7345, NX-3980, Solight TE82S, TFA 30.3209 temperature/humidity sensor", "rssi": -53, "duration": 956000 }

thanks in advance

[DigiH]

Hi @Dattel , I have been in touch with @NorthernMan54 about the recent new changes in rtl_433_ESP, with radiolib and a particular rtl_433 issue with my LaCrosse-TX141THBv2. While the LaCross decoder specific changes are not applicable to your Nexus-T, I also found that the other signal change improved the reception of my neighbours Nexus-TH.

Not quite your model, but it might be worth you also trying the changes in my current test branch, by changing the rtl_433_ESP URL to

rtl_433_ESP = https://github.com/DigiH/rtl_433_ESP#new-signal-plus-knarrff

in platformio.ini.

Let us know how you get on.

[Dattel]

Hi @DigiH , thanks for your quick reply. i just compiled with your lib and tested the sender. Same here: recognized in close distance, ignored one room away (where it was recognized before with the older versions) Must be somethinge different..

I also have a LaCrosse-TX141THBv2 (what is actually a TFA-Dostmann) ... it works flawless without any problems.... before and after the update

[DigiH]

Shame about your Nexus-T, as the changes I tried with @NorthernMan54 really improved the neighbour's Nexus-TH reception big time for me, and his Nexus is quite a bit further away than just in next room. Must be something else you are seeing. :(

I also have a LaCrosse-TX141THBv2... it works flawless without any problems.... before and after the update

Don't you ever get it to be wrongly recognised as a TFA-303221?

This has been an ongoing issue with trl_433(_esp) for a while now

https://github.com/merbanan/rtl_433/issues/1652

https://github.com/merbanan/rtl_433/issues/2206

and as you can see from my comments there, this has, at least for me, dramatically improved with the changes in my current test branch.

[DigiH]

@Dattel , one other test branch with some different changes might be worth also trying

rtl_433_ESP = https://github.com/DigiH/rtl_433_ESP#signal-only

[Dattel]

just flashed it ... will give feedback later since i'm on the go now..

[Dattel]

Update: seems to work better.. NEXUS-T Readings are coming in from the same distance as before with this build.. Do you have a clue, what's the major difference?

[NorthernMan54]

@Dattel Myself and @DigiH had been working on some fine tuning of the signal reception settings. I had completed testing with my devices, and had seen similar performance with the change, but @DigiH had reported degraded reception. We then went thru each of the changes individually to identify the one that caused degraded reception. Under the covers the change was to use a transceiver specific end of transmission gap value. It seems that the CC1101 and SX127X handle RSSI differently at the end of a transmission.

[NorthernMan54]

FYI - This is the final version of the proposed change - https://github.com/NorthernMan54/rtl_433_ESP/pull/43

[Dattel]

is there anything i can provide to help`?

@DigiH: https://github.com/NorthernMan54/rtl_433_ESP/issues/19 <-- That's my issue from earlier times... i have a TFA which is recognized as LaCrosse, i think because they both speak the same coding. Earlier the signal got parsed sometimes as LaCrosse or TFA... but i remembered, that it only happend, if you manual transmit or power on the device because of multiple signals in a row.

[NorthernMan54]

If your up for it, you can try a build with the pull request included ( I do realize this could be a bit complex ). But feedback from the testing with DigiH's suggestion is good.

[Dattel]

is there a way to define these version with pullrequest in platformio?

[DigiH]

FYI - This is the final version of the proposed change - #43

This has the change as in my test branch new-signal-plus-knarrff above

else if (micros() - signalEnd < PD_MAX_GAP_MS)

which didn't remedy @Dattel's reception issue, only with the previous test branch with its

else if (micros() - signalEnd < 40000 && micros() - signalStart > 30000)

did it work for him - if I understood everything correctly above.

For me both changes saw improved reception.

[NorthernMan54]

@Dattel - make this change to your ini file

  https://github.com/NorthernMan54/rtl_433_ESP.git#CC1101-Tweaks
;  ${libraries.rtl_433_ESP}

[Dattel]

@NorthernMan54 currently looking good with your suggested fix

[Dattel]

Happy too soon... left the device over night - no signal from nexus-t since 8 hours captured. I just reverted back to OpenMqttGateway 0.9.16 with rtl_433_ESP = https://github.com/NorthernMan54/rtl_433_ESP.git#v0.0.4 to ensure, that its a firmware related problem.

Captured 4 signals immediately in a few minutes...

[NorthernMan54]

@Dattel I revered back the signal gap change, can you try it again

[Dattel]

doesn't work either.. :-(

[NorthernMan54]

I'm curious on how close the signal reception threshold your devices are, and what the signal reception threshold your OMG is using

In the OMG status message, it will share details of the OMG RSSI threshold, can you share this ( This is from MQTT Explorer )

And the device message will also contain the signal RSSI

In my case, this device is right on the edge for being able to receive a signal ( -81 ), hence I only receive sporadic updates.

[Dattel]

Between my experiements, i downgrade the device to OpenMqttgateway 0.9.16 - so currently i only can provide received messaged from there.

Here are some samples... Device posts updates very frequently

[DigiH]

Thanks @Dattel, to be able to pin the issue down though, and hopefully get it nicely addressed and sorted for OMG 1.0.0 and future versions, would you mind giving 1.0.0 with the previously successful

rtl_433_ESP = https://github.com/DigiH/rtl_433_ESP#signal-only

another try. Especially to see if it keeps receiving the Nexus-TH when it is running over night?

Then also posting some of the above rssi values of the received messages, and also the different rssi related values from the SYStoMQTT message of the gateway.

Many thanks

[Dattel]

ok, just flashed it...

one question... how can i detect, if the correct LIB is used in VS.Code? To be sure, i always remove the rtl_433_ESP directory from the .pio dir when switching to another branch. But i'm not sure if that is neccesary.

Are you addressing the deaf-state of the chip, when you asking for overnight messages? I'm already using the bugfix for the chip because i was one of the first, who asked with these problem here as issue :-D

I'll come back later hopefully with some captured values.

[DigiH]

one question... how can i detect, if the correct LIB is used in VS.Code? To be sure, i always remove the rtl_433_ESP directory from the .pio dir when switching to another branch. But i'm not sure if that is neccesary.

Once you change the library URL in platformio.ini the new referenced library should be downloaded and be used. To be on the completely safe side though, especially when testing different library branches, I usually also use the Clean All/Clean tasks of the environment. If you do that while watching the .pio directory you should see that this is doing exactly what you did manually - well, it not only deletes rtl_433_ESP, but all libraries, to make sure that for a new build all libraries will be downloaded afresh, making sure they are the latest.

Also when watching the build process, you will see which changed librar(y|ies) is/are being downloaded.

Are you addressing the deaf-state of the chip, when you asking for overnight messages?

Yes, to hopefully see if this slightly different branch does not have any deaf-state after an overnight run.

Thanks

[Dattel]

so i think, i can stop... doesn't work very vell The capture is running since over 3 hours and nothing really received from Nexus-T but the temperature changed in the meantime.

The RSSI is at -71 for all measurements

Other devices are working... please notice the high number of "unknown" signals. Not sure, if that fits together.

Here some systomqtt

{
  "uptime": 5167,
  "version": "v1.0.0-development",
  "freemem": 174812,
  "mqttport": "1883",
  "mqttsecure": false,
  "freestack": 5736,
  "rssi": -61,
  "SSID": "XXXXXX",
  "BSSID": "XXXXXXX",
  "ip": "192.168.178.77",
  "mac": "XXXXXXXXX",
  "actRec": 3,
  "mhz": 433.92,
  "RTLRssiThresh": -66,
  "RTLRssi": -74,
  "RTLAVGRssi": -75,
  "RTLCnt": 297,
  "modules": [
    "RF",
    "rtl_433"
  ]
}

{
  "uptime": 5047,
  "version": "v1.0.0-development",
  "freemem": 174812,
  "mqttport": "1883",
  "mqttsecure": false,
  "freestack": 5736,
  "rssi": -62,
  "SSID": "XXXXXX",
  "BSSID": "XXXXXX",
  "ip": "192.168.178.77",
  "mac": "XXXXXX",
  "actRec": 3,
  "mhz": 433.92,
  "RTLRssiThresh": -65,
  "RTLRssi": -72,
  "RTLAVGRssi": -74,
  "RTLCnt": 292,
  "modules": [
    "RF",
    "rtl_433"
  ]
}

{
  "uptime": 4927,
  "version": "v1.0.0-development",
  "freemem": 174812,
  "mqttport": "1883",
  "mqttsecure": false,
  "freestack": 5736,
  "rssi": -64,
  "SSID": "XXXXX",
  "BSSID": "XXXXX",
  "ip": "192.168.178.77",
  "mac": "XXXXXX",
  "actRec": 3,
  "mhz": 433.92,
  "RTLRssiThresh": -66,
  "RTLRssi": -74,
  "RTLAVGRssi": -75,
  "RTLCnt": 286,
  "modules": [
    "RF",
    "rtl_433"
  ]
}

[Dattel]

just a 30 Minutes with the reverted openmqttgateway... As you can see, there are some diffenent type of devices also shown up.. I'm not sure, if that are real devices in the neighborhood or missinterpreted noises-

[DigiH]

Thanks for the testing @Dattel . I hope this will give @NorthernMan54 more info on pinning down your issue.

[NorthernMan54]

@Dattel Dattel, tks for suppling that data. Looking at this data, it shows that the calculated RSSI Signal threshold is -66, meanwhile your sensor signal strength is -71. This explains why the signal is not being received.

I'm going to add an option to remove the calculated RSSI Signal threshold function, should have a new release for testing in about 24 hours.

[Dattel]

This is worth a try, but i'm wondering if there is something wrong with the calculation of the RSSI Signal threshold. As you can see in my provided screenshot Nexus-T/LaCrosse/Prologue are sending messages in a similar frequence. So why does the "autosense" kicks out a frequent sender.

So that would mean, if a have a sender a few centemeters beside the receiver sending with the same clockfrequency as a far away sender, these far away sender will be ignored over time because of the average rssi threshold?

[NorthernMan54]

Some background on the recent change

We added support for the SX127x transceiver chipset by switching to the RadioLib library as it offered support for both the CC1101 transceiver and the SX127x chipset. And as part of the change we needed to change the algorithm used to determine if a signal is present as the existing method did not work with the SX127x.

The old method, continuously polled the transceiver for the current RSSI value, and if it was above MINRSSI ( defaults to -82 ) it would start receiving a signal, and when the RSSI value dropped below MINRSSI it would stop receiving a signal and send it for processing. The only method of changing the MINRSSI was to recompile the binary.

The new method is an enhancement on the above, where the code is averaging the current RSSI value RTLAVGRssi and applying a delta RSSI_THRESHOLD to the average to calculate the RTLRssiThresh that is used to determine start and end of signal. This method of applying a delta to the average RSSI is pretty standard approach in transceiver chipsets.

I'm thinking that your environment may have a lot of background RF noise, and it is causing the average calculation to use a higher value.

[Dattel]

thanks for your detailed informations. let me know, if i can pull a new version for testing

[NorthernMan54]

Both myself and DigiH did some testing, and are feeling pretty good about this version. You can install it by using this version of the rtl_433_ESP library https://github.com/NorthernMan54/rtl_433_ESP.git#CC1101-Tweaks, cleaning your environment then rebuilding.

With this version we have introduced a new compiler directive DISABLERSSITHRESHOLD if you define this directive it will disable the new RSSI averaging signal threshold. ie enable to older behaviour.

[Dattel]

looks really promising let it running for about 30 min now and readings seems to be fine. Got a lot of unknown-categorised data but i think that could be ignored.

{
  "uptime": 2408,
  "version": "v1.0.0-development",
  "freemem": 174936,
  "mqttport": "1883",
  "mqttsecure": false,
  "freestack": 5784,
  "rssi": -58,
  "SSID": "xxxxx",
  "BSSID": "xxxxxx",
  "ip": "192.168.178.77",
  "mac": "xxxxxxx",
  "actRec": 3,
  "mhz": 433.92,
  "RTLRssiThresh": -82,
  "RTLRssi": -84,
  "RTLAVGRssi": -74,
  "RTLCnt": 246,
  "modules": [
    "RF",
    "rtl_433"
  ]
}

[Dattel]

Would like to give some feedback.. Device is running and receiving data since yesterday.. Thanks a lot for your help... i think the issue can be closes :-D

[NorthernMan54]

Great news I will publish this as the latest version

Did you use DISABLERSSITHRESHOLD ?

[NorthernMan54]

This has now been published as release v0.1.4

[Dattel]

yes, i used DISABLERSSITHRESHOLD ...
i assume a constant value "RTLRssiThresh": -82 indicated that proper working, right?

[NorthernMan54]

Correct

well I’m closing this as resolved, thanks for your support in this

[Dattel]

always my pleasure :-D