search

NorthernMan54 / rtl_433_ESP

Trial port of the rtl_433 Library for use with OpenMQTTGateway on a ESP32 and a CC1101 Transceiver
GNU General Public License v3.0
501 stars 115 forks source link

915MHz with Lilygo LoRA32 v2.1 #87

Closed antamy closed 1 year ago

antamy commented 1 year ago

Hi, just getting into this with the hope of getting my ecowitt temperature / humidity sensors recognized.

I have this working with the regular rtl_433 code and an SDR, albeit with a small mod to fineoffset.c which I applied to the code in this repository. I am not able to pick up the messages with the Lilygo.

My output is:

N: ****** setup ******
rtl_433_ESP(6): SX1276 gpio receive pin: 32
rtl_433_ESP(6): SX1276 receive frequency: 915.000000
rtl_433_ESP(6): # of device(s) configured 80
rtl_433_ESP(6): ssizeof(r_device): 112
rtl_433_ESP(6): cfg->devices size: 8960
----- SX127x Status -----
RegOpMode: 0x0c
RegPacketConfig1: 0x00
RegPacketConfig2: 0x00
RegBitrateMsb: 0x07
RegBitrateLsb: 0x40
RegRxBw: 0x12
RegAfcBw: 0x02
-------------------------
RegLna: 0x20
RegRxConfig: 0x08
RegRssiConfig: 0x00
-------------------------
RegDioMapping1: 0x00
----------- FSK --------------
FDEV_MSB: 0x02
FDEV_LSB: 0x8f
----- SX127x Status -----
N: ****** setup complete ******
                               ----- SX127x Status -----
RegOpMode: 0x0c
RegPacketConfig1: 0x00
RegPacketConfig2: 0x00
RegBitrateMsb: 0x07
RegBitrateLsb: 0x40
RegRxBw: 0x12
RegAfcBw: 0x02
-------------------------
RegLna: 0x20
RegRxConfig: 0x08
RegRssiConfig: 0x00
-------------------------
RegDioMapping1: 0x00
----------- FSK --------------
FDEV_MSB: 0x02
FDEV_LSB: 0x8f
----- SX127x Status -----
rtl_433_ESP(6): Ignored Signal length: 5, Time since last bit length: 6051, Gap length: 1062787, Signal RSSI: -59, Current RSSI: -107, pulses: 24, noise count: 7

This is after resetting the device. After the last message, there is a 15-20 second pause, then the Ignored messages just stream in:

rtl_433_ESP(6): Ignored Signal length: 1, Time since last bit length: 6051, Gap length: 98478, Signal RSSI: -99, Current RSSI: -108, pulses: 26, noise count: 2
rtl_433_ESP(6): Ignored Signal length: 1, Time since last bit length: 6051, Gap length: 169479, Signal RSSI: -99, Current RSSI: -101, pulses: 25, noise count: 1
rtl_433_ESP(6): Ignored Signal length: 2, Time since last bit length: 6051, Gap length: 93467, Signal RSSI: -98, Current RSSI: -109, pulses: 25, noise count: 2
rtl_433_ESP(6): Ignored Signal length: 2, Time since last bit length: 6051, Gap length: 127478, Signal RSSI: -99, Current RSSI: -103, pulses: 24, noise count: 1
rtl_433_ESP(6): Ignored Signal length: 2, Time since last bit length: 6051, Gap length: 211456, Signal RSSI: -99, Current RSSI: -120, pulses: 28, noise count: 1

I'm not sure how to interpret this - there is a carrier, there are pulses, and significant gaps. I also see (not shown) many instances of single digit signal lengths, then an occasional 4 digit signal length, but nothing in between.

I am using the following build flags:

board = ttgo-lora32-v21 ; ~/.platformio/packages/framework-arduinoespressif32/variants/.../pins_arduino.h
build_flags = 
  '-DLOG_LEVEL=LOG_LEVEL_TRACE'
  '-DONBOARD_LED=LED_BUILTIN'          ; Onboard LED is GPIO 25 on the Heltec Board
  '-DRF_MODULE_FREQUENCY=915.00'  
 '-DDEMOD_DEBUG=true'  ; display signal debug info
  '-DOOK_MODULATION=false'       ; False is FSK, True is OOK
  '-DRF_MODULE_INIT_STATUS=true'    ; Display transceiver config during startup

One big difference between my working rtl_433 setup and this is that the former uses a sample rate of 1000000Hz. I've tried setting #define DEFAULT_SAMPLE_RATE 1000000 in rtl_433.h but it doesn't appear to have any effect.

Let me know if anything comes to mind. I can start digging in the code, but would need some pointers as to where to look!

Thanks

NorthernMan54 commented 1 year ago

The log is showing pulses being detected, but they are very weak ( signal rssi ). Also your config shows fsk mode, is this what you device uses?

antamy commented 1 year ago

The log is showing pulses being detected, but they are very weak ( signal rssi ). Also your config shows fsk mode, is this what you device uses?

Yes, the device maps to this in fineoffset.c:

Fine Offset Electronics WH25 / WH32B Temperature/Humidity/Pressure sensor protocol.

The sensor sends a package each ~64 s with a width of ~28 ms. The bits are PCM modulated with Frequency Shift Keying.

Seems my antenna needs some attention. I'm just using the tiny one that came with the board and my device is only a few feet away in the same room. I'll see if I can rig something up and will report back.

NorthernMan54 commented 1 year ago

During testing I found that a vertical orientation worked best.

antamy commented 1 year ago

During testing I found that a vertical orientation worked best.

Yes, I've tried all orientations and even extending the antenna away from the board on a cable. It's odd that I see a couple of log entries initially with RSSI around -30dbM, then everything else is around -100dbM.

antamy commented 1 year ago

I bought an antenna that was recommended by the meshtastic group, but that made no difference. Seems that I may need another board, since with only one I can't test it with basic send and receive code to make sure the device is working in other scenarios.

antamy commented 1 year ago

I've looked into this further and there is something going on within all the noise.

My device sends out a signal roughly every 64 seconds. I do see a burst of activity that coincides with that time period:

rtl_433_ESP(6): Signal length: 9000, Gap length: 2191483, Signal RSSI: -29, train: 1, messageCount: 261, pulses: 64
rtl_433_ESP(6): # of messages decoded 0
rtl_433_ESP(6): Signal length: 9000, Gap length: 29783, Signal RSSI: -28, train: 0, messageCount: 262, pulses: 63
rtl_433_ESP(6): # of messages decoded 0

Is there a debug switch I can use to display what is being received? When I was looking at this with rtl_433 I would use the -vv switch and would see something like:

codes : {170}d5aaaaaaaaaaaa2dd4d122553cffff825c000000000

I have applied the code fix from https://github.com/merbanan/rtl_433/issues/2303, which was needed to recognize / decocde my device. If I can confirm that the TTGO is receiving the data, then I can start to track down the decoding.

NorthernMan54 commented 1 year ago

For more information on the signal, DEMOD_DEBUG is one option to try

Or these

RTL_VERBOSE=## ; Enable RTL_433 device decoder verbose mode, ## is the decoder # from the appropriate memcpy line in rtl_433_ESP.cpp RTL_ANALYZER ; Enable pulse stream analysis ( note is very resource intensive and will not work with other modules ) RTL_ANALYZE=## ; Enable pulse stream analysis for decoder ##

antamy commented 1 year ago

ok, getting somewhere now. Not exactly sure where, but this is what I am seeing with -DRTL_VERBOSE=18 (which corresponds to fineoffset_WH25, which is the decoder for my device) and -DRTL_ANALYZER=TRUE.

rtl_433_ESP(6): # of messages decoded 0
pulse_slicer_pcm: Exact bit width (in us) is 58.11 vs 58.00, 44 bit preamble
pulse_slicer_pcm: Fine Offset Electronics, WH25, WH32B, WH24, WH65B, HP1000, Misol WS2320 Temperature/Humidity/Pressure Sensor 
codes [{256}5aaaaaaaaaaaa2dd4d122883bffffb45c000000001948a323de092b061bf9bb4]
Analyzing pulses...
Total count:   67,  width: 14.89 ms             (14886 S)
Pulse width distribution:
...
Gap width distribution:
...
Pulse period distribution:
...
Pulse timing distribution:
...
Guessing modulation: No clue...

This isn't consistent, but the variants of it I see all contain the data from my device. However, with rtl_433, what I see is

{170}d5aaaaaaaaaaaa2dd4d122553cffff825c000000000

and this gets decoded correctly. This is from the most current version on git with the fineoffset.c fix applied.

I did, once, see the correct message, albeit too long.

Decoder 18 doesn't fire for this message, but does for others, and gives links to triq.org, but I haven't pursued those (I don't have any other devices).

Not sure how to proceed - the data is being received but is truncated at the beginning and too long overall!

NorthernMan54 commented 1 year ago

If you remove this directive, -DRTL_ANALYZER=TRUE it will be less chatty, and should only give verbose info for your device.

Looking at the data you shared, this looks close

codes [{256}5aaaaaaaaaaaa2dd4d122883bffffb45c000000001948a323de092b061bf9bb4]

to this

{170}d5aaaaaaaaaaaa2dd4d122553cffff825c000000000

The first bit is being truncated, and it doesn't appear to detect end of signal correctly.

d     5     a     a
1101 0101 1010 1010

Looks like the first couple of bits are being missed

antamy commented 1 year ago

If you remove this directive, -DRTL_ANALYZER=TRUE it will be less chatty, and should only give verbose info for your device.

Looking at the data you shared, this looks close

codes [{256}5aaaaaaaaaaaa2dd4d122883bffffb45c000000001948a323de092b061bf9bb4]

to this

{170}d5aaaaaaaaaaaa2dd4d122553cffff825c000000000

The first bit is being truncated, and it doesn't appear to detect end of signal correctly.

d     5     a     a
1101 0101 1010 1010

Looks like the first couple of bits are being missed

Exactly my point. The data is there, just not all there. I will try again today with the stock antenna and see if I can get a hit. The radio and the device are in the same room.

antamy commented 1 year ago

I've had this running for a few hours with different antennas. I have not been able to pick up a well-formed signal from my device. I see a lot of stuff that I don't see when running my SDR, and I see the signal (the ones with all a's) from my device but it is often missing the first 4 bits and is always too long. Otherwise, it contains the correct data. I'm officially stuck!

pulse_slicer_pcm: Fine Offset Electronics, WH25, WH32B, WH24, WH65B, HP1000, Misol WS2320 Temperature/Humidity/Pressure Sensor codes [{339}802001000006bf40000000000014440000000000000000000000000105c00000000028b3a248462bbe9e0]
rtl_433_ESP(6): # of messages decoded 0
rtl_433_ESP(6): Signal length: 9000, Gap length: 955810, Signal RSSI: -56, train: 1, messageCount: 231, pulses: 57
pulse_slicer_pcm: Exact bit width (in us) is 57.43 vs 58.00, 44 bit preamble
pulse_slicer_pcm: Fine Offset Electronics, WH25, WH32B, WH24, WH65B, HP1000, Misol WS2320 Temperature/Humidity/Pressure Sensor codes [{240}aaaaaaaaaaa8b753448890e7fffd3970000000001c53c0c355fe3851d88f]
rtl_433_ESP(6): # of messages decoded 0
rtl_433_ESP(6): Signal length: 10000, Gap length: 28815, Signal RSSI: -56, train: 0, messageCount: 232, pulses: 67
pulse_slicer_pcm: Exact bit width (in us) is 58.00 vs 58.00, 44 bit preamble
pulse_slicer_pcm: Fine Offset Electronics, WH25, WH32B, WH24, WH65B, HP1000, Misol WS2320 Temperature/Humidity/Pressure Sensor codes [{257}0ab5555555555545ba9a2444873fffe9cb80000000012475df0056d291c7da260]
rtl_433_ESP(6): # of messages decoded 0
rtl_433_ESP(7): Average RSSI Signal -126 dbm, adjusted RSSI Threshold -117, samples 50000
rtl_433_ESP(6): Signal length: 9000, Gap length: 31272815, Signal RSSI: -98, train: 1, messageCount: 233, pulses: 39
pulse_slicer_pcm: Fine Offset Electronics, WH25, WH32B, WH24, WH65B, HP1000, Misol WS2320 Temperature/Humidity/Pressure Sensor codes [{243}80000000000013a7fcee9604b439e20600000000852119373ec1ae2b8874a]
rtl_433_ESP(6): # of messages decoded 0
rtl_433_ESP(6): Signal length: 9000, Gap length: 32654813, Signal RSSI: -56, train: 0, messageCount: 234, pulses: 64
pulse_slicer_pcm: Exact bit width (in us) is 57.84 vs 58.00, 44 bit preamble
pulse_slicer_pcm: Fine Offset Electronics, WH25, WH32B, WH24, WH65B, HP1000, Misol WS2320 Temperature/Humidity/Pressure Sensor codes [{243}5aaaaaaaaaaaa2dd4d1222439ffff4e5c000000000702d5c441b04a5e5a2c]
rtl_433_ESP(6): # of messages decoded 0
rtl_433_ESP(6): Signal length: 8999, Gap length: 29815, Signal RSSI: -57, train: 1, messageCount: 235, pulses: 61
pulse_slicer_pcm: Exact bit width (in us) is 57.22 vs 58.00, 46 bit preamble
pulse_slicer_pcm: Fine Offset Electronics, WH25, WH32B, WH24, WH65B, HP1000, Misol WS2320 Temperature/Humidity/Pressure Sensor codes [{236}aaaaaaaaaaaa2dd4d1222439ffff4e5c00000000002a846530705d4a357]
rtl_433_ESP(6): # of messages decoded 0
rtl_433_ESP(7): Average RSSI Signal -126 dbm, adjusted RSSI Threshold -117, samples 50000
rtl_433_ESP(6): Signal length: 10000, Gap length: 22088817, Signal RSSI: -111, train: 0, messageCount: 236, pulses: 59
pulse_slicer_pcm: Fine Offset Electronics, WH25, WH32B, WH24, WH65B, HP1000, Misol WS2320 Temperature/Humidity/Pressure Sensor codes [{259}ff7bdaf3acf0b48e79d73215640d8aa38840389b87448848200d040127f9bd450]
rtl_433_ESP(6): # of messages decoded 0
rtl_433_ESP(6): Signal length: 13000, Gap length: 3648811, Signal RSSI: -115, train: 1, messageCount: 237, pulses: 30
pulse_slicer_pcm: Fine Offset Electronics, WH25, WH32B, WH24, WH65B, HP1000, Misol WS2320 Temperature/Humidity/Pressure Sensor codes [{313}0000000000000000000280000000001000000000015b48000000000001aa1182d89c24f33fa3508]
rtl_433_ESP(6): # of messages decoded 0
rtl_433_ESP(7): Average RSSI Signal -126 dbm, adjusted RSSI Threshold -117, samples 50000
rtl_433_ESP(6): Signal length: 9000, Gap length: 38170812, Signal RSSI: -56, train: 0, messageCount: 238, pulses: 64
pulse_slicer_pcm: Exact bit width (in us) is 57.93 vs 58.00, 44 bit preamble
pulse_slicer_pcm: Fine Offset Electronics, WH25, WH32B, WH24, WH65B, HP1000, Misol WS2320 Temperature/Humidity/Pressure Sensor codes [{242}e55aaaaaaaaaaaa2dd4d1222439ffff4e5c0000000002a45b41d7ea6cdd00]
rtl_433_ESP(6): # of messages decoded 0
NorthernMan54 commented 1 year ago

Looking at the fineoffest decoder logic, it looks for a preamble of {0xAA, 0x2D, 0xD4}, which is present. So am thinking that the issue is receiver on time. The ones with the preamble appear to have a signal length of 9000.

rtl_433_ESP(6): Signal length: 9000, Gap length: 32654813, Signal RSSI: -56, train: 0, messageCount: 234, pulses: 64
pulse_slicer_pcm: Exact bit width (in us) is 57.84 vs 58.00, 44 bit preamble
pulse_slicer_pcm: Fine Offset Electronics, WH25, WH32B, WH24, WH65B, HP1000, Misol WS2320 Temperature/Humidity/Pressure Sensor codes [{243}5aaaaaaaaaaaa2dd4d1222439ffff4e5c000000000702d5c441b04a5e5a2c]

There is logic in rtl_433_ESP.cpp to keep listening for a minimum time after the end of the signal

#if defined(RF_SX1276) || defined(RF_SX1278)
      // If we received a signal but had a minor drop in strength keep the
      // receiver running for an additional 150,000
      else if (micros() - signalEnd < PD_MAX_GAP_MS * 1000)
#else

You could try tuning PD_MAX_GAP_MS, to see if that improves reception.

antamy commented 1 year ago

I'm about to head out of town for a few days so will look at this when I return. I did a quick scan through the code and only see PD_MAX_GAP_MS in one place which is commented out and wouldn't be included with RF_SX1276:

#if defined(RF_SX1276) || defined(RF_SX1278)
      // If we received a signal but had a minor drop in strength keep the
      // receiver running for an additional 150,000
      else if (micros() - signalEnd < MINIMUM_SIGNAL_LENGTH)
#else
      // If we received a signal but had a minor drop in strength keep the
      // receiver running for an additional 40,000
      else if (micros() - signalEnd < MINIMUM_SIGNAL_LENGTH && micros() - signalStart > 30000)
      // else if (micros() - signalEnd < PD_MAX_GAP_MS)
#endif

Are you saying I should replace else if (micros() - signalEnd < MINIMUM_SIGNAL_LENGTH) with else if (micros() - signalEnd < PD_MAX_GAP_MS * 1000) or am I missing something? Also PD_MAX_GAP_MS is not defined anywhere.

NorthernMan54 commented 1 year ago

I was looking at an older version earlier, what you included is current. Sorry about that

Try tuning and reducing MINIUM_SIGNAL_LENGTH

antamy commented 1 year ago

I was looking at an older version earlier, what you included is current. Sorry about that

Try tuning and reducing MINIUM_SIGNAL_LENGTH

I guess there's no real science to this? I just dropped it to 500 and it worked immediately:

pulse_slicer_pcm: Fine Offset Electronics, WH25, WH32B, WH24, WH65B, HP1000, Misol WS2320 Temperature/Humidity/Pressure Sensor codes [{172}aaaaaaaaaaaa2dd4d122233affff4e5c00000000103]
rtl_433_ESP(6): # of messages decoded 1
rtl_433_ESP(6): Signal length: 9001, Gap length: 33720, Signal RSSI: -56, train: 1, messageCount: 1, pulses: 44
pulse_slicer_pcm: Exact bit width (in us) is 58.07 vs 58.00, 42 bit preamble
rtl_433_ESP(6): data_output {"model":"Fineoffset-WH32","id":18,"battery_ok":1,"temperature_C":14.7,"humidity":58,"mic":"CRC","protocol":"Fine Offset Electronics, WH25, WH32B, WH24, WH65B, HP1000, Misol WS2320 Temperature/Humidity/Pressure Sensor","rssi":-56,"duration":9001}
N: Received message : {"model":"Fineoffset-WH32","id":18,"battery_ok":1,"temperature_C":14.7,"humidity":58,"mic":"CRC","protocol":"Fine Offset Electronics, WH25, WH32B, WH24, WH65B, HP1000, Misol WS2320 Temperature/Humidity/Pressure Sensor","rssi":-56,"duration":9001}

Since this is the only FSK device I'm interested in, I'll leave it as is.

Now to figure out if this will work with OpenMQTT ...

antamy commented 1 year ago

I'm happy to report that after grabbing OpenMQTTGateway, replacing the fineoffset.c file, enabling FSK, and modifying MINIMUM_SIGNAL_LENGTH, I was able to see my device and get MQTT messages. In the OpenMQTTGateway code, I did also set CC1101_FREQUENCY in config_RF.h to 915 - that seemed to work, but I'm not sure if that was the correct thing to do.

Thanks for your help getting me up and running!

allistermaguire commented 1 year ago

Changing MINIMUM_SIGNAL_LENGTH fixed the issue I was having with my Ecowitt WS80 (Fineoffset). With the default value, it would only decode a signal every hour or so instead of every 4.5 seconds.

I thought it might have been a poor signal as I just have the standard stubby one that came with the LIlygo LaRA32, but thought I would try this as the WH31E and WH40 had no issue.

NorthernMan54 commented 1 year ago

Any recommendations on an updated value to use?

NorthernMan54 commented 1 year ago

Also just reviewing my code and my variable naming is a bit incorrect

This calculation

micros() - signalEnd < MINIMUM_SIGNAL_LENGTH

Is the MINIMUM SIGNAL GAP LENGTH ( how long to wait after a signal has finished before sending it for processing ) and not the signal length , which is micros() - signalStart aka time since start of signal.

antamy commented 1 year ago

Is there a different variable that I should try? Seems odd that changing a timing fixed the issue.

allistermaguire commented 1 year ago

I think he is just saying the variable MINIMUM_SIGNAL_LENGTH is the incorrect name, should be something like MINIMUM_SIGNAL_GAP_LENGTH.

From a tuning perspective, I am not 100% sure how this works; but assuming if the value is too low, data could be lost/truncated, and too high additional data would be collected by the receiver?

Entropy512 commented 3 months ago

Also just reviewing my code and my variable naming is a bit incorrect

After looking through your code, it looks like instead of renaming the variable, a new one should be created as this one is being used for two different purposes.

There's also this, for which MINIMUM_SIGNAL_LENGTH is the appropriate name:

          if ((_nrpulses > PD_MIN_PULSES) &&
              ((signalEnd - signalStart) >
               MINIMUM_SIGNAL_LENGTH)) // Minimum signal length of MINIMUM_SIGNAL_LENGTH MS
          {

In general, I would expect that MINIMUM_SIGNAL_LENGTH would depend on the exact protocol being targeted. A high-bitrate protocol is going to have shorter messages that would be problematic, but if your target is a low-bitrate protocol, it will be too low.

In general, I really don't think targeting more than one protocol at a time is going to work well due to the restrictive nature of the receivers we're working with - rtl_433 with an SDR has a lot more data available to it for decisionmaking. The exception might be protocols that are extremely similar (close to each other in length, same bitrate or very similar).

Entropy512 commented 3 months ago

I have a rough proof of concept at https://github.com/Entropy512/rtl_433_ESP/commits/neptune_r900/ - I am now able to successfully receive messages from my water meter when I wasn't able to before. It seems like adjacent channels will sometimes be successfully received, and sometimes not. (In general, it seems to be when rtl_433 reports a frequency lower than my dwell channel, not higher.).

This may require tightening the receive bandwidth even more if the receive application implements hopping, since receiving from an adjacent channel might result in an improper hop. (I've managed to decipher the hopping sequence of the R900 - https://github.com/Entropy512/r900_esphome/tree/main) Of course, it may be more beneficial to accept a worst-case of one report every 700 seconds and not attempt to follow the hops.