search

BastilleResearch / gr-lora

GNU Radio OOT module implementing the LoRa PHY, based on https://github.com/matt-knight/research/tree/master/2016_05_20_jailbreak
GNU General Public License v3.0
470 stars 92 forks source link

Corrupt packets received from HopeRF RFM98W at 434MHz #9

Open mikaelnousiainen opened 7 years ago

mikaelnousiainen commented 7 years ago

I'm trying to receive LoRa packets sent by my Raspberry Pi LoRa shield with RFM98W chip, but I've only succeeded to receive corrupt data (very randomly).

I'm using 2M sample rate on 434.250 MHz frequency with NESDR SMArt (RTL-SDR) USB dongle and I've verified using Gqrx that when tuning to 434.250, it's receiving at the center of the signal bandwidth.

I've tested another gr-lora implementation (by rpp0) and had similar problems: https://github.com/rpp0/gr-lora/issues/15 although I was able to receive parts of the data correctly.

I'm submitting the same example IQ file and the GNU Radio graphs for this project to help development. (Unfortunately, my skills in signal processing are almost nonexistent, so I'm not able to contribute to the code...)

I've made a cfile (converted to complex from rtl_sdr bin capture) of using settings: SF7, CR4:8, bandwidth 125kHz, 2M sample rate and preamble of 0x08, and tested that reception "works" from the file in GNU Radio. I verified that packets are missed and data in all packets is corrupt. All data in packets should be plain text (as listed below).

I've attached the GNU radio graph here: rtlsdr-lora-receive-mattknight.grc.zip

(BTW Why are the additional resampler and rotator elements needed? This could be added to the documentation.)

The captured signal cfile can be downloaded here (for the next 48 hours): https://expirebox.com/download/cb1750ca2f660e6290186ff0cedf79a7.html

For capturing the signal I used the following command:

rtl_sdr -f 434250000 -s 2000000 capture-lora-rfm98w-434250000-sf7-bw125k-cr48-crcon-2msps.bin

And for conversion I used this GNU Radio graph: http://ham.stackexchange.com/a/2117

The transmission contains the following messages (without the quotes):

2016-12-20 08:55:04.512213 INFO   24911 ert main.c:main:320 -- Transmitting message: "MESSAGE 0001: SHORT MESSAGE END"
2016-12-20 08:55:04.598480 INFO   24911 ert main.c:main:329 -- Transmitting message: "MESSAGE 0002: LONG MESSAGE 1 LONG MESSAGE 2 LONG MESSAGE 3 LONG MESSAGE 4 LONG MESSAGE 5 LONG MESSAGE 6 LONG MESSAGE 7 LONG MESSAGE 8 LONG MESSAGE 9 LONG MESSAGE A LONG MESSAGE B LONG MESSAGE C LONG MESSAGE D LONG MESSAGE E LONG MESSAGE F LONG MESSAGE END"
2016-12-20 08:55:05.012920 INFO   24911 ert main.c:main:320 -- Transmitting message: "MESSAGE 0003: SHORT MESSAGE END"
2016-12-20 08:55:05.092218 INFO   24911 ert main.c:main:329 -- Transmitting message: "MESSAGE 0004: LONG MESSAGE 1 LONG MESSAGE 2 LONG MESSAGE 3 LONG MESSAGE 4 LONG MESSAGE 5 LONG MESSAGE 6 LONG MESSAGE 7 LONG MESSAGE 8 LONG MESSAGE 9 LONG MESSAGE A LONG MESSAGE B LONG MESSAGE C LONG MESSAGE D LONG MESSAGE E LONG MESSAGE F LONG MESSAGE END"
2016-12-20 08:55:05.505725 INFO   24911 ert main.c:main:320 -- Transmitting message: "MESSAGE 0005: SHORT MESSAGE END"
2016-12-20 08:55:05.585291 INFO   24911 ert main.c:main:329 -- Transmitting message: "MESSAGE 0006: LONG MESSAGE 1 LONG MESSAGE 2 LONG MESSAGE 3 LONG MESSAGE 4 LONG MESSAGE 5 LONG MESSAGE 6 LONG MESSAGE 7 LONG MESSAGE 8 LONG MESSAGE 9 LONG MESSAGE A LONG MESSAGE B LONG MESSAGE C LONG MESSAGE D LONG MESSAGE E LONG MESSAGE F LONG MESSAGE END"
2016-12-20 08:55:05.995854 INFO   24911 ert main.c:main:320 -- Transmitting message: "MESSAGE 0007: SHORT MESSAGE END"
2016-12-20 08:55:06.074197 INFO   24911 ert main.c:main:329 -- Transmitting message: "MESSAGE 0008: LONG MESSAGE 1 LONG MESSAGE 2 LONG MESSAGE 3 LONG MESSAGE 4 LONG MESSAGE 5 LONG MESSAGE 6 LONG MESSAGE 7 LONG MESSAGE 8 LONG MESSAGE 9 LONG MESSAGE A LONG MESSAGE B LONG MESSAGE C LONG MESSAGE D LONG MESSAGE E LONG MESSAGE F LONG MESSAGE END"
2016-12-20 08:55:06.486301 INFO   24911 ert main.c:main:320 -- Transmitting message: "MESSAGE 0009: SHORT MESSAGE END"
2016-12-20 08:55:06.564632 INFO   24911 ert main.c:main:329 -- Transmitting message: "MESSAGE 0010: LONG MESSAGE 1 LONG MESSAGE 2 LONG MESSAGE 3 LONG MESSAGE 4 LONG MESSAGE 5 LONG MESSAGE 6 LONG MESSAGE 7 LONG MESSAGE 8 LONG MESSAGE 9 LONG MESSAGE A LONG MESSAGE B LONG MESSAGE C LONG MESSAGE D LONG MESSAGE E LONG MESSAGE F LONG MESSAGE END"
2016-12-20 08:55:06.976913 INFO   24911 ert main.c:main:320 -- Transmitting message: "MESSAGE 0011: SHORT MESSAGE END"
2016-12-20 08:55:07.055899 INFO   24911 ert main.c:main:329 -- Transmitting message: "MESSAGE 0012: LONG MESSAGE 1 LONG MESSAGE 2 LONG MESSAGE 3 LONG MESSAGE 4 LONG MESSAGE 5 LONG MESSAGE 6 LONG MESSAGE 7 LONG MESSAGE 8 LONG MESSAGE 9 LONG MESSAGE A LONG MESSAGE B LONG MESSAGE C LONG MESSAGE D LONG MESSAGE E LONG MESSAGE F LONG MESSAGE END"
2016-12-20 08:55:07.468894 INFO   24911 ert main.c:main:320 -- Transmitting message: "MESSAGE 0013: SHORT MESSAGE END"
2016-12-20 08:55:07.547670 INFO   24911 ert main.c:main:329 -- Transmitting message: "MESSAGE 0014: LONG MESSAGE 1 LONG MESSAGE 2 LONG MESSAGE 3 LONG MESSAGE 4 LONG MESSAGE 5 LONG MESSAGE 6 LONG MESSAGE 7 LONG MESSAGE 8 LONG MESSAGE 9 LONG MESSAGE A LONG MESSAGE B LONG MESSAGE C LONG MESSAGE D LONG MESSAGE E LONG MESSAGE F LONG MESSAGE END"
2016-12-20 08:55:07.959763 INFO   24911 ert main.c:main:320 -- Transmitting message: "MESSAGE 0015: SHORT MESSAGE END"
2016-12-20 08:55:08.037965 INFO   24911 ert main.c:main:329 -- Transmitting message: "MESSAGE 0016: LONG MESSAGE 1 LONG MESSAGE 2 LONG MESSAGE 3 LONG MESSAGE 4 LONG MESSAGE 5 LONG MESSAGE 6 LONG MESSAGE 7 LONG MESSAGE 8 LONG MESSAGE 9 LONG MESSAGE A LONG MESSAGE B LONG MESSAGE C LONG MESSAGE D LONG MESSAGE E LONG MESSAGE F LONG MESSAGE END"
2016-12-20 08:55:08.447643 INFO   24911 ert main.c:main:320 -- Transmitting message: "MESSAGE 0017: SHORT MESSAGE END"
2016-12-20 08:55:08.526002 INFO   24911 ert main.c:main:329 -- Transmitting message: "MESSAGE 0018: LONG MESSAGE 1 LONG MESSAGE 2 LONG MESSAGE 3 LONG MESSAGE 4 LONG MESSAGE 5 LONG MESSAGE 6 LONG MESSAGE 7 LONG MESSAGE 8 LONG MESSAGE 9 LONG MESSAGE A LONG MESSAGE B LONG MESSAGE C LONG MESSAGE D LONG MESSAGE E LONG MESSAGE F LONG MESSAGE END"
2016-12-20 08:55:08.938190 INFO   24911 ert main.c:main:320 -- Transmitting message: "MESSAGE 0019: SHORT MESSAGE END"
2016-12-20 08:55:09.017061 INFO   24911 ert main.c:main:329 -- Transmitting message: "MESSAGE 0020: LONG MESSAGE 1 LONG MESSAGE 2 LONG MESSAGE 3 LONG MESSAGE 4 LONG MESSAGE 5 LONG MESSAGE 6 LONG MESSAGE 7 LONG MESSAGE 8 LONG MESSAGE 9 LONG MESSAGE A LONG MESSAGE B LONG MESSAGE C LONG MESSAGE D LONG MESSAGE E LONG MESSAGE F LONG MESSAGE END"

What I receive in Wireshark is just garbage in the data portion of the UDP packets.

matt-knight commented 7 years ago

Hi @mikaelnousiainen, thanks for providing a detailed report along with captures. I'll try to find some time this weekend to look into your issue. In the meantime, can you confirm that you're using implicit header mode?

The resampler and rotator are there to channelize the signal. I typically capture with the signal offset from the center frequency to avoid the SDR's DC spike. The rotator "shifts" the band so that the signal is centered and the DC spike is offset, and then the resampler reduces the sample rate to match the bandwidth of the LoRa signal -- effectively discarding whatever is outside of said bandwidth. This leaves you with a neatly channelized signal, which is what the LoRa demodulator expects.

Try using Baudline or an FFT/Waterfall sink to visualize each stage of this transformation and you'll see it in action.

mikaelnousiainen commented 7 years ago

I was using explicit header in the IQ sample. I just noticed that it isn't supported yet by gr-lora. I'll do another IQ recording with implicit header mode next week (once I'm back at where I have the hardware) and see how it works. One question though: how does the demodulator/decoder know the packet length with implicit header mode? Isn't the packet length what is left out when not using the header?

Is there something in particular blocking the decoding of explicit headers?

Thanks explaining how the GR components are used. I assumed something like that, but wasn't sure.

matt-knight commented 7 years ago

Packet framing: The demodulator uses a squelch to find the end of the frame. Check it out here: https://github.com/BastilleResearch/gr-lora/blob/master/lib/demod_impl.cc#L392

Why it doesn't do explicit headers: Explicit header decoding is blocked by not having been able to fully derive the whitening sequence that applies to said header. This gets into some of the weedy details of LoRa. Here's an abridged explanation:

One of the operations within the LoRa modulation is whitening, where your outgoing frame is XORed against the a predetermined pseudorandom string. LoRa's demodulation performs the same operation, XORing the incoming frame against said predetermined pseudorandom string, which returns the outgoing frame because XOR is its own inverse.

I was able to derive the implicit whitening sequences because I have control of every bit within the frame, including bits that would otherwise be used by the explicit header. Thus, I can zero them out and exploit XOR to get an off the shelf LoRa module to tell me what it's whitening sequence is.

My initial assumption was that implicit and explicit header mode would share the same whitening sequence (because why wouldn't they?) but empirically I found this wasn't the case. Additionally, under explicit mode I do not have control of the header bits, meaning I cannot force their value to 0. Thus I cannot derive the whitening bits that apply to the header.

The good news is that I've made good progress on mapping out a few whitened values within the header format. Thus I'll be able to implement some of the explicit header fields and functionality. Perhaps if I can guess their unwhitened values I'll be able to take a whack at the PHY header's CRC as well (the header has its own CRC).

Explicit header mode is being tracked in issues https://github.com/BastilleResearch/gr-lora/issues/3 and https://github.com/BastilleResearch/gr-lora/issues/4.

tl;dr some explicit mode support is coming soon™

mikaelnousiainen commented 7 years ago

Ok, yes, that makes sense (for the explicit headers). Thanks for the detailed explanation!

I made another two test captures with the same data and settings as described above, but with implicit header enabled. This time I gr-lora (nor rpp0's gr-lora) couldn't really decode any of the data. Anyway, here are the recordings in case they are useful to you.

The first one was recorded with the SDR dongle antenna (for 434MHz) a bit further away, so I decided to bring it very close (30cm) to see if it makes a difference.

https://expirebox.com/download/d36c3e50fe65efaff49885d1fbf4a612.html

https://expirebox.com/download/c0ce4cf6561bccaa6236b51fa02d8ad6.html

Again, the links expire in 48h.