search

rpp0 / gr-lora

GNU Radio blocks for receiving LoRa modulated radio messages using SDR
GNU General Public License v3.0
526 stars 113 forks source link

Failed to connect to analyzer. #42

Closed g0hww closed 6 years ago

g0hww commented 6 years ago

Hi, Trying this out to see if I can decode raw LORA test messages sent by my new LoPy device. When I run the lora_receive_realtime.grc flowgraph, I see "Failed to connect to analyzer." in the console output. Any idea why this might be? This with latest code (as of yesterday) built freshly with pybombs.

g0hww commented 6 years ago

Hmm. After I actually got the frequency and spreading factor configured correctly, it crashed a few times when the LORA beacons were sent. After a couple of crashes, the next time it started to decode packets :) Not sure what is up, but it does now seem to be printing consistent bytes when the lopy beacons. Now to make some sense of them!

rpp0 commented 6 years ago

Hi, my bad. I added some new debugging functionality a few days ago and forgot to disable it. The "Failed to connect to analyzer." message can be safely ignored. The crashing might be an issue however. If you have the time, could you provide me with the coredump file of the crash? That way I could determine the cause.

rpp0 commented 6 years ago

I pushed a possible fix today. Could you check to see if the crashes still occur?

g0hww commented 6 years ago

Urgh. I updated gr-lora-rpp0 to pull in your latest changes, but I have not yet managed to get any decodes today, though I believe that nothing else had changed in any significant way. I manually reverted your change to decoder_impl.cc to see if that made any difference. It didn't improve the lack of decodes, but it did crash once when my LORA beacon was sent (though not repeatably so, though yesterday it strangely only started to decode after having crashed a few times). With your latest change in place, it hasn't yet crashed. I saw the test results in your latest commit, and noted that "cr4-6 bw125 sf7" looked like it had good results, so I tried that, but with no luck. I do note, however, that yesterday I was using the default 4-5 with sf12 and once the flowgraph started to decode, it was repeatably decoding packets to the same hex values, a couple of bytes of which did seem to convey clear text characters that I recognised from my beacon strings. Anyway, sorry for waffling on. I suppose this ticket can be closed, as the "failure to connect to analyzer" message no longer appears, and I have also found the analyser now :) though it has n't had anything to analyse yet. I'll keep pottering around here with this stuff and see what happens. Hopefully my 2nd lopy will arrive soon, so the first one will have something to talk to, and I can check that the modems are behaving correctly.

g0hww commented 6 years ago

Hmm. Given another 10dB or so dB of signal, it has now started to decode at "bandwidth=LoRa.BW_125KHZ, sf=7, preamble=8, coding_rate=LoRa.CODING_4_5". That's at -30dBFS on my USRP B100/wbx.

rpp0 commented 6 years ago

Cool :) glad you managed to decode some signals. To comment on your findings: SF 12 works only for a few bytes because of clock drift between the SDR and transmitter, which I currently have no viable solution for. I'm experimenting with some new things right now, but so far no success. It's a difficult problem because there is no documentation on how the hardware maintains accurate time and frequency synchronization. The analyzer is something I recently developed to plot signals live from GNU Radio. If you want to experiment with it, you can call debugger::store_samples to store samples in a buffer and debugger::analyze_samples to plot them using matplotlib.

As for increasing the dB: there is a configurable detection threshold in the LoRa Receiver block. You could try to set the threshold lower and see if you have better results with a weaker signal this way.

Thanks for investing your time in this!

rpp0 commented 6 years ago

Hi @g0hww, I finally figured out the clock drift correction, so if you want to try SF 12 again feel free to give it a go!

pierreduf commented 6 years ago

I've updated to the last version and still have the error "Failed to connect to the analyzer".

Apart from that, the real problem is that I experience a lot of crashes as well. I've noticed that the crash happens just when a LoRa signal is received and the receiver tries to decode it.

I thought it was initially because of the load of the CPU (I'm working with the rpp0 samples @10MHz) but if I slightly change the receive frequency there's no crash at all (and of course no decoding). I even try to resample to reduce the CPU usage but I've got the same result.

When it doesn't crash, it seems that I've got only part of the payload. For instance, for the "00_crcon_cr1_sf7.cfile" file :

10 0b 0b 00 10 0b 0b 00 10 0b 0b 00 10 0b 0b 00 10 0b 0b 00 10 0b 0b 00 10 0b 0b 00 10 0b 0b 00 10 0b 0b 00

At last, I've noticed another strange thing : the "Bits per symbol" seems to be miscalculated. It should be "7" (for SF7) but instead "35" is displayed :

Using Volk machine: avx_64_mmx_orc Bits per symbol: 35 Bins per symbol: 128 Header bins per symbol: 32 Samples per symbol: 1024 Decimation: 8 Failed to connect to analyzer.

One last thing : when I enable the "real time scheduling" I get a nice segfault !

I guess it's not a good practice to comment in a closed issue but as it's exactly the same and the fix hasn't resolved it...

Do you have any idea ?

rpp0 commented 6 years ago

Hi! Thanks for reporting.

I've updated to the last version and still have the error "Failed to connect to the analyzer".

Hmm, are you sure? Because this line has been commented out in the latest version. See: https://github.com/rpp0/gr-lora/blob/95131c0719225c915d5dec059714b87a3292bae1/lib/debugger.cc#L33

Apart from that, the real problem is that I experience a lot of crashes as well. I've noticed that the crash happens just when a LoRa signal is received and the receiver tries to decode it.

That's not good at all! Could you send me a coredump file or perhaps the file /tmp/grlora_debug_txt? Same for the real time scheduling scenario.

When it doesn't crash, it seems that I've got only part of the payload. For instance, for the "00_crcon_cr1_sf7.cfile" file :

These packets are supposed to contain 3 bytes header followed by 00, so looks okay to me.

At last, I've noticed another strange thing : the "Bits per symbol" seems to be miscalculated. It should be "7" (for SF7) but instead "35" is displayed :

You are correct, thanks. I spotted the error and this will be fixed in the next version.

Thanks again for the detailed testing.

pierreduf commented 6 years ago

Thank you for your quick reply !

I'm pretty sure because I've :

I definetely see the previous fix and the patch. I see that that string is only present in this file (and commented now), so I really can't understand how's that possible !

Good news for the miscalculation. I guess it's in the "decoder_impl.cc" file ?

I'm gonna try to get the core dump ASAP.

I wasn't aware that the sample were not real LoRa packets, that's why I thought it was only a partial decoding.

rpp0 commented 6 years ago

Perhaps an old version of the gr-lora is still in your shared library cache? You could try running sudo ldconfig.

Good news for the miscalculation. I guess it's in the "decoder_impl.cc" file ?

Yes, the calculation should be: d_bits_per_second = (double)d_sf * (4.0f / (4.0f + d_cr)) / (1u << d_sf) * d_bw;. However, this value is purely calculated for printing some information to stdout, so it shouldn't affect the decoder's accuracy.

pierreduf commented 6 years ago

Yes, I did a sudo ldconfig after each build & install 😁

But it's probably something like that, I'm going to search more thoroughly. Thx for the tip !

rpp0 commented 6 years ago

Hi @pierreduf, any news yet regarding the crashes you encountered? I'll see if I can set up a Docker container today, so that any system-dependent factors can be eliminated.

rpp0 commented 6 years ago

The Docker container should now be available. Could you please check whether the problem persists?

g0hww commented 6 years ago

Hi @rpp0 This is now working very well for me. I've mostly been running sf8, but I did try sf12 again and that worked too. I've been able to decrypt the AES-128 CFB mode encrypted messages that I've been sending using socket.AF_LORA, socket.SOCK_RAW sockets between my pair of LoPys, using a python script listening to UDP port 40868. Sometimes the decryptions fail partially. I suppose there's no error detection mechanism in play here, is there? I was thinking of adding a MAC anyway to my OTA format. I'll have a go now. Thanks again :) UPDATE: Oh, I forgot to mention that it is working nicely with both my HackRF One and my USRP B100/wbx.

rpp0 commented 6 years ago

Hi @g0hww, great to hear it's working for you! Yeah, the decoding accuracy on my test setup went from about 60% to 100% with the new algorithm, so I'm happy with the result as well :). However, I didn't test yet under practical circumstances / long distances where you could have a lot of noise. So there's still room for improvement. CRC calculation for error detection is something I will focus on next. It's nice that you combined gr-lora with software to decrypt LoRaWAN packets. Did you write this yourself, or did you use a library for that?

g0hww commented 6 years ago

Hi @rpp0 I'm not using LoRaWAN with the LoPys, but raw LoRaMAC sockets, as described here: https://docs.pycom.io/chapter/tutorials/lopy/lopy-lopy.html

For now I've just implemented HMAC-MD5 and used that with MD5 and AES-128 from the "uhashlib" and "crypto" micropython modules that interface to the crypto accelerator and used that to encrypt and authenticate the traffic between the LoPys.

https://docs.pycom.io/chapter/firmwareapi/micropython/uhashlib.html https://docs.pycom.io/chapter/firmwareapi/pycom/aes.html

rpp0 commented 6 years ago

Oh I see, cool! Anyway, I suppose this issue can be closed now. If you encounter anything else, feel free to make a new issue :).

pierreduf commented 6 years ago

For the sake of completeness, I investigated a bit further and my problem was that an old library file (from a previous installation in /usr/local/lib) was used....my bad, you were right !

Oddly enough, before the update, I still had the segfault but not when running it through GDB.

Another strange thing is that I don't have the same decoded result before and after the update (the "header" before the 00 is not the same).

Some outputs below. Hope it can help any future problem !

I haven't had time to try the docker container yet but I'm gonna keep you updated as soon as I have !

rpp0 commented 6 years ago

Thanks @pierreduf for the detailed report!

Oddly enough, before the update, I still had the segfault but not when running it through GDB.

The version before the update had a buffer overread bug that could occur if the receiver detected the preamble near the end of the sample buffer. I only noticed this bug after this issue was created, because on my machine the preamble was always detected near the beginning of the sample buffer. Therefore, since timing plays a role, this sounds like a Heisenbug to me, which might also explain why it runs fine inside gdb.

Another strange thing is that I don't have the same decoded result before and after the update (the "header" before the 00 is not the same).

I improved the synchronization and decoding algorithms, so that would probably be the reason. The header from the old version looks invalid to me :).

pierreduf commented 6 years ago

That's good to know. Thank you for the explanation !

I tried the Docker container. Everything is working smoothly and correctly, thank you !

I still have some warning messages (Gtk warnings, what is the "canberra" module ??) but I guess that's not a problem :

The only thing odd is that the number of bits per symbol should 7, as the messsage I'm decoding is in SF7.

I'm a bit surprise as well about Python not being installed inside the container. Is this on purpose ?

Gtk-Message: Failed to load module "canberra-gtk-module"
<<< Welcome to GNU Radio Companion 3.7.11 >>>

Block paths:
    /usr/share/gnuradio/grc/blocks

Loading: "/src/apps/lora_receive_file.grc"
/usr/lib/python2.7/site-packages/wx-3.0-gtk2/wx/_core.py:16629: UserWarning: wxPython/wxWidgets release number mismatch
  warnings.warn("wxPython/wxWidgets release number mismatch")
>>> Done

Generating: '/src/apps/lora_receive_file.py'

Executing: /usr/bin/python2 -u /src/apps/lora_receive_file.py

/usr/lib/python2.7/site-packages/wx-3.0-gtk2/wx/_core.py:16629: UserWarning: wxPython/wxWidgets release number mismatch
  warnings.warn("wxPython/wxWidgets release number mismatch")
Gtk-Message: Failed to load module "canberra-gtk-module"
Bits (nominal) per symbol:  3
Bins per symbol:    128
Samples per symbol:     1024
Decimation:         8

(lora_receive_file.py:99): Gtk-WARNING **: gtk_widget_size_allocate(): attempt to allocate widget with width -5 and height 20

(lora_receive_file.py:99): Gtk-WARNING **: gtk_widget_size_allocate(): attempt to allocate widget with width -5 and height 20

(lora_receive_file.py:99): Gtk-WARNING **: gtk_widget_size_allocate(): attempt to allocate widget with width -5 and height 20

(lora_receive_file.py:99): Gtk-WARNING **: gtk_widget_size_allocate(): attempt to allocate widget with width -5 and height 20
 10 09 08 00
 10 09 08 00
 10 09 08 00

>>> Done
rpp0 commented 6 years ago

I tried the Docker container. Everything is working smoothly and correctly, thank you !

Great to hear!

I still have some warning messages (Gtk warnings, what is the "canberra" module ??) but I guess that's not a problem

Libcanberra is for playing sounds, so it's not really needed in the Docker image and Gtk complains about it.

The only thing odd is that the number of bits per symbol should 7, as the messsage I'm decoding is in SF7.

The nominal bit rate indicates the bit rate without considering error correction bits. You are correct that the number of bits per symbol is 7, but this includes error correction bits. The formula to calculate the nominal bit rate is (sf×(4÷(4+cr))÷(2^sf÷bw)) ÷ (bw÷(2^sf)). So the correct answer is 3,5 for SF 7 and CR 4. It's rounded to 3 because it's an integer. I will change it to a float next version :).

I'm a bit surprise as well about Python not being installed inside the container. Is this on purpose ?

Python2 is required by GNU Radio, but Python3 is not installed. You can run a Python shell by typing python2 in the terminal.

sankaran1408 commented 6 months ago

Hmm. After I actually got the frequency and spreading factor configured correctly, it crashed a few times when the LORA beacons were sent. After a couple of crashes, the next time it started to decode packets :) Not sure what is up, but it does now seem to be printing consistent bytes when the lopy beacons. Now to make some sense of them!

I am transmitting va a dragino lora module attached to an arduino uno with arduino ide , but i am not able to get the messages decoded. can you help me to figure out why. issue

sankaran1408 commented 6 months ago

@rpp0 @pierreduf @g0hww can you help me with arduino code that can be successfully decoded by the lora_receive_realtime.py ...and i am using an rtlsdr dongle