some questions

[karatemir]

Hi there,

When I am using the gr-droneid branch with a mavic air 2, gnuradio console displays offsets/phase angle etc. and i see some extracted constellation results in the QT constellation diagram which I assume corresponds to gnuradio extractor/demodulator blocks working properly. But when I check the /tmp/bursts folder I only see a zc and channel file.

I have watched the DragonOS youtube videos and in those videos it seems that thee bursts are getting recorded as well (file_1, file_2, etc.). As I mentioned I only got files named zc and channel getting created in the folder. Does this mean something is not working correctly, or is this normal behavior?

My other question is regarding computation time for the process_file.m script. I have a fairly decent machine (16 cores, 16GB RAM) and running the octave script on 2GB of fc32 recording takes a very long time. Is this normal, or have I configured something incorrectly?

Thanks for all your work, very interesting repo!

[proto17]

The writing of the individual bursts was disabled [1]. I honestly don't recall why I did that, but I did =\ You can re-enable by uncommenting the linked line. You can also try out the gr-droneid-update branch which has some updates to the handling of temp files.

Octave doesn't do nearly as good a job using multiple cores as MATLAB. It's crazy slow. To the extent that I created a C MEX extension to plow through the correlations faster. The main time suck is the normalized cross correlation done in [2]. There is a custom cross correlation function in use that's much faster than MATLAB's xcorr(x,y,0,'normalized') function, but still something like 100x slower than filter(). This huge processing time issue is why the GNU Radio module was created. I wanted something that could pluck bursts out of the air and save just the burst for later processing.

[1] https://github.com/proto17/dji_droneid/blob/gr-droneid/gnuradio/gr-droneid/lib/time_sync_impl.cc#L85 [2] https://github.com/proto17/dji_droneid/blob/main/matlab/updated_scripts/find_zc_indices_by_file.m

[karatemir]

thanks for the clarification. Are we supposed to feed the shorter burst recordings (the ones that have been disabled) directly to the .m file or should I just stick with the full IQ recording?

[alphafox02]

I think the bursts, but in my case none of my bursts ran individual or smashed together produce a result (at least for me) with the octave script. If instead I run the full IQ file that produced the bursts through octave, I hit jackpot. Be great to hear if you get results with just the bursts from your IQ file.

[karatemir]

I guess you are cemaxucuter, right? I watched your youtube vids, and in that you directly used the ~2GB IQ recording, so that was part of my confusion. Maybe @proto17 can clarify this specific issue further.

[proto17]

I am not 100% sure why the processing_file.m script doesn't work with the extracted bursts from GNU Radio. Might have to do with not having enough padding on either end of the file. There is logic in the ZC sequence finder [1] that will abort if there aren't enough samples around the burst. It's definitely something that should work, so I'll take a look this week.

[1] https://github.com/proto17/dji_droneid/blob/main/matlab/updated_scripts/find_zc_indices_by_file.m#L159

[alphafox02]

I guess you are cemaxucuter, right? I watched your youtube vids, and in that you directly used the ~2GB IQ recording, so that was part of my confusion. Maybe @proto17 can clarify this specific issue further.

That’s me :). I had a 8GB file at first.. that took a long while, so bursts would be great.

[karatemir]

I have done some extra experimentation with gr-droneid branch and have some additional questions.

I have uncommented Line 85 in time_sync_impl.cc as per @proto17's recommendation. Now burst files end up in /tmp/bursts as expected. When feeding the bursts to process_file.m however I get the following warning messages:

warning: Skipping burst at offset 6288 as the beginning of the burst has been clipped
warning: called from
    extract_bursts_from_file at line 62 column 13
    process_file at line 47 column 8

warning: Skipping burst at offset 8562 as the beginning of the burst has been clipped
warning: called from
    extract_bursts_from_file at line 62 column 13
    process_file at line 47 column 8

warning: Skipping burst at offset 9093 as the ending of the burst will be clipped
warning: called from
    extract_bursts_from_file at line 67 column 13
    process_file at line 47 column 8

error: Did not find any bursts
error: called from
    assert at line 109 column 11
    process_file at line 50 column 1

Is this normal expected behavior?

Another thing I notice is that when running gr-droneid branch .grc file, sometimes the flowgraph crashes/stops working immediately after catching the first burst. And sometimes it just stops working after a while (not more than few minutes). Is this something the droneid module decides itself or can it be due to something I have done (I am running gnuradio/fosphor/uhd/droneid within a docker container). When it stops, I get the following: >>> Done (return code -11)

I also tried running the IQ recording through the octave file. (I recorded a Mavic Air 2 with B210 @ 2.4295GHz with gain set to 0.8) remove_turbo binary just outputs: [ERROR] CRC did not zero out. Got a5a072 after calculation. Does this mean that I failed in capturing the DroneID packets? (QT constellation diagram shows results similar to @alphafox02's youtube videos)

Another thing I noticed is that droneid constellation detects some bursts even when there is no drone within the vicinity of the SDR. Is droneid somehow capturing WiFi/802.11 packets? Should I try to record outdoors far from other 2.4GHz sources?

[alphafox02]

I moved away from WiFi and into an empty parking lot for captures once I noticed it was picking up WiFi (or it was just getting in my way) or so it seemed. Initially the flow graph crashed on me to, detailed in a now closed ticket. After some code updates by the developer the flow graph was much more stable and if I recall only stopped on me when I cranked the gain up real high (maybe unrelated).

I recorded maybe 3-4 IQs which I’ve used to run back through the flow graph. I get bursts, but basically same results as you. However, running the entire IQ file(s) through octave produce numerous good frames that translate to accurate info. Very neat you have it all running in docker!

[karatemir]

@alphafox02 yeah I have experienced the same, high gain is probably the common denominator in the "crashes" though I don't why. I will check the closed ticket you mentioned.

One other problem I have is calling remove_turbo from octave. For some reason when calling remove_turbo from octave I get an error saying libturbofec is not found. However, I can directly use remove_turbo from the command line myself. ldd remove_turbo also shows /usr/local/lib/ for libturbofec.so , therefore I assume it is properly linked. I don't know whether calling binaries from octave using system somehow misses the linked libraries.

[proto17]

The WiFi detection is quite interesting. I am not in a very WiFi heavy area, so maybe I never encountered that issue just due to not having a lot of activity there. I can say that high gain will cause crashes in the gr-droneid branch. I made some changes in gr-droneid-update to help prevent crashes due to high gain. I think the issue ended up being that the correlation scores went through the roof when the gain was jacked up too high. Not totally sure if I fixed it, but I made it a little less of a problem in my tests.

The original crashing bug was due to walking off the edge of an array. Make sure that you have the most up to date gr-droneid as it has the fix for that problem. If you're still experiencing then I'll do what I can to recreate the bug under GDB.

I suspect that the PATH variable in Octave doesn't know about /usr/local/lib. Try running getenv('LD_LIBRARY_PATH') in Octave and see if it matches echo $LD_LIBRARY_PATH in the Linux terminal. If they don't match then that's likely the problem. You might be able to get around the issue by preloading the library when you run Octave:

LD_PRELOAD=/usr/local/lib/libturbofec.so octave

And to fix an issue with the LD_LIBRARY_PATH variable:

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib octave

Edit: I'm not sure exactly how Octave gets its environment variables set when using system. You might have to change the system to something like:

    [retcode, out] = system(sprintf("/bin/bash -c 'source /etc/profile; %s %s'", turbo_decoder_path, "/tmp/bits"));

Which might force the use of Bash and will bring up the environment specified in /etc/profile. A quick look at help system in Octave shows that it uses /bin/sh by default which might be the reason that you're seeing different LD_LIBRARY_PATH settings from your terminal vs Octave's system function

[karatemir]

@proto17 , yeah looks like octave's LD_LIBRARY_PATH was different since I installed it with snap. Thanks for the pointer!

In terms of high gain; gr-droneid-update branch is way more resilient than gr-droneid branch to high gain. New branch hasn't crashed on me yet! What I have noticed is though; gr-droneid branch after gain 0.8 starts somehow capturing the WiFi and usually crashes as gain goes above 0.9. With gr-droneid-update I need to push gain all the way up to 1.0 for the "WiFi" captures but haven't seen any crashes.

I have another question regarding the update branch: I get stuff getting printed on the GR console as the flowgraph is running and when bursts are captured they are recorded in tmp/droneid_debug folder. Most of the time what is printed is like

Max: 13047 offset: 6736 - 6311
short: 144 long: 160 fft: 2048
((start_idx . 6311))
Found burst @ 4808094779 / 4808094778
Sending message with 23856 elements (23856) starting at 4808094778
Found burst @ 4808122193 / 4808122192
Sending message with 23856 elements (23856) starting at 4808122192
Found burst @ 4808146134 / 4808146133
Sending message with 23856 elements (23856) starting at 4808146133
ZC start index: 13047, est start index: 6311
Max: 14913 offset: 6736 - 8177
short: 144 long: 160 fft: 2048
((start_idx . 8177))

but I don't get any hex frames dumped. When I get the hex frames they are always CRC Check Failed! . What does this mean? Did I somehow fail in capturing DroneID frames? Or are the frames written to console only if they fail?

Also this is a trivial suggestion; but for ease of installation you might want to include CRC.h in the relevant include folder (unless there is a license conflict) and probably cmake can look for libturbofec.so to catch any issues before make.

[proto17]

Right now the frequency offset and equalization in GNU Radio isn't working well. In fact the equalization is turned off as it causes more problems than it solves. This means that the constellations are almost certainly going to be rotated too much for demodulation to work properly. I have improvements pending in MATLAB that once tested there will be moved over the the GNU Radio module. The GNU Radio module was not actually meant to do full demod. The original purpose was to just extract bursts so that I didn't have to make a multi-gigabyte recording that takes ages to process. But, that plan has shifted a bit to trying to make the GNU Radio code as robust as the MATLAB. All of that to say that the CRC failure you're seeing is because the frame CRC-24 failed [1]. You definitely should see some hex, so I'm not quite sure what's going on there [2]. So long as there isn't an error in the Turbo decoder/rate matcher you should get some bytes printed to the terminal.

I didn't include CRC.h because I wasn't sure about just taking that person's code. I felt it more responsible to the author to force the end user to go get it from it's actual home. I'm fairly new to open source and didn't want to seem rude to those that work so hard to make our lives easier for free :) As an alternative I might just make it a submodule which will maintain that link to the original author.

As for the CMake, I do have libturbofec as a dependency to gr-droneid [3]. Are you saying I should do something in addition? I'm not a wizzard when it comes to modern CMake, so there's definitely a chance that I missed something.

Now that I think about it, I might be able to instruct CMake to clone the repo containing CRC.h instead of requiring the user to install it.

[1] https://github.com/proto17/dji_droneid/blob/gr-droneid-update/gnuradio/gr-droneid/lib/demodulation_impl.cc#L461 [2] https://github.com/proto17/dji_droneid/blob/gr-droneid-update/gnuradio/gr-droneid/lib/demodulation_impl.cc#L455 [3] https://github.com/proto17/dji_droneid/blob/gr-droneid-update/gnuradio/gr-droneid/lib/CMakeLists.txt#L44

[karatemir]

my bad, I didn't see libturbofec in cmake. Instead of just cloning the repo, I thought it might be more straightforward to just include the CRC.h file in your repo itself but I also see your point. licensing is difficult :)

As I mentioned, terminal usually prints: Found burst @ 4808094779 / 4808094778 Sending message with 23856 elements (23856) starting at 4808094778 but no hex most often the time. But when it prints a hex it is CRC Check Failed!. I will be looking for further changes in the gr-droneid-update branch.

Btw, have you tried running your extracted bursts through octave/matlab? When I do that I can't get any hex dumps. I either get:

error: Did not find any bursts
error: called from
    assert at line 109 column 11
    process_file at line 50 column 1

or

warning: Skipping burst at offset 9093 as the ending of the burst will be clipped
warning: called from
    extract_bursts_from_file at line 67 column 13
    process_file at line 47 column 8

error: Did not find any bursts
error: called from
    assert at line 109 column 11
    process_file at line 50 column 1

[proto17]

The issue with running the bursts through MATLAB is that some amount of padding either side of the burst is required in order to do some of the operations. You might be able to get away with increasing the extracted sample count [1] and increasing the delay in the GNU Radio graph [2]. Perhaps setting [1] to something like

extract_samples_count_((fft_size_ * 15) + (long_cp_len_ * 2) + (short_cp_len_ * 7)){

And setting [2] to something like

int(fft_size * 6) + long_cp_len + (short_cp_len * 3) - 1

I'm not 100% sure this will work, but I think it will add more padding to either side of the burst and give MATLAB/Octave a better chance.

[1] https://github.com/proto17/dji_droneid/blob/gr-droneid-update/gnuradio/gr-droneid/lib/extractor_impl.cc#L47 [2] https://github.com/proto17/dji_droneid/blob/gr-droneid-update/gnuradio/drone_id_test.grc#L71

[karatemir]

thanks for the answer! another thing I noticed is the flowgraph in gr-droneid uses a default delay of 5416 whereas the one in gr-droneid-update is set to 8783, is this somehow significant or just random?

[proto17]

Hmm, that's not ideal. I don't recall if there was a reason for this 😳

[karatemir]

explanation for discrepancy: gr-droneid:

samp_rate = 15.36e6*2
decimation = 2
fft_size = samp_rate / decimation / 15e3 = 1024
long_cp_len = round(0.0000052 * samp_rate / decimation) = 80
short_cp_len = round(0.00000469 * samp_rate / decimation) = 72

delay = int(fft_size * 5) + long_cp_len + (short_cp_len * 3) = 5416

gr-droneid-update:

decimation = 1
samp_rate = 15.36e6*2
long_cp_len  = round(0.0000052 * samp_rate / decimation) = 160
short_cp_len = round(0.00000469 * samp_rate / decimation) = 144
fft_size = samp_rate / decimation / 15e3 = 2048

delay = int(fft_size * 4) + long_cp_len + (short_cp_len * 3) - 1 = 8783

[proto17]

So the older code gave more padding than the newer one. The newer graph doesn't add any real padding from the start of the burst which will likely make MATLAB angry. The symbol being used for correlation is the 4th symbol, so I think that the newer graph doesn't provide any additional padding at the start. I'd have to look at the plots to know for sure.

As some background, the delay is all about stalling the complex samples long enough that by the time the correlation filter sees the spike, the delayed samples are some amount before the start of the burst. If the second OFDM symbol is the one that we are interested in, then without the delay the first sample we would see when the correlation spike is found would be the first (or last depending on the correlation) sample of the second OFDM symbol, losing the first symbol entirely.

[karatemir]

@proto17, I had the chance to look more into this project and I have some observations that might be of interest.

I am using gr-droneid-update branch. I am processing the extracted bursts in /tmp/droneid_debug folder with the octave script. I have also changed the delay in grc and extract_samples_count_ to values you've recommended two weeks ago. I can get proper hex frames that can be decoded now and then.

I have mainly observed two distinct abs_scores correlation levels: One level is around 0.69-0.71 which gets interpreted properly. The other one is around 0.26 for which you always get  CRC did not zero out error. Though interestingly for 0.26 bursts; the normalized correlation when plotted has a very obvious peak that is similar in overall shape to when correlation is around 0.7. Any intuition/ideas regarding this observation? Can lower correlation bursts still be leveraged?

Another thing about extractor in gnuradio is: as I understand you pass the signal through the FFT filter of the zc sequence to implement the correlation. Then in Line 73 you use a threshold of 0.5, to populate the burst whereas in octave/matlab, you use normalized correlation to determine the bursts. Obviously, there is some difference here, can you provide some intuition?

Also, this is slightly weird but I can only get the hex frames when the drone is sitting next to the USRP. If I move the drone something like 10 meters away then I can't seem to extract any bursts. I tried increasing the gain but that doesn't seem to do much (except pushing the gain all the way to 1.0 generates spurious bursts). Any recommendations regarding this observation? Should I maybe reduce the correlation_threshold in extractor's implementation?

Thanks once again for your great work!

[proto17]

Any intuition/ideas regarding this observation? Hmm, not off hand. It's unlikely that anything other than the ZC sequence is generating a correlation score that high. Considering that the spike looks very much like the higher scores, it might be a bug in my code. I'd be curious if you drop the correlation score threshold [2] and then look at the burst(s) in question to see if they look correct. Maybe they have a much lower amplitude? It's also possible that you're getting an intermittent interferer that's messing up the correlation score. Perhaps WiFi?

Can lower correlation bursts still be leveraged? Yes, you can lower the correlation threshold using [1] for MATLAB and [2] for GNU Radio (this value needs to be exposed by the OOT module in GNU Radio Companion). Since the correlation is normalized you should be able to bring that value pretty low and still not have false positives. That's assuming that you're RX gain isn't jacked to the moon such that you're clipping on at the front end.

The extractor module does indeed use an FFT filter to do the correlation to search for the ZC sequence. For reasons I do not understand the output of the FFT filter correlation is magically normalized. The MATLAB filter function wasn't always working that way, so there's a special normalized correlation function in there (this is one of the places where MATLAB and GNU Radio differ in implementation). It might be that the GNU Radio FFT filter has the same issue that the MATLAB filter function does, but I haven't had anyone run into it yet. The issue being the output of the filter function in MATLAB not necessarily coming out normalized. This might be related to the SDR used, but I'm not sure. Anywho, the MATLAB correlates through all of the samples in the file and stores off the normalized correlation scores. Then it looks through those scores for values that meet the specified threshold [1] and calls those "bursts" and extracts out samples before and after to be processed later. In GNU Radio I don't have access to all of the samples at once, so the FFT filter output is sent to the extractor along with a delayed copy of the samples. Once the FFT filter output reaches the threshold [2] it calls that a burst, and begins counting samples until it has received X [3] number of samples. Those samples are then packaged up and sent to the next block.

I have not really put much effort into detections at a longer range. The original intent for the GNU Radio code was to extract bursts without having to record literal minutes of IQ at 30.72 MSPS to then have to run through MATLAB for several more minutes. The real limiting factor for the GNU Radio code is that there's no equalizer. The code is there [4] but when I try to use it things go all screwy. I stopped working on it because it became apparent that integer frequency offset was going to hose everything, so I focused more on trying to solve that problem. After several days of trying things I thought I had something that worked down to just a few dB SNR, but once I tried it on real data everything fell apart :frowning_face: So, to just get something that people could use for more instant gratification than MATLAB/Octave I smashed the descrambler, Turbo decoder, and rate matching code into the GNU Radio module and called it a day.

[1] https://github.com/proto17/dji_droneid/blob/main/matlab/updated_scripts/process_file.m#L37

[2] https://github.com/proto17/dji_droneid/blob/gr-droneid-update/gnuradio/gr-droneid/lib/extractor_impl.cc#L73

[3] https://github.com/proto17/dji_droneid/blob/gr-droneid-update/gnuradio/gr-droneid/lib/extractor_impl.cc#L47

[4] https://github.com/proto17/dji_droneid/blob/gr-droneid-update/gnuradio/gr-droneid/lib/demodulation_impl.cc#L196

[karatemir]

Thanks for the detailed answer @proto17.

I have changed the OOT module and made correlation threshold a parameter that can be controlled from GRC. Once the ZC correlation exceeds the threshold; the bursts are registered properly as you described. However, I did not notice much of an improvement just by dropping the correlation threshold, maybe qualitatively more bursts were getting detected but I am not exactly sure about that. I have tried with a correlation threshold of 0.1 and 0.01 and they both seemed to work fine.

Can lower correlation bursts still be leveraged? Regarding this question, I guess I did not really convey what I was meaning. With gnuradio extracted bursts, I basically see three max. correlation levels in matlab/octave: ~0.7, ~0.26, and ~0.02. When the octave computed correlation is around 0.7; remove_turbo works properly and I can get a hex frame that can be decoded in DroneID values. When octave-computed correlation is around 0.02 the correlation plot is basically jitter and there is no notable peak. However; when octave-computed max correlation is ~0.26, there is a very obvious peak and the shape of the overall correlation plot (plot(abs_scores)) is pretty much identical to the 0.7 case. The problem is that remove_turbo gives CRC did not zero out error in the case of 0.26 correlation. As you mentioned such dominant peak in the correlation should indicate a DroneID burst but I can't seem to get any frames in that case. Any ideas here? Could it be due to frequency offset?

Another thing I noticed is about frequency offset calculation. I tried pushing the index discovered by find_zc_indices_by_file here to right and left. Adding up to +75 to the index, and frames can still get interpreted properly by the descramble and turbo remover. Hovewer, moving the index by -1 to left immediately leads to CRC errors. I believe this is somehow due to the frequency offset calculation channel_phase_adj = (channel1_phase - channel2_phase) / 2;. I was wondering whether this is expected behavior. Also given this observation do you think it is a good idea to use a guard interval and use something like index+10 instead of index to make sure CRC always zeroes out?

[proto17]

Apologies for the delay!

I suspect that my assumption that the FFT filter in GNU Radio would just magically provide normalized output was false. This would explain why you have to drop the correlation score randomly, and why MATLAB and GNU Radio do not agree. I started work on a normalized cross correlator for GNU Radio, but it's crazy slow. On the order of ~ 2 MSPS max throughput with a filter size of 1024 samples which is the best case for the GNU Radio graph. It will take a while to figure out how to get the throughput up, and to test that everything is working properly.

There are a few possible alternatives to the normalized cross correlation:

• Don't normalize and instead look for a sudden rise in correlation scores (this will likely sacrifice low SNR performance and be more susceptible to false triggering)
• Use an auto correlator instead of a cross correlator. This would work because the ZC sequence is symmetric in the time and frequency domains. And it would be less expensive than the cross correlation when not normalized since I only need to compute the dot product of 1/2 of an OFDM symbol. It would probably be just as expensive in the normalized case due to needing to convert both vectors to zero mean, and get the variance of both vectors for every single shift of the correlator. This might also have some complexity around the fact that the second half of the sequence needs to be reversed each time before computing the dot product.
• Only cross correlate for part of the ZC sequence. Perhaps just looking at the first half of the sequence is enough. That's at least 512 * N samples (where N is the oversampling ratio) which is better than 1024 * N

The frequency offset estimation code might not be perfect. I have an adjustment to it in my local repo that uses all of the cyclic prefixes to compute a (hopefully) better coarse estimate.

Moving the starting index to the left (in time) causes the starting index of each OFDM symbol to end up in the cyclic prefix. Starting inside the cyclic prefix is fine as it just causes a cyclic shift in the frequency domain. This is actually something that I have done in the past when I wasn't sure about the true starting sample index. When in doubt, move the starting sample into the cyclic prefix. Now, what's surprising here is that you say you added to the start index which would push the first sample into the data samples. This would cause the 1024 samples that go to the FFT to include cyclic prefix samples of the next OFDM symbol. That's bad and will break things. That's a bit confusing to me.

The channel estimation logic is definitely load bearing code. The line you reference for channel_phase_adj is an attempt to deal with residual time offset. Without using that line you will end up with a constellation that is rotating OFDM symbol to OFDM symbol. It's not usually bad enough to be an issue, but it looks nasty, so I wrote that logic to lock the constellations. I haven't yet run across a situation where that code causes issues, but you can remove it if you would like to try. It's used here [1]. You can just comment that line out. You won't always see the constellation rotating without that line, but it will be more likely.

The biggest issue with starting sample estimation right now is that if you have a frequency offset then the starting offset of the ZC sequence will be wrong. It's some strange feature of the ZC sequence that the cross or auto correlation of the sequence in the presence of a frequency offset will correlate well, but the peak of the correlation will be before or after the actual starting index of the sequence. Since I use the peak to determine the starting index, a large enough frequency offset will break everything. This frequency offset comes from a combination of the transmitter and receiver clock inaccuracies. So even if you have a GPS disciplined oscillator on your SDR, the drone might have a worst case offset and throw everything off. Ultimately I think you're right, that the only real option is to back the starting index off into the cyclic prefix. I've tried that in the past and had issues, but it could have just been that I was doing it wrong =\

[1] https://github.com/proto17/dji_droneid/blob/main/matlab/updated_scripts/process_file.m#L166

[karatemir]

@proto17 , hi there! I just revisited this repo and had a thought. In the grc file, you feed the pseudo-correlation to the extractor block via filtering the input with the zc sequence (droneid.misc_utils.conj(droneid.misc_utils.create_zc_sequence(samp_rate/decimation,600))).

Though for it to be a (pseudo)correlation should not the zc sequence be reversed? For instance, check gnuradio's correlation implementation here.

[proto17]

You're absolutely right about needing to reverse the filter taps! It just so happens that the ZC sequence is mirrored about the center in time and frequency :)

[alphafox02]

Thanks for the detailed answer @proto17.

I have changed the OOT module and made correlation threshold a parameter that can be controlled from GRC. Once the ZC correlation exceeds the threshold; the bursts are registered properly as you described. However, I did not notice much of an improvement just by dropping the correlation threshold, maybe qualitatively more bursts were getting detected but I am not exactly sure about that. I have tried with a correlation threshold of 0.1 and 0.01 and they both seemed to work fine.

Can lower correlation bursts still be leveraged? Regarding this question, I guess I did not really convey what I was meaning. With gnuradio extracted bursts, I basically see three max. correlation levels in matlab/octave: ~0.7, ~0.26, and ~0.02. When the octave computed correlation is around 0.7; remove_turbo works properly and I can get a hex frame that can be decoded in DroneID values. When octave-computed correlation is around 0.02 the correlation plot is basically jitter and there is no notable peak. However; when octave-computed max correlation is ~0.26, there is a very obvious peak and the shape of the overall correlation plot (plot(abs_scores)) is pretty much identical to the 0.7 case. The problem is that remove_turbo gives CRC did not zero out error in the case of 0.26 correlation. As you mentioned such dominant peak in the correlation should indicate a DroneID burst but I can't seem to get any frames in that case. Any ideas here? Could it be due to frequency offset?

Another thing I noticed is about frequency offset calculation. I tried pushing the index discovered by find_zc_indices_by_file here to right and left. Adding up to +75 to the index, and frames can still get interpreted properly by the descramble and turbo remover. Hovewer, moving the index by -1 to left immediately leads to CRC errors. I believe this is somehow due to the frequency offset calculation channel_phase_adj = (channel1_phase - channel2_phase) / 2;. I was wondering whether this is expected behavior. Also given this observation do you think it is a good idea to use a guard interval and use something like index+10 instead of index to make sure CRC always zeroes out?

Would it be safe to assume that a correlation threshold of 0.1 and 0.01 will be good to enter by default into the gr-drone_update flow graph? I wanted to put a default setting in there to ensure the flow graph runs by default.

[proto17]

The GNU Radio update branch (gr-droneid-update) might not have the latest and greatest MATLAB. I worked on the GNU Radio branch totally separate from the MATLAB code. So that's something I need to look at. For now, assume that main contains the most up to date MATLAB, and gr-droneid-update contains the latest GNU Radio code.

I would worry that as you lower the correlation threshold, the chance of having a timing offset increases. This shouldn't be an issue for the MATLAB/Octave code, but might for the GNU Radio. I don't think the GNU Radio logic makes a second, more targeted pass at the ZC sequence start index. If it doesn't, then a very low threshold would cause the GNU Radio logic to see the start time offset (STO) as being several samples early, which would break demod later on. The other issue with a low threshold is false triggering which will both consume loads of CPU, and cause missed DroneID bursts since the software is busy consuming samples for a phantom burst.

It's also worth noting that a large frequency offset will cause you to get low correlation scores. I don't have access to the flow graph right now, but I seem to recall that I do have a low pass filter in the graph so that you can only see ~ 10 MHz in the center of your baseband. If you have a DroneID burst that only partially comes into your passband, then you might get a tiny correlation spike and think that something was wrong. I recommend looking at the figure; plot(10 * log10(abs(burst_samples).^2)) (log power in the time domain) view of the samples in the burst that you think you missed. You should see an obvious and clean signal present. You can also check the frequency domain to be sure that the signal wasn't clipped by the low pass filter figure; plot(10 * log10(abs(fftshift(fft(burst_samples))).^2);

In the end, lowering the threshold has the most effect on CPU. The lower you make it, the more false positives, and the more time your CPU spends chugging on samples. Because of this, you should be on the lookout for Os in the terminal (assuming you're using an Ettus radio) signifying that your CPU wasn't keeping up. You can also open up top, press H and look for processes that are using 100% of a core. If you see that, and the name matches up to a block in the graph, then you are going to get bad correlations because samples are randomly getting dropped.

Hope that helps!

[alphafox02]

Think I'm on the right track then, plan was to update the gnuradio with the gnuradio update branch, but then switch back to main after it's installed so that the matlab code was newer. That way if someone wanted to save off the IQ file/bursts they could then use matlab/octave scripts? But.. if they wanted to try and get decoding running in the bottom left window of gnuradio - that ability would also be there if they open the new flowgraph.

I just wasn't completely sure what to stick as a default value in the correlation setting in the flow graph. I need to grab another drone and try this out again. I have the b205 and for a sort period, the x310.