alternative to "wine+lrpt offline decoder" combo; and cpu load question.

[ghost]

Sugestion: It is pretty fresh project: https://github.com/artlav/meteor_decoder (with some precompiled binaries: http://orbides.org/page.php?id=1023) Try it instead of "wine+lrpt offline decoder" combo. I will try to feed above decoder with output generated by metor_qspk.py but there comes the question.

Question: meteor_qpsk.py is very heavy on cpu, my raspberry pi3 is loaded on all cores to 100%, (I was experimenting with gui version: https://www.dropbox.com/s/8kc89wriludrrb8/meteor_qpsk_rx_rtl.zip
and cpu load was not that heavy) should it look like that?

Edit: https://wiki.gnuradio.org/index.php/Guided_Tutorial_GRC - copy&pasting missing lines with "blocks_throttle" from gui version at the moment... Edit2: copypasta failed, meddling around rational_resampler_xxx_0 atm. Edit3: Following didn't work as expected, script is somehow running, but still max load:

127,128c127
<         self.blocks_throttle_0 = blocks.throttle(gr.sizeof_gr_complex*1, samp_rate,True)
<       self.blocks_float_to_char_0 = blocks.float_to_char(1, 127)
---
>         self.blocks_float_to_char_0 = blocks.float_to_char(1, 127)
144c143
<         ########self.connect((self.rational_resampler_xxx_0, 0), (self.analog_agc_xx_0, 0))
---
>         self.connect((self.rational_resampler_xxx_0, 0), (self.analog_agc_xx_0, 0))
146d144
<       self.connect((self.blocks_throttle_0, 0), (self.analog_agc_xx_0, 0))
148c146
<       self.connect((self.rational_resampler_xxx_0, 0), (self.blocks_throttle_0, 0))
---
>
180d177
<       self.blocks_throttle_0.set_sample_rate(self.samp_rate)

[cyber-atomus]

As for the suggestion, thanks - I'm already using it. But since my LNAs died, I can't work on Meteor (or any other satcom, it took me near 3 months to collect money for stupid LNA last time). Maybe today I'll push updates I made so far.

As for the rPI3 - I do not own it so I can't tell. Basically it's the same script as the meteor_m2_airspy with totally removed GUI & few things recalculated. It should be faster than GUI version. Which Gnuradio version are you using? What VOLK machine does it run when running meteor_qpsk.py? What's the max CPU load using grc file, does it use Realtime Scheduling? btw: I won't follow links on dropbox, could you paste it on pastebin/gh?

Under normal circumstainces there's absolutely no need to use Throttling unless you feed Gnuradio with file or network source which may 'flood' your flow. IMHO 1.4Msps is way too much for rPI, try lowering samplerate (e.g. 937500, decimation 6).

My experimental setup (rtl_sdr -> csdr u8 to float -> doppler -> netcat -> gnuradio UDP src) takes up to 100% but because of doppler tool which is somehow CPU heavy.

[ghost]

This is probably very similar to the meteor_m2_airspy - https://pastebin.com/dZCgDXmq - one difference is samp_rate_airspy vs. samp_rate_rtl I also tried to change it, without effect on cpu-load. This is load with gui version - (pastebin): This load with your cli version:

Gui script is samplerate 1024000 and decimation 8 and it is ok. But even with lower values your script eats all cores.

When running meteor_qpsk.py there are following information: gr-osmosdr 0.1.3 (0.1.3) gnuradio 3.7.5 Volk warning: no arch found, returning generic impl I think this "volk" it is what you are asking. Unfortunately I have no idea what grc files are - I'am green as grass in this playground.

Both of these .py scripts were able to record today, some weak singnal from meteor-m 2, on DIY QFH antenna directly connected to RTL2838 on rPI3 - without LNA/filters/etc. Receiving NOAA signal on this setup is not a problem. It is first piece of software which generated such a load and heat - the latter is my only concern - 83-84C is too much.

My python skills are very low, so I was just looking for obvious differences, between these two scripts. I can try to deconstruct gui script, droping functions or something, to see when it will become wild on cpu. Or I can try to force running gui script from shell on DISPLAY and kill it after work.

[cyber-atomus]

I'm guessing you're using vanilla raspbian gnuradio. Volk warning: no arch found, returning generic impl - here's your first problem I think. Actually I had something similar on HP t5720 which uses ancient Sempron core which wasn't supported by Gnuradio. Maybe try running volk_profile first to profile the machines (you'll need the libvolk1-bin package)?

As I said before, main script functions are nearly the same as the GUI version and it's compiled from Gnuradio flow with all WX GUI options removed. It shouldn't really eat more CPU than GUI version. I'll check it on site 'cause my satcom devel machine is installed at my garden. btw: 128kSym/s (samplerate/decim) should be OK for Meteors doppler, personally I prefer 156250Sym/s.

Lucky you with the QFH. My QFH is made from random copper wire and I believe it's so misshapened that it needs LNA to run. I had stable and readible Meteors from ~25° elev, now - nothing. Even NOAA is crappy as hell :) Still waiting for copper tubes which my friend promissed me.

[ghost]

Yes, it is vanilla rasbian gnuradio. After launching volk_profile I have: Using Volk machine: neon_hardfp_orc Also I downloaded some grc files (ie. https://github.com/otti-soft/meteor-m2-lrpt) and launched them in gnuradio-companion. CPU load was acceptable: But there is some problem with generated python scripts from grc, about half of a second after launching script window became unresponding for input (at the same time graph is live and the output file is growing). The same amount of load (and also input freeze) happens with samplerate=937500 and decimation=6. Not sure about Realtime Scheduling. Each grc file had it disabled. I tried with Realtime Scheduling enabled, also as root, but I didn't saw any difference or any indicator if grc is using it or not.

As for 100% cpu load: maybe drawing of graph or other internal gui redrawing function is controlling cycles and without that "clock control" script is taking all free cpu resources for monkey business.

My QFH loops are made from coax/tv cable. It is very amateur/test/temporary solution, placed between walls from N/W/S. I saw a very popular guide about how to make QFH antenna in which the wiring seemed to be wrong, author is suggesting - in simple words - to connect small loop with big loop on top (good) AND also on bottom (wrong). I was laughing that this could be one of the best marketing campaigns for some shady LNA sellers, but I will get LNA in the unknown future. The same goes to copper tubes for loops. But first I need stable software platform which will be able to get me comparative material for few weeks in unattended manner, then I will start to modify QFH and other expensive HW. A few days ago I was still using "Horizontal V-dipole" made of metal cloth hanger, it was ok-ish for NOAA's, but I didn't get nothing from Meteor-M2.

QFH:

First contact with Meteor-M2 - only small segment as I was testing today different .py scripts:

NOAA15:

NOAA15 - on "Horizontal V-dipole":

[cyber-atomus]

I feel that rPi is just not fast enough for a GNUradio. I suggest creating very simple FM receiver using gnuradio blocks and comparing CPU usage. On my system it never takes more than 60% of CPU power and there's no difference between GUI/nonGUI version. But my setup uses much, much beefier machine (scrapped laptop mobo with dualcore Intel P9300 CPU).

I pushed latest devel python script I'm using for a comparison - it creates config files for Oleg LRPT decoder and simple script for autodecoding usign medet. Remember to change variables. Later I'll push changes to main "loop" script.

As for QFH+LNA - here are my results few days ago before LNA died.

Pozdrowienia :)

[ghost]

My cutting-edge tech rPi3's 1.2 GHz 64-bit quad-core ARM Cortex-A53 can't be that bad... ;)

Ok, It looks like volk_profile did help a little. 3x100% and only ~65% on fourth core.

Funny thing, when launched while rtl was busy by another process load was 3x100% and 15% on fourth core - while virtually doing nothing.

I wonder if, in your case, it is taking advantage of all cores / multiprocessing? (bug test and some more info) 50% on dual core is still 100% on one core.

I tried to force meteor_qpsk.py to single threaded run (just for test if it will record anything on one core) with:

psutil.Process()
p.cpu_affinity() 
p.cpu_affinity([0])

But as it works with mentioned "bug test", it does not work with meteor_qpsk.py (it's beyond my python knowledge). So, maybe I'll get back to it later. Also when I get some free time+place sync to run linux there, then I'll test meteor_qpsk.py on my main i7 pc - I am curious if it will eat 8 threads.

I'll fiddle with GRC a little more, it seems to be a lot of fun, I'll start with some basics as you suggested.

Meteor_qpsk.py is doing its job - until summer rPi3 should withstand @ ~83C for <15m periods while receiving Meteor's signal. Probably in the meantime some solution, SW or HW, will born by itself and if by software then I'll let you know.

Can I have your grc file (this one which was used to compile meteor_qpsk.py)?

There is no doubt I'll get some LNA, your results are amazing. Today's morning MeteorM2 pass - received with your meteor_qpsk.py - this is all I get:

Również pozdrawiam!

[cyber-atomus]

I believe A53 used on rPi3 can be that bad on GNURadio. I don't have access to any of the raspberrys so I can't test anything. In my case it's taking advantage on all cores and CPU usage "floats" across CPUs. Running the script with radio busy will replace rtl_sdr source with NULL source and will run the same way as with rtl_sdr - it's GNURadio thing.

Anyways I recreated flowgraph with latest GNURadio. It shouldn't matter but it seems to be a little bit faster.

Also consider using radiator/radiator+fan as these temperatures are way too high, even SoC needs to have proper cooling sometimes.

[ghost]

Yeah, cutting edge tech <- that was a small joke.

I was checking official specs and user reports to calm down my doubts. It has 85C max by design and cpu frequency is throttling down from around 83C. Radiators are on board, I don't sail completely barefoot.

Your last modification did the trick, cpu load is still high, but it is reasonable high: Also temps are 1-2C lower.

Thank You!

[cyber-atomus]

Glad that it helped a little. I'll try to optimize it further. I'll just wait till morning pass to be 1000% sure it really works better :)

[ghost]

I am looking forward to the updates.

Last 3 passes look like below for me on this new meteor_qpsk.py, but I can't tell if this is script fault or weather and soaking wet. I'll do more records to see if it will get better.

[cyber-atomus]

Mine today and yesterday evening

But yesterday's morning pass was a mess. I'm using the very same script as latest push on gh. Have you checked ALL variables (especially PPM/gain)?

[ghost]

I get shredded images - at best - with every settings and versions of meteor_qpsk*.py. The only one mile-stone which changed between something and almost nothing was volk_profile - I can undo this and try testing from beginning.

But I think, that I need to take few steps back and work out meteor from windows tools. If I get good repeatable results, then I'll get back to rPi3 & linux knowing what I can expect. It was long shot after positive results with NOAA's, but I ran out of shell-luck and I need to make up some basics concepts. ;)

Yesterdays NOAA19:

[ghost]

My experimental setup (rtl_sdr -> csdr u8 to float -> doppler -> netcat -> gnuradio UDP src).

^ Is this experimental setup a realtime one or can you push somehow rtl_sdr generated file to the gnuradio? (I'll get to the "why am I asking")

I was testing lately various setups: "rPi3 rtl_tcp -> win SDR# + Orbitron doppler + QPSK Modulator -> Realtime TCP -> LRPTofflineDecoder"; above + offline "SDR# IF Recorder -> LrptRx.exe (deWAVulator) -> LRPTofflineDecoder"; and of course "rPi3 rtl_tcp -> win various GRC/Python scripts" ( Meteor QPSK demodulators included);

Working on pre-recorded stuff by "SDR# IF Recorder" got me worse effects than in real-time (and using in "real-time rtl_tcp -> GRC" gives even better effects) so I was searching for another way to do offline test routine, especially for use in linux&GRC.

The case is that at one time I recorded Meteor pass with rtl_sdr, and I can see his spectrum in gqrx (I did bit shift/conversion which I think is mentioned by you in above quote "csdr u8 to float". Screen below, this is GRC conversion but I also found some .c converter for use in pipe).

Gqrx's export options are pretty bad, so I just used it to confirm that there is some Meteor signal in rtl_sdr dump spectrum. I wanted to push this file into python QPSK scripts via osmocom or other file source ,

but I get errors from clock recovery MM module:

when using bit-converted (also tested raw without conversion) rtl_sdr output. I tested with the same method if fm_radio in GRC can take and use such file and it can. There is probably thousand possibilities what gone wrong, but I am intrigued by this question in quote of yours. It looks to me that you are able to feed GRC - indirectly - with rtl_sdr output? Are you using rtl_sdr "-S" switch to force "sync"? Does it looks to you that I am using the same bit-conversion? Or is the "UDP Source" that magicly works with piped output through netcat?

I also tried other ways to offline feed GRC with files from others sources, with no luck. Obviously having that kind of offline test method would help me with testing scripts settings in GRC. At the moment I have scheduled generating ".s" files via rPi3 rtl_tcp & GRC on my main pc with windows, but it is live testing. I would like to do GRC tests parallel on windows & rPi3 for comparisons - using the same offline source.

Aside from searching test routine: I know that there is something horribly wrong my with rPi3+GRC. I'll try to build gnuradio from source. Freezing wx interfaces and this cpuload - for example I tried above bit conversion on rPi3, it was very awful - I should not compare it to seconds on one of i7 cores, but it was 100% and very slow file size rises.

Today's Meteor via rtl_tcp -> GRC - best result so far and it was overlaping with NOAA18 signal today. It is without use of doppler corrections - I hope that this is not the one-time victory, but indicator of progress:

[cyber-atomus]

As for the experimental setup - it was exactly this:

rtl_sdr -s 1406250 -f 137900000 -g 49.6 -p 63 -  |\
csdr convert_u8_f |\
doppler track --samplerate 1406250 --intype f32 --outtype f32 --frequency 137900000 --tlefile /tmp/weather.txt --tlename "METEOR-M 2" --location lat=XX,lon=XX,alt=XX |\
nc -l 127.0.0.1 1234

which generally runs rtl_sdr which produces 8bit unsigned int, csdr converts it to float - as doppler prefers float32 (I didn't notice any difference in CPU usage using int16/f32) IQ - pushes it to doppler tool - and then f32 IQ is sent via netcat to GRC. And yes, GRC is using UDP source. There's no need to use throttling but you have to convert f32 to complex in GRC.

If you're feeding gqrx/GRC script with prerecorded IQ file you have to convert unsigned int to complex.

I'll try to record IQ data today/tommorrow and play with it a little bit too.

rPI3 as an embedded device is IMHO way to sloow to use such complex tools (but hey, I was compiling my kernel on MMX Pentium back in the good ol' days!). I'm trying at my best - but as I said I can't do too much right now as I have too low SNR, my WXsat machine is installed ~3kms from the room I live, GSM connection is waaaayyy too slow and eats a lot of data. Plus I never had rPI in my hands and I stopped using MS Windows very, very long time ago :)

I can compare two different Gnuradio compilations - stock Debian and 3.7.12git. But with other script I didn't notice any huge changes in resource usage, plus I'm using core2duo P9300 (I have the very same mobos in my laptop and WXsat machine).

[cyber-atomus]

I put to test rtl_tcp+doppler "emulator" to check tomorrow passes - now I can use osmocom rtl_tcp source :) Unfortunately doppler tool is crazy resource hungry, can't tell why. I already tried several cargo versions to compile it...

rtl_sdr -s 1600000 -f $FREQ -g 49.6 -p 63 -  |\
csdr convert_u8_f |\
doppler track --samplerate 1600000 --intype f32 --outtype f32 --frequency $FREQ --tlefile "$TLE" --tlename "$SAT" --location lat=X,lon=X,alt=X |\
csdr convert_f_u8 |\
nc -l 127.0.0.1 1234

BTW: I made some different tries - I've disabled everything leaving only Osmocom source->Null sink and CPU load was only 5-10% lower than running full script. Running no-gui version causes CPU usage drop to ca. 20% on my machine. Really no huge difference between precompiled 3.7.5 (Debian Jessie standard) and optimized 3.7.12 build, but signifficant difference between x86/x64 builds/hosts.

[ghost]

After befriend with the gnuradio on windows (without strange freezes) I wanted to do some funny&simple tests on rPi3 with "simple fm receiver". But while experimenting on "WX GUI FFT Sink" on rPi3 I discovered that changing "Refresh rate" parameter from 15 to 2 causes no more freezes! Moreover it causes much lower cpu usage!

I wanted to compare "Osmocom source->Null sink" vs. "Osmocom source-> some stuff -> Wav file sink", but this collateral outcome is so huge:

Src -> Null sink (WX NO GUI): CPU ~77% Src -> Null sink (WX GUI FFT Sink Ref.rate 15): CPU ~52% Src -> Null sink (WX GUI FFT Sink Ref.rate 2): CPU ~20%

Src -> Wav sink (WX NO GUI): CPU ~86% Src -> Wav sink (WX GUI FFT Sink Ref.rate 15): CPU ~60% Src -> Wav sink (WX GUI FFT Sink Ref.rate 2): CPU ~33%

Generating Wav file with some more modules takes 8-13% of CPU; Lowering "Refresh rate" of "WX GUI FFT Sink" gives 27-32% of CPU and gets rid of freezing problem; NO GUI option makes script go wild all over again - 53-57% of CPU more - it is madness. Realtime Scheduling - gives no difference. I didn't try with throttle - it has no matter now. I tested created .wav file to see if there is no garbage inside and it was good quality FM Radio mono recording.

So, I will now be modifying grc meteor/qpsk scripts to use gui and lowest "refresh rate" of gui modules and I'll get back to the idea of scheduling records with use of: DISPLAY=:0.0 python meteor_qpsk_xxxx.py

It seems to be the best "low effort" option ATM, if gnuradio can take advantage of above "refresh rate" trick and will produce some nice "soft bit" files - on rPi3 - without drowning, then I will take that chance. We'll see.

As for doppler, it is on my list, because after while with Orbitron+SDR# it seems strange not using it even for NOAA's. I got some readings to start with.

Screenshots of tests, because I would not believe without seeing this, it is so wicked: Src -> Null sink (WX NO GUI): CPU ~77%

Src -> Null sink (WX GUI FFT Sink Ref.rate 15): CPU ~52%

Src -> Null sink (WX GUI FFT Sink Ref.rate 2): CPU ~20%

Src -> Wav sink (WX NO GUI): CPU ~86%

Src -> Wav sink (WX GUI FFT Sink Ref.rate 15): CPU ~60%

Src -> Wav sink (WX GUI FFT Sink Ref.rate 2): CPU ~33%

PS. After quick look on above screens clocks my heart stopped for a moment, but rtl wasn't busy with passing NOAA's... Triple checked. That would be crap-test of the decade. ;)

Edit: Just one more quick thought - situation like below, both of them have good elevation - with rtl_fm it is impossible, but with rtl_sdr and recording whole 137-138MHz... I need to think about it too. :thinking:

[ghost]

Nope. Tricks from fm radio tests didn't work for qpsk. GUI is less freezed than before, but I get "OOOOOO" in console output of the script and buffer in rtl_tcp is also overloading. It is just piece of junk for gnuradio purposes, as you were saying in the beginning. No more hope.

From now on I'll better focus on recording IQ of Meteor passes on rPi3 for processing it offline on more capable pc.

[cyber-atomus]

:artificial_satellite:

Nah, I made little mistake and destroyed morning tests. But I made a little progress with GRC. It seems that using Osmocom source is the killer. Then I tried using deprecated TCP source with float input (_rtlsdr -> convert u8 to float -> netcat -> GRC) and CPU usage dropped by ~50%! Also I've found that using unbuffered file output drops IO usage by 2-3%. So there's a good chance it will work if GRC is fed with float over TCP or UDP. :goal_net:

For more tests I'll need to have huuuge 1.6Msps RAW I/Q file created by rtl_sdr - I'll try to do it with the evening pass. Couldn't find any usable raw IQ on the net.

Never give up they say...

[cyber-atomus]

TCP with GUI - 2fps refresh rate - CPU goes wild 50-75% TCP without GUI - no throttle used, CPU stays on 13-30%

Can't tell if I'll get usable QPSK, but RMF FM doesn't stutter (for tests I've added WBFM decoder+audio sink after AGC).

[ghost]

Ha! So, I have new test setup and it is mindblowing. [ It is based on two versions of meteor_lrpt_baseband_processing by Martin Blaho. The second one is named soft_symb_wave.grc and for input takes wav file. It shouldn't matter what version, because everything important happens before AGC, screens somewhere below ]

1. rtl_tcp -> grc (live and at the same time):
   • --> live processing to soft-bit file.s;
   • --> recording to wav file (IQ as I presume);
2. Offline processing of wav file -> tweaks on wav file -> wav into grc -> .s file into image decoding

This is how wav file is exported just after Low Pass Filter (here throttle was yellow=bypassed and not used):

This is how wav file is provided to grc (here throttle is quite important it is set to samp_rate*3, with bypassing this module it took 15s to process 7,5min of wav@130kHz on i7, but I'll consider using it on rPi3 later):

In live mode I was messing too much with PLL Alpha, etc. and live generated .s file was not usable - no image at all from its .s file.

And then I started playing with generated WAV. I processed it with original 156.25kHz (as it was recorded samp_rate/decim 1406250/9) and nothing. But when I resampled this wav to 130kHz things changed. It was still awful-piece-of-garbage, but I knew that there is "something".

I was reading lately some stuff from dropbox link and sow there an idea:

After few tests in Audacity it seems that Amplification 15db + Normalization with DC removing AND resampling to 130kHz gives best effects. Below image generated on the base of tweaked wav file:

I'd called it progress!

Now only porting this to rPi3 in command line versions skipping that part where .s file is produced live. Recording that wav IQ(?) file, even with grc should be possible (I'll try it later). Resampling etc. in sox. Then offline processing with grc at some slow settings. Medet. And last thing missing will be rectifying. Hope is back. ;)

Edit: Recorded wav file was 140MB and resampled 234MB for 7m30s. Earlier recorded output from rtl_sdr was in gigabytes so this is better.

Edit2: Resampled wav is also compatible with with this deWAVulator which runs on windows, but this tool took 15m to convert mentioned wav, when grc based conversion on windows - w/o throttling - runs fast as hell: Edit3(last): Usable data was @elevation from ~30deg inbound - 34deg outbound with max at 71deg - antenna below roof level. :) And now I am wondering how many times I was using this script with wav output... Maybe something can be recovered... ;)

[ghost]

I found in browser-history (that was hard) source from which I get to these scripts: PDF from meteor.robonuka.ru and inside is description and links to dropboxed rars with grc's.

Also they experimented with ".sam" files, I wonder what this is:

rPi3 as expected is refusing to even record wav without sick cpu loads, and I don't like how this wav look. I don't know maybe I'll try to record Meteor pass with rtl_fm, and then process it somehow, I don't even remember if tried to do it. Even so, it was probably on 137.1MHz and I was tracking METOP-B instead Meteor-M2. Somehow it looked more russian-ish :man_facepalming: and then I switched to autowx and grc. Will try, later.

[ghost]

I was trying to record wav and create .s file from wav with grc in command line mode. It needs clean exit, otherwise files are empty. I got "clean exit" only with WX mode by closing window... So, no matter if it is linux or windows it needs freaking window, inexplicable. I can try to kill it with other signals, but come on...

While recording yesterdays pass of Meteor there was also singnal of NOAA18, so I got images from both satellites based on one wav file and resampling. I think that I'll try to record Meteor with rtl_fm and then process it somehow. Csdr looks promising, maybe it'll be not so hard to complete tools to retract qspk without GRC. Some fast C language based would be the best. Medet results are not always good, it is a little throwback, so always compare with a certain method.

I was lucky today, full pass without black "reset line" in the middle. (Recorded with rPi3 rtl_tcp -> GRC in windows to WAV -> audacity -> GRC .wav to .s -> LRPTOfflineDecoder 2015.5.10.0027 -> rectify in LrptImageProcessor. Overall very manual tasks - need automation ASAP :) ):

[ghost]

I get rid of recording Meteor passes with Osmocom source in GRC totally. Even with gnuradio 2.7.10 using this source was erratic on rPi3 (but I didn't build gnuradio or gr-osmosdr from source, I've used testing rep and even with apt-get it took many hours on rPi3, I am too lazy to set up some cross-compiler, yet).

I used your tip about csdr and after few days of using Proob&Buendoff method, I ended up with something like this:

Step 1: Live - IQ recording | u8 to float | decimate 9 | Write to wav (Signed/16bit/2ch/156kHz) file timeout 540 rtl_sdr -p 55 -s 1406250 -g 43.4 -f 137.9M - | csdr convert_u8_f | csdr fir_decimate_cc 9 | sox -t raw -r 156250 -e float -b 32 -c2 -V1 - -e s -b 16 -c2 IQ_WAv_156kHz.wav Step 2: Offline - "Effect Normalize"
sox --temp ./ -R IQ_WAv_156kHz.wav IQ_WAv_156kHz_norm.wav --norm Step 3: Offline - Resampling to 130kHz sox --temp ./ -R IQ_WAv_156kHz_norm.wav IQ_WAv_130kHz_norm.wav rate 130000 Step 4: Offline - Wav to "72k Softbit" wav2s.py --infile IQ_WAv_130kHz_norm.wav --outfile Softbit_file.s --pll-alpha 5m Step 5: Offline - "72k Softbit" to Image medet Softbit_file.s ImageName -cd -S -q Step 6: Rectifying ????

It works without gui. The most CPU-killing tasks are done out of GRC, so it is pretty light even for rPi3. GRC is used only to convert wav into ".s file", in Step 4, and it eat ~45% CPU, for 10min long record in wav it needs 2m15s ( wav2s.grc wav2s.py ). I didn't used GRC for conversion of raw dumps to wav because they were too big and it was too CPU heavy. In this order wav files are relatively small, 10min wav file after Step 1 is only ~330MB, then after Step 2 130kHz wav is ~275MB. Probably there is a field to shorten the steps list further, but sox can't do "--norm" on stream, so it must work on dump. Better results in end-image are with "--norm" on 156kHz than 130kHz, so there is another step. In original versions there were things like "low pass filter", "rational resampler" or "Root raised cosine filter" with some serious looking parameters apart from decimation, but I use "csdr fir_decimate_cc 9" and no other decimation in grc (as I don't know anything about filters I took first option from a row of csdr's options with decimation and it worked). ;) In sox command "--temp ./" is only because my /tmp is on too small tmpfs.

Offline method gives better control over results IMHO, but the most important is that rPi3 can now do it alone without getting fever and losing parts of the stream while overloaded. I just need to put there some old pen-drive for temp files.

I was sniffing in gpredict and it have some rotor/radio control settings, but for now I think that I may more in need of LNA than doppler correction. Still I didn't found rectifying tool for linux.

Wav2s flow:

Yesterday evening pass of Meteor - Negative and rectify done in windows, but other than that all using rPi3 shell:

[ghost]

I've done a quick test of this doppler now, with rtl_fm and it takes only one core for himself, so it's 25% cpu consumed - could be worse. Although I don't know yet if output isn't scattered, but there is always offline dump processing option...

[cyber-atomus]

So OK, I spent last few weeks changing cables, shielding and creating a LNA. I got few conclusions from all this 'rework' (you should see the mess I did at the attic). At first - one needs a very clean 137.900MHz +/-160kHz signal - this is a must, any spikes will destroy any qpsk signal. Second - gnuradio 3.7.5 from Debian Jessie was the worst problem which took me some time to resolve. Here's the difference on 3.7.5 (left) and GIT 3.7.12 (right) on my computer. I thought something's odd - but I didn't catch it before. Looks like resampler or AGC is not working as it should. Upgrading GnuRadio to 3.7.10 (from Jessie Backports) resolved it immediately: But on rPI you're on your own since I don't have rPI and being too poor to afford one ;)

Now it seems that script work flawlessly. As for the CPU usage - the most CPU consuming blocks are: Osmocom RTL-SDR Source, Constellation Soft Decoder and Resampler IIRC. I lost screenshots somewhere - but if you want to check which block is most IO/CPU consuming I suggest using iotop and htop. IOtop shows block names & PID very nicely.

I think there's no need for doppler as the signal shifts only 2-3kHz - I tested it and it seems it just eats CPU power while doing little work. Samplerate of 1250000 and decimation of 8, which gives 156250S/s , seems to be a perfect choice.

The results are awesome despite having bad antenna. I'm still short of money to build a new one.

Todays 24° elev. pass Yesterday 11.37. Yesterday at 23.09 at only 14° elev.

[ghost]

Yeah, in the meantime I've purchased LNA4ALL and changed "feedline" - beetwen receiver and antenna/LNA from cheap 75Ω DVB-T cable to <10m of H155 50Ω. My QFH antenna is still from 75Ω DVB-T cable - for now I'll just want to take few days/weeks for sampling some results and then perhaps next change.

I ended doppler tests, I will not use it, it is dead end, at least with rPi3.

My last automation looks like this:

# Live stream processing
timeout 660 /usr/local/bin/rtl_sdr -p 55 -s 1250000 -g 49.6 -f 137.9M - | \
csdr convert_u8_f | \
csdr fir_decimate_cc 8 | \ 
sox -t raw -r 156250 -e float -b 32 -c2 -V1 - -e s -b 16 -c2 IQ_WAV_156kHz.wav

# Offline processing
sox --temp ./ -V -R -D IQ_WAV_156kHz.wav IQ_WAV_156kHz_N+DC.wav --norm dcshift +0.001041
wav2s_156kHz.py --infile IQ_WAV_156kHz_N+DC.wav --outfile IQ_WAV_156kHz_N+DC.s --pll-alpha 4

# Image rendering
medet ...

I am recording 11 minutes only (400MB file). I've get rid of this step with resampling to 130kHz just tweaked my gnuradio script to use 156.25kHz instead. In last-minute today I've changed rtl_sdr sampling and decimation as you suggested, and I think I'll try at evening pass to set rtl_sdr -s 156250 and get rid of csdr fir_decimate_cc 8.

I've added dcshift - it is hard set to +0.001041 (it is not changing too much) but there is possibility to get that value from sox file.wav -n stats with awk. "-D" in sox is forcing to not dithering output. "pll-alpha" 0.004 seems to be sweet spot for this setup.

This is automated. But I am comparing above outputs/results with manual operating on "IQ_WAV_156kHz.wav", which is output file from first step "Stream processing". I am still trying some amplifications, etc - sometimes it gives nothing, but sometimes edges of recording will get additional 10-20 lines, etc. Also I am trying different pll-alpha values, just-to-be-sure. Test tree can grow very fast. Four different amplification values four different pll-alpha values, gives 16 different results bmp channels... and so on... and on... :deciduous_tree:

Now with this two HW changes in system (feedline and LNA) I need to overcome gain settings to see what is good and what is bad. I should consider some shielding on receiver and LNA (it is bare pcb - "oh, it doesn't need housing for now, it is just for tests..." - was my onion-hearted words). It will probably be better to use bias-T for powering LNA and not this "220V->smartphone charger->usb cable->lna" under open sky...

But there was small window in rain and there is some result of this brand new setup:

Recorded between 11:19 - 11:30 with max @50°

WAV spectrum in Audacity - gives me notion of signal strength - with some hand drawings (red is start/end of qpsk/rendered image):

Pass overwiev - same colors - more or less exact positions:

Examples o little differences between output "as is" and with amplifying(no pattern in settings to automate, still just Proob&Buendoff methods):

Your results at those low elevations are just wow! I should try to record some pass with max elevation below 45° - I didn't even bother earlier. And still, my results are from 20-24°. I was considering some outdoor activity at some high point out of city, to exclude city-noise and get some good horizon visibility, but it makes me a little nervous, so I am delaying that kind of event.

I'll look into your new gnuradio script. My "filtered spectrum" (from processed wav file) looks like this - (at the strongest signal level near middle of recording):

Should it be flat? I did't change flow in grc from last screenshots, fft-sink is just after AGC.

And qpsk looks just like qpsk:

Edit: I was looking on your script, and in my there was no "Root Raised Cosine Filter", I don't know yet what it does and if rPi3 will stand that, but it gave me additional coverage:

[ghost]

Yesterdays test without decimation was a failure, because rtl_sdr didn't like so low sample rate:

Invalid sample rate: 156250 Hz
WARNING: Failed to set sample rate.

Todays 11:00 - 11:09 @68° was nice for tests. Best results on pll-alpha 0.004, but southest 100px with Mallorca Island were possible only with pll-alpha 0.001. And "Root Raised Cosine Filter" adds 10% of load to whole 50% of the cpu on rPi3, but it doesn't matter.

Left - without "Root Raised Cosine Filter" - ".s file" generated on rPi3; Middle - with "Root Raised Cosine Filter" - ".s file" generated on rPi3; Right - combined from 5 different versions of settings/wav file tweaks:

And gif with few of tested setups ;)

[ghost]

I had 1m30s of some interference on todays evening pass. ~I wonder if that was some kind of reset on meteor board or signal from other source.~

~Was your reception of this pass ok?~

Edit: I confirmed with AmigosViewer, that Meteor was more than ok. My LNA probably fryed at this very moment, oh well...