iq_server continues fft scanning mode

[flux242]

I tried to request fft scans in a loop but then my script stopped scanning after some time. Didn't look into the problem but added a continues fft scanning mode. So basically if the fft slot allocated during start is not in continues mode then reuse it, otherwise use another available slot. Solution works stable. Didn't want to create a PR for that but here are changes

https://github.com/flux242/radiosonde/commit/72666057cbf20e459a1bdf4b4a2840cf60d6e438#diff-9c15688c9d0f74763039a4867455fd9d13e543df4a8958fb4ed4da802a6e478e

[rs1729]

Don't know how frequently your loop was trying to scan, maybe the old connection was still open. There is a more verbose --dbg output for the server, and you could also check with lsof -i | grep :1280 or ss -ntmp | grep :1280 the connections (assuming PORT 1280), when it stops working.

Newer processors probably don't have problems running an additional FFT thread and continuous scanning. On older laptops or raspberry, I had in mind maybe one scan per minute or two, and then all the resources for demodulation and decoding. Although if there is no signal found yet, scanning more frequently would probably make sense. Also on newer/faster computers continuous FFT data could be used for spectrum display I guess.

I don't have much time this weekend, will look at it next week. A few remarks: Don't know if you use the same FFT data format, for continuous FFT data, probably a more compact csv format (similar to rtl_power) with one line per FFT would be better. I did this when I wanted data for waterfall display, i.e. the FFT output for waterfall is not the average over 2 seconds but several consecutive FFTs, shorter intervalls within the 2 seconds. Maybe I include this next week. A continuous stream of csv-FFT-lines is probably easier to process. Something like (with old in //comments):

    if (fpo != NULL) {
        //fprintf(fpo, "# <freq/sr>;<dB>  ##  sr:%d , N:%d\n", dsp.DFT.sr, dsp.DFT.N);
        fprintf(fpo, "%s, %s, ", "2020-10-01", "00:00:00"); // time steps?
        //fprintf(fpo, "%.2f, %.2f, ", bin2fq(&(dsp.DFT), dsp.DFT.N/2), bin2fq(&(dsp.DFT), dsp.DFT.N/2 - 1));
        fprintf(fpo, "%d, %d, ", (int)bin2freq(&(dsp.DFT), dsp.DFT.N/2), (int)bin2freq(&(dsp.DFT), dsp.DFT.N/2 - 1));
        fprintf(fpo, "%.2f, ", dsp.DFT.sr/(double)dsp.DFT.N);
        fprintf(fpo, "%d, ", dsp.DFT.N);
        for (j = dsp.DFT.N/2; j < dsp.DFT.N/2 + dsp.DFT.N; j++) {
            //fprintf(fpo, "%+11.8f;%7.2f\n", bin2fq(&(dsp.DFT), j % dsp.DFT.N), sum_db[j % dsp.DFT.N]);
            fprintf(fpo, "%7.2f", db[j % dsp.DFT.N]);
            if (j < dsp.DFT.N/2 + dsp.DFT.N - 1) fprintf(fpo, ",");
        }
        fprintf(fpo, "\n");
        //fclose(fpo);
    }

I hope I copied the right section, have to go through this next week. Then I noticed, you use the filename fname_fft = "/dev/stdout". Independent of how portable the rest of the code is, I would probably open/use stdout directly for stdio.h library functions (e.g. FILE *fpo = stdout;) or the STDOUT_FILENO file descriptor.

[flux242]

I've made a change in my repo to address the issues you mentioned. Actually, this was my initial intention to use a single string with power levels because it makes things easier on the receiving side but I decided to keep your syntax. Now CSV string is sent by the server. Changes on the client side of your original code are minimal. Please check the diff here https://github.com/flux242/radiosonde/compare/89d5d00c158509a378d505d41e4a7ba971f32e39...8c9aa1fafb78f4ec83e12187c4cd7c9871da98c4

only changes to iq_server.c and iq_client.c are relevant from the diff above

[rs1729]

The txt output is easier to read, but the more compact csv is better if there are multiple FFTs needed e.g. for waterfall. Don't know if you thought about stopping the "continuous" FFT output at some point to keep file sizes small. There are many different possible scenarios, I did not want to add too many options, but I'm not sure which options I would really use.

I changed everything to csv and added the FFT for waterfall, so there are now --fft_avg and --fft_all("avg"->sum of FFTs to reduce noise, "all"->waterfall (one could think of other abbreviations)). The client options would have the suffix _clor _sv, the latter to save the FFT at the server, though this is only possible, if iq_server is started with option --enable_clsv_out. Then an integer parameter is needed for the number of FFTs, and a filename, i.e. --fft_avg <m> <fft_avg.csv>. m=-1 would be continuous output, which can be stop, if a client sends -<n> where nis the channel of the FFT. filename="-" is for stdout. https://github.com/rs1729/RS/tree/test_iq_svcl_dc/demod/iq_svcl For FFT only one channel can be used, and this channel would be reused if another FFT is requested by a client, but only if the last FFT is finished.

Example: ./iq_server --fft_avg 1 fft_avg.csv [...] python plot_fft.py fft_avg.csv

./scan_fft fft_avg.csv
avg=-87.45
min=-91.66
peak: -0.31732178 =  -649875 Hz  (8.9dB)
peak: -0.21972656 =  -450000 Hz  (10.1dB)
peak: +0.36627197 =  +750125 Hz  (25.0dB)
bin = 125 Hz

./iq_client --fft_all_cl 4 fft_all.csv # 4: 4*2=8 seconds python plot_wfft.py fft_all.csv

It should be possible to have live matplotlib updates. For the more frequent waterfall-FFTs a continuous output to file could mean huge file sizes; for live waterfall display could keep e.g. the latest 10 seconds from stdout. I didn't add your partial tcp-transfer (yet), I didn't see any problems, but I didn't test this very much.

[rs1729]

plot_fft_ani.py <n> <filename> plots db-spectrum (n=1), waterfall (n=2), or both (n=3). If filename="-", it reads stdin. E.g. [Terminal 1]

T1$ rtl_sdr -f 405.0M -s 960000 - | ./iq_server --fft_avg 1 fft_server.txt --bo 32 - 960000 8

and after initial server-scan [Terminal 2]

T2$ ./iq_client --fft_all_cl -1 - | python plot_fft_ani.py 3 -

UPDATE: test_iq_svcl_dc -> test, i.e. https://github.com/rs1729/RS/tree/test_iq_svcl_dc/demod/iq_svcl -> https://github.com/rs1729/RS/tree/test/demod/iq_svcl

[flux242]

I haven't tried your update yet, but it should do. The file size is not an issue for me because I don't create a file with scans and I also never stop the fft client. I've added some description to my script . That's an example of your iq_server concept usage ; )