threadpool.py init error

[Laserdance100]

Problem1: soapy_power -r 2.56M -f 88M:98M -B 500k -F rtl_power -O output.txt --even -T 1 --debug Detached kernel driver Found Rafael Micro R820T tuner Reattached kernel driver DEBUG: pyfftw module found (using 2 threads by default) DEBUG: scipy.fftpack module found Detached kernel driver Found Rafael Micro R820T tuner DEBUG: Applying fixes for RTLSDR quirks... INFO: Using device: RTLSDR DEBUG: SoapySDR stream - args: {'buffers': '100'} [INFO] Using format CF32. DEBUG: SoapySDR stream - buffer size: 8192 DEBUG: SoapySDR stream - read timeout: 0.103200 INFO: repeats: 106667 INFO: samples: 640002 (time: 0.25000 s) INFO: max_buffer_size (samples): 32768000 (repeats: 5461333.33, time: 12.80000 s) INFO: buffer_size (samples): 647168 (repeats: 107861.33, time: 0.25280 s) INFO: buffer_repeats: 1 Traceback (most recent call last): File "/usr/bin/soapy_power", line 9, in load_entry_point('soapy-power==1.4.0', 'console_scripts', 'soapy_power')() File "/home/X/.local/lib/python3.5/site-packages/soapypower/main.py", line 310, in main max_threads=args.max_threads, max_queue_size=args.max_queue_size File "/home/X/.local/lib/python3.5/site-packages/soapypower/power.py", line 280, in sweep reset_stream=reset_stream, max_threads=max_threads, max_queue_size=max_queue_size File "/home/X/.local/lib/python3.5/site-packages/soapypower/power.py", line 192, in setup lnb_lo=lnb_lo, max_threads=max_threads, max_queue_size=max_queue_size) File "/home/X/.local/lib/python3.5/site-packages/soapypower/psd.py", line 31, in init thread_name_prefix='PSD_thread' File "/home/X/.local/lib/python3.5/site-packages/soapypower/threadpool.py", line 7, in init super().init(max_workers or os.cpu_count() or 1, thread_name_prefix) TypeError: init() takes from 1 to 2 positional arguments but 3 were given

code in threadpool.py: class ThreadPoolExecutor(concurrent.futures.ThreadPoolExecutor): """ThreadPoolExecutor which allows setting max. work queue size""" def init(self, max_workers=0, thread_name_prefix='', max_queue_size=0): super().init(max_workers or os.cpu_count() or 1, thread_name_prefix)

fix to the problem1 super().init(max_workers or os.cpu_count() or 1)

Then it works. I have even done my first .csv export spectrum analysis with my airspy. I know two others softwares that do the .csv export for the RTLSDR, but none with the airspy.

Problem2: soapy_power params: {'bin_size': 100.0, 'crop': 90.0, 'device': 'rtlsdr', 'gain': 20, 'hops': 0, 'interval': 60.0, 'ppm': 0, 'sample_rate': 2160000, 'single_shot': True, 'start_freq': 24.0, 'stop_freq': 1765.0} Found Rafael Micro R820T tuner Found Rafael Micro R820T tuner Exact sample rate is: 2160000.051498 Hz INFO: Using device: RTLSDR WARNING: number of overlapping FFT bins should be even, changing overlap/crop factor to 0.90909 [INFO] Using format CF32. INFO: repeats: 665 INFO: samples: 14630 (time: 0.00677 s) INFO: max_buffer_size (samples): 32768000 (repeats: 1489454.55, time: 15.17037 s) INFO: buffer_size (samples): 16384 (repeats: 744.73, time: 0.00759 s) INFO: buffer_repeats: 1 INFO: overlap: 0.90909 INFO: bin_size: 98181.82 Hz INFO: bins: 22 INFO: bins (after crop): 2 INFO: sample_rate: 2.160 MHz INFO: sample_rate (after crop): 0.196 MHz INFO: freq_range: 1741.000 MHz INFO: hopping: YES INFO: hop_size: 0.196 MHz INFO: hops: 8867 INFO: min_center_freq: 24.098 MHz INFO: max_center_freq: 1765.058 MHz INFO: min_freq (after crop): 24.000 MHz INFO: max_freq (after crop): 1765.156 MHz ERROR: len(x_axis) != len(y_axis) It seems that len(y_axis) = hops * bins_after_crop, but that len(x_axis) is slightly bigger. Flooring required?

Problem3: soapy_power params: {'bin_size': 1000.0, 'crop': 90.0, 'device': 'rtlsdr', 'gain': 20, 'hops': 0, 'interval': 60.0, 'ppm': 0, 'sample_rate': 2160000, 'single_shot': True, 'start_freq': 24.0, 'stop_freq': 1765.0} Found Rafael Micro R820T tuner Found Rafael Micro R820T tuner Exact sample rate is: 2160000.051498 Hz INFO: Using device: RTLSDR WARNING: number of FFT bins should be even, changing to 4 WARNING: number of overlapping FFT bins should be even, changing overlap/crop factor to 1.00000 Traceback (most recent call last): File "/usr/bin/soapy_power", line 9, in load_entry_point('soapy-power==1.5.0', 'console_scripts', 'soapy_power')() File "/usr/lib/python3.5/site-packages/soapy_power-1.5.0-py3.5.egg/soapypower/main.py", line 330, in main File "/usr/lib/python3.5/site-packages/soapy_power-1.5.0-py3.5.egg/soapypower/power.py", line 105, in freq_plan ZeroDivisionError: float division by zero It happen when cropping is set to 100%, which is not a great idea anyway.

code in power.py: freq_range = max_freq - min_freq hopping = True if freq_range >= sample_rate_crop else False hop_size = self.nearest_freq(sample_rate_crop, bin_size) hops = math.ceil(freq_range / hop_size) if hopping else 1

Problem4: When I set a 2dB gain in the program, it set a 20dB gain in the device. So a 0-49.6dB device can be used only with 0, 10, 20, 30, 40dB gains!

Best regards.

[xmikos]

Problem 1 is now fixed in Git master (thread_name_prefix is only supported on Python >= 3.6).

I will look into problems 2 and 3 tomorrow. But why are you doing so much cropping? 90% crop is extremely high, you are discarding most FFT bins.

Problem 4 seems strange, will look into it.

Btw. thanks for your report.

[ZeroChaos-]

I can confirm, I have this issue in soapy_power 1.5.0 but not in git master

zero@gato soapy_power % qspectrumanalyzer                                                                                                                                                          (git)-[master] 
soapy_power params:
{'bandwidth': 20000000.0,
 'bin_size': 1.0,
 'crop': 0.0,
 'device': '',
 'gain': 0,
 'hops': 0,
 'interval': 1.0,
 'ppm': 0,
 'sample_rate': 20000000.0,
 'single_shot': False,
 'start_freq': 100.0,
 'stop_freq': 300.0}

[INFO] Opening HackRF device instance {device=HackRF One, driver=hackrf, part_id=a000cb3c005f434b, serial=000000000000000015d463dc38437825, version=2017.02.1}...
INFO: Using device: HackRF One
INFO: repeats: 100
INFO: samples: 2000000 (time: 0.10000 s)
INFO: max_buffer_size (samples): 32768000 (repeats: 1638.40, time: 1.63840 s)
INFO: buffer_size (samples): 2097152 (repeats: 104.86, time: 0.10486 s)
INFO: buffer_repeats: 1
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.4/soapy_power", line 11, in <module>
    load_entry_point('soapy-power==1.5.0', 'console_scripts', 'soapy_power')()
  File "/usr/lib64/python3.4/site-packages/soapypower/__main__.py", line 348, in main
    max_threads=args.max_threads, max_queue_size=args.max_queue_size
  File "/usr/lib64/python3.4/site-packages/soapypower/power.py", line 280, in sweep
    reset_stream=reset_stream, max_threads=max_threads, max_queue_size=max_queue_size
  File "/usr/lib64/python3.4/site-packages/soapypower/power.py", line 192, in setup
    lnb_lo=lnb_lo, max_threads=max_threads, max_queue_size=max_queue_size)
  File "/usr/lib64/python3.4/site-packages/soapypower/psd.py", line 31, in __init__
    thread_name_prefix='PSD_thread'
  File "/usr/lib64/python3.4/site-packages/soapypower/threadpool.py", line 7, in __init__
    super().__init__(max_workers or os.cpu_count() or 1, thread_name_prefix)
TypeError: __init__() takes 2 positional arguments but 3 were given

[Laserdance100]

I've just rebuilt simplesoapy, qspectrumanalyzer and soapy_power from git.

Seems that the problems1 and 4 are fixed. Perhaps you should transform the gain qt-spin-box to a qt-double-spin-box 00.0, so we can use gains like 49.6dB.

[xmikos]

Is this precision in setting gain really useful? Aren't whole numbers enough?

[xmikos]

Btw. if you need really precise control of gain, you can set gain in GUI to -1 and add --specific-gains option to additional backend parameters in Settings. Then you can specify gains of individual amplification elements (e.g. LNA, VGA and AMP on HackRF). Or you can add '-g' option and specify total gain with more precision than in GUI.

[Laserdance100]

Ok. I see why you kept the gain setting integer. Since there are possibility to specify a more precise gain easily, I guess it's enough :)

[xmikos]

@Laserdance100 Nonetheless I have implemented this change ;-)

[xmikos]

@Laserdance100 I will leave that Problem 2 and Problem 3 for now, I can't reproduce it with sane settings.

If I will get more time, I may look at it again, but if you would like to look at it yourself, I am ready to merge pull request ;-)

[Laserdance100]

I looked at problem3 a bit: It seems that the code (on my computer) suffers a bit from floating arithmetic precision.

related Python packages used: numpy (1.12.1) QSpectrumAnalyzer (2.1.0) SimpleSoapy (1.4.1) SimpleSpectral (1.0.0) soapy-power (1.6.0)

How to reproduce: with a RTLSDR, set the sampling rate to 2.56MHz, then do a singleshot from 100MHz to 110.24MHz with 640kHz bin size (1s setting, 0.0dB gain, 0 ppm, 0% crop)

It will perform 4 hops of 2.56MHz (100M-102.56M; 102.56M-105.12M; 105.12M-107.68M; 107.68M-110.24M) Each hop contain 4 bins of 640kHz first hop: (100M-100.64M; 100.64M-101.28M; 101.28M-101.92M; 101.92M-102.56M) then for each hop the SoapyPowerBinFormat function of writer.py(soapypower) provide an y-axis to the parse_output function of soapy_power.py(qspectrumanalyser) this y-axis has a size of 4 (of course there are 4 bins in each hop)

so the x-axis should have a size of 4 too. it should be from my point of view (100.32M; 100.96M; 101.6M; 102.24M) and absolutely not (100M; 100.64M; 101.28M; 101.92M; 102.56M) and possibly (100M; 100.64M; 101.28M; 101.92M) depending on how you wrote the rest of your program.

you computed the x-axis like that: x_axis = np.arange(start_freq, stop_freq, step)

note that you are using float numbers. from numpy.arange docs: "When using a non-integer step, such as 0.1, the results will often not be consistent. It is better to use linspace for these cases."

np.arange(100, 102.56, 0.64) gives array([ 100. , 100.64, 101.28, 101.92, 102.56]) but np.arange(100e6, 102.56e6, 0.64e6) gives array([ 1.00000000e+08, 1.00640000e+08, 1.01280000e+08, 1.01920000e+08]) That's the well known float precision problem. Here, it leads to the array being of size 5 or 4!

another example, you compute 8/2and 8%2 with floats. you could have 4.00000000000000000000000000000026 and 0.00000000000000000000000000000037 (it's not a great example cause there is actually no problem with 8 and 2, but you should have understood what I was pointing, and what could possibly occur)

So I guess you should code something like this def parse_output(self, data): """Parse data from soapy_power""" header, y_axis = data

    time_start = header.time_start
    time_stop = header.time_stop
    start_freq = int(header.start)
    stop_freq = int(header.stop)
    step = int(header.step)
    samples = header.samples

note the integer casts!

    linspace_step_count = (stop_freq - start_freq)/step
    linspace_start = start_freq+step/2
    linspace_stop = linspace_start+(linspace_step_count*step)
    x_axis = np.linspace(linspace_start, linspace_stop, linspace_step_count)

As I don't really know if that change is breaking your code somewhere else, I think that you have to think about this problem.

About the problem4: It's quite easy. It's a problem which occurs if you have a near 100% crop setting, which is then rounded as 100% crop by the program. I think it would be great to check if the crop value of 100%, if that's the case, report and stop.

Best regards

[xmikos]

Can you please test latest QSpectrumAnalyzer Git master branch? Problem 3 should be hopefully fixed with latest commit https://github.com/xmikos/qspectrumanalyzer/commit/4571075bd910d2242ef0303b1d7405475e69a2a0.

[Laserdance100]

I've just tested it with master, the problem3 is fixed :)

About the problem4 (near 100% crop setting, which is then rounded to 100%): a 100% crop error would be more helpful than the current ZeroDivisionError: float division by zero But who would have a near 100% crop anyway? I'll close the issue. Thanks for the fix ;)