Performance

[maxnoe]

I profiled the offset_calibration:

This is the result:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   10.426   10.426  133.520  133.520 offset_calculation.py:37(offset_calc)
  1600000   51.105    0.000   51.317    0.000 runningstats.py:12(add)
  1600000   10.002    0.000   37.301    0.000 numeric.py:1327(roll)
   100001    0.318    0.000   23.226    0.000 _tqdm.py:422(__iter__)
   100001    0.078    0.000   22.828    0.000 io.py:99(__next__)
   100001    0.362    0.000   22.750    0.000 io.py:109(next)
   100000    2.868    0.000   19.086    0.000 io.py:123(read_adc_data)
  1800032    6.536    0.000   13.360    0.000 {built-in method numpy.core.multiarray.empty}
  3200000   12.330    0.000   12.330    0.000 {built-in method numpy.core.multiarray.arange}
  1600032    2.637    0.000   11.178    0.000 numeric.py:247(full)
  1600000    7.893    0.000    7.893    0.000 {method 'take' of 'numpy.ndarray' objects}
   200000    1.615    0.000    6.824    0.000 _internal.py:149(_commastring)
   100000    6.634    0.000    6.634    0.000 {built-in method numpy.core.multiarray.fromfile}
  1600000    5.013    0.000    5.013    0.000 {built-in method numpy.core.multiarray.concatenate}
  1600032    4.239    0.000    4.239    0.000 {built-in method numpy.core.multiarray.copyto}
   200000    3.147    0.000    3.194    0.000 {built-in method builtins.eval}
   100001    1.574    0.000    3.162    0.000 io.py:256(read_header)
  4800032    1.197    0.000    1.197    0.000 {built-in method numpy.core.multiarray.array}
  1000000    0.682    0.000    1.145    0.000 py3k.py:28(asbytes)
  1600000    0.861    0.000    1.070    0.000 numeric.py:476(asanyarray)
  1600000    0.994    0.000    0.994    0.000 {method 'reshape' of 'numpy.ndarray' objects}
   200003    0.727    0.000    0.727    0.000 {method 'read' of '_io.BufferedReader' objects}
   200008    0.538    0.000    0.538    0.000 {method 'match' of '_sre.SRE_Pattern' objects}
  1000001    0.295    0.000    0.295    0.000 {method 'encode' of 'str' objects}
   106598    0.178    0.000    0.178    0.000 {method 'format' of 'str' objects}
  1000669    0.169    0.000    0.169    0.000 {built-in method builtins.isinstance}
   100000    0.166    0.000    0.166    0.000 {built-in method numpy.core.multiarray.frombuffer}
604947/604876    0.155    0.000    0.155    0.000 {built-in method builtins.len}
   100000    0.074    0.000    0.142    0.000 <string>:12(__new__)
   200001    0.137    0.000    0.137    0.000 {built-in method __new__ of type object at 0x7f39efcd6020}
   200000    0.106    0.000    0.106    0.000 {method 'groups' of '_sre.SRE_Match' objects}
   100001    0.105    0.000    0.105    0.000 {built-in method _struct.unpack}
        1    0.000    0.000    0.060    0.060 io.py:325(EventGenerator)
        2    0.000    0.000    0.060    0.030 io.py:49(__init__)
     1294    0.019    0.000    0.046    0.000 _tqdm.py:100(format_meter)
        1    0.000    0.000    0.040    0.040 io.py:332(guess_version)
        3    0.039    0.013    0.039    0.013 {built-in method io.open}
   200000    0.037    0.000    0.037    0.000 {method 'end' of '_sre.SRE_Match' objects}
   100001    0.036    0.000    0.036    0.000 {built-in method _struct.calcsize}
   201573    0.035    0.000    0.035    0.000 {method 'append' of 'list' objects}

[maxnoe]

The most expensive calls are:

• RunningStats.add
• numpy.roll

[dneise]

I think there is a okayish way to get around the np.roll before each stats.add.

I have shown it in that performance branch I created recently

[dneise]

1c49a3c6