Insert new scaling algorithm

[skjerns]

as suggested in #29

[raphaelvallat]

Thanks @skjerns for the PR and @EtienneCmb for your input! I think a final version could be:

from scipy.stats import iqr

# Assume that the inter-quartile amplitude of EEG data is ~50 uV
iqr_chan = iqr(data[:, :int(data.shape[1] / 4)], axis=-1)
bad_iqr = iqr_chan < 1.

if np.any(bad_iqr):
    mult_fact = np.zeros_like(iqr_chan)
    mult_fact[bad_iqr] = np.floor(np.log10(50 / iqr_chan[bad_iqr]))
    data *= 10 ** mult_fact[..., np.newaxis]

Advantages:

• no for loop
• we only compute the IQR on 1/4 of the data (should be enough to have a good idea of what the real IQR is, and speed up the computation by a factor of four)

I did some tests on my computer and for ~8 hours of data, sampled at 100 Hz and with 20 channels, the computation of the IQR takes ~300 ms (and 1.3 second for the full data).

Do you see any way that we could further speed up / improve the code?

[EtienneCmb]

I did some tests on my computer and for ~8 hours of data, sampled at 100 Hz and with 20 channels, the computation of the IQR takes ~300 ms (and 1.3 second for the full data).

Do you see any way that we could further speed up / improve the code?

The 300ms comes from the iqr() function?

[raphaelvallat]

Yes. By the way, if we want to remove the call to scipy, the iqr function is essentially doing: np.diff(np.percentile(x, [25, 75], axis=1), axis=0)[0]

The numpy version being just a bit faster (but this is almost negligible).

[EtienneCmb]

The numpy version being just a bit faster (but this is almost negligible).

Probably because there's checks inside the iqr function. If it's only a question os ms, I propose to keep the iqr version.

[EtienneCmb]

@raphaelvallat can you please take a moment to review this PR, we could merge it today because tests passed

[codecov[bot]]

Codecov Report

Merging #33 into master will increase coverage by 0.01%. The diff coverage is 88.88%.

@@            Coverage Diff             @@
##           master      #33      +/-   ##
==========================================
+ Coverage    86.2%   86.22%   +0.01%     
==========================================
  Files         132      132              
  Lines       19207    19213       +6     
  Branches     1463     1463              
==========================================
+ Hits        16558    16566       +8     
+ Misses       2085     2083       -2     
  Partials      564      564

Impacted Files	Coverage Δ
...isbrain/gui/sleep/interface/ui_elements/ui_menu.py	81.81% <ø> (ø)	:arrow_up:
visbrain/io/read_sleep.py	31.46% <88.88%> (+1.94%)	:arrow_up:

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 6311e8e...0c98f6e. Read the comment docs.