search

neuropsychology / NeuroKit

NeuroKit2: The Python Toolbox for Neurophysiological Signal Processing
https://neuropsychology.github.io/NeuroKit
MIT License
1.54k stars 410 forks source link

NeuroKit2 for PolarH10 data, references? #1005

Open Mario-0 opened 3 months ago

Mario-0 commented 3 months ago

Hello all, i'm trying to use Neurokit2 to compute features from a dataset of 53 participants that worn PolarH10 devices. Trying to classify some timestamped affect ratings. Would love to find another Github repository or research paper that used Neurokit with this specific device because I feel I'm reinventing the wheel.

Also running into an issue; I'm trying to compute features about the H10 ECG measurements within specific time intervals however i get the error below. I use 2 files, 1 is the PolarH10 output csv file with a column 'ecg [uV]'; the other file contains timestamped valence ratings and I want to compute features about time windows starting 15 seconds before the rating and ending 15 seconds after the rating. I think there's an error in selecting the right rows from the H10 file, because nk.ecg_process seems to crash because the input dataframe is empty. For some participants it's working well, for some it's not, can't figure out why there seems to be a difference in length of some dataframe. 

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[12], [line 59](vscode-notebook-cell:?execution_count=12&line=59)
     [57](vscode-notebook-cell:?execution_count=12&line=57) # Process raw ECG data into QRS metrics
     [58](vscode-notebook-cell:?execution_count=12&line=58) print(f'start nk.ecg_process for timestamp {timestamp}')
---> [59](vscode-notebook-cell:?execution_count=12&line=59) filtered_ecg_ecgonly_signals, _ = nk.ecg_process(filtered_ecg_ecgonly, sampling_rate=100, method='neurokit')
     [61](vscode-notebook-cell:?execution_count=12&line=61) # Calculate HRV features
     [62](vscode-notebook-cell:?execution_count=12&line=62) print(f'start nk.hrv for timestamp {timestamp}')

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\neurokit2\ecg\ecg_process.py:95, in ecg_process(ecg_signal, sampling_rate, method)
     [93](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_process.py:93) # Sanitize and clean input
     [94](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_process.py:94) ecg_signal = signal_sanitize(ecg_signal)
---> [95](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_process.py:95) ecg_cleaned = ecg_clean(ecg_signal, sampling_rate=sampling_rate, method=method)
     [97](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_process.py:97) # Detect R-peaks
     [98](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_process.py:98) instant_peaks, info = ecg_peaks(
     [99](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_process.py:99)     ecg_cleaned=ecg_cleaned,
    [100](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_process.py:100)     sampling_rate=sampling_rate,
    [101](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_process.py:101)     method=method,
    [102](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_process.py:102)     correct_artifacts=True,
    [103](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_process.py:103) )

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\neurokit2\ecg\ecg_clean.py:110, in ecg_clean(ecg_signal, sampling_rate, method, **kwargs)
    [108](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:108) method = method.lower()  # remove capitalised letters
    [109](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:109) if method in ["nk", "nk2", "neurokit", "neurokit2"]:
--> [110](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:110)     clean = _ecg_clean_nk(ecg_signal, sampling_rate, **kwargs)
    [111](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:111) elif method in ["biosppy", "gamboa2008"]:
    [112](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:112)     clean = _ecg_clean_biosppy(ecg_signal, sampling_rate)

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\neurokit2\ecg\ecg_clean.py:163, in _ecg_clean_nk(ecg_signal, sampling_rate, **kwargs)
    [161](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:161) def _ecg_clean_nk(ecg_signal, sampling_rate=1000, **kwargs):
    [162](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:162)     # Remove slow drift and dc offset with highpass Butterworth.
--> [163](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:163)     clean = signal_filter(
    [164](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:164)         signal=ecg_signal,
    [165](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:165)         sampling_rate=sampling_rate,
    [166](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:166)         lowcut=0.5,
    [167](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:167)         method="butterworth",
    [168](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:168)         order=5,
    [169](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:169)     )
    [171](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:171)     clean = signal_filter(
    [172](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:172)         signal=clean, sampling_rate=sampling_rate, method="powerline", **kwargs
    [173](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:173)     )
    [174](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/ecg/ecg_clean.py:174)     return clean

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\neurokit2\signal\signal_filter.py:160, in signal_filter(signal, sampling_rate, lowcut, highcut, method, order, window_size, powerline, show)
    [157](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/signal/signal_filter.py:157)     raise ValueError("NeuroKit error: signal_filter(): you need to specify a 'lowcut' or a 'highcut'.")
    [159](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/signal/signal_filter.py:159) if method in ["butter", "butterworth"]:
--> [160](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/signal/signal_filter.py:160)     filtered = _signal_filter_butterworth(signal_sanitized, sampling_rate, lowcut, highcut, order)
    [161](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/signal/signal_filter.py:161) elif method in ["butter_ba", "butterworth_ba"]:
    [162](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/signal/signal_filter.py:162)     filtered = _signal_filter_butterworth_ba(signal_sanitized, sampling_rate, lowcut, highcut, order)

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\neurokit2\signal\signal_filter.py:249, in _signal_filter_butterworth(signal, sampling_rate, lowcut, highcut, order)
    [247](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/signal/signal_filter.py:247) freqs, filter_type = _signal_filter_sanitize(lowcut=lowcut, highcut=highcut, sampling_rate=sampling_rate)
    [248](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/signal/signal_filter.py:248) sos = scipy.signal.butter(order, freqs, btype=filter_type, output="sos", fs=sampling_rate)
--> [249](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/signal/signal_filter.py:249) filtered = scipy.signal.sosfiltfilt(sos, signal)
    [250](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/neurokit2/signal/signal_filter.py:250) return filtered

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\scipy\signal\_signaltools.py:4451, in sosfiltfilt(sos, x, axis, padtype, padlen)
   [4449](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/scipy/signal/_signaltools.py:4449) ntaps = 2 * n_sections + 1
   [4450](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/scipy/signal/_signaltools.py:4450) ntaps -= min((sos[:, 2] == 0).sum(), (sos[:, 5] == 0).sum())
-> [4451](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/scipy/signal/_signaltools.py:4451) edge, ext = _validate_pad(padtype, padlen, x, axis,
   [4452](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/scipy/signal/_signaltools.py:4452)                           ntaps=ntaps)
   [4454](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/scipy/signal/_signaltools.py:4454) # These steps follow the same form as filtfilt with modifications
   [4455](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/scipy/signal/_signaltools.py:4455) zi = sosfilt_zi(sos)  # shape (n_sections, 2) --> (n_sections, ..., 2, ...)

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\scipy\signal\_signaltools.py:4220, in _validate_pad(padtype, padlen, x, axis, ntaps)
   [4218](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/scipy/signal/_signaltools.py:4218) # x's 'axis' dimension must be bigger than edge.
   [4219](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/scipy/signal/_signaltools.py:4219) if x.shape[axis] <= edge:
-> [4220](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/scipy/signal/_signaltools.py:4220)     raise ValueError("The length of the input vector x must be greater "
   [4221](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/scipy/signal/_signaltools.py:4221)                      "than padlen, which is %d." % edge)
   [4223](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/scipy/signal/_signaltools.py:4223) if padtype is not None and edge > 0:
   [4224](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/scipy/signal/_signaltools.py:4224)     # Make an extension of length `edge` at each
   [4225](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/scipy/signal/_signaltools.py:4225)     # end of the input array.
   [4226](https://file+.vscode-resource.vscode-cdn.net/c%3A/<private>/all/scripts/~/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0/LocalCache/local-packages/Python311/site-packages/scipy/signal/_signaltools.py:4226)     if padtype == 'even':

**ValueError: The length of the input vector x must be greater than padlen, which is 18.**
welcome[bot] commented 3 months ago

Hi 👋 Thanks for reaching out and opening your first issue here! We'll try to come back to you as soon as possible. ❤️ kenobi

DerAndereJohannes commented 2 months ago

the ecg_process function only supports a list / np.array / pd.Series as input for the ecg_signal. You have to specify this in the function input. i.e., nk.ecg_process(filtered_ecg_ecgonly["<your ecg column here>"], sampling_rate=100, method='neurokit')