sccn / eeglab

EEGLAB is an open source signal processing environment for electrophysiological signals running on Matlab and developed at the SCCN/UCSD
https://eeglab.ucsd.edu/
Other
604 stars 244 forks source link

pop_clean_rawdata - sample size error during vis_artifacts #593

Closed amisepa closed 1 year ago

amisepa commented 1 year ago

CASE 1

1) Load samplefileBDF.set 2) launch pop_cleanrawdata with basic settings just to remove some bad segments, and vis_artifacts to compare

cleanEEG = pop_clean_rawdata(EEG, 'FlatlineCriterion','off','ChannelCriterion','off',...
    'LineNoiseCriterion','off','Highpass',[0.25 0.75] ,'BurstCriterion',20,...
    'WindowCriterion',0.25,'BurstRejection','on','Distance','Euclidian',...
    'WindowCriterionTolerances',[-Inf 7]);
vis_artifacts(cleanEEG, EEG);

The following error appears:

Unable to perform assignment because the size of the left side is 64-by-148769 and the size of the right side is 64-by-148767.
Error in vis_artifacts/expand_rejections (line 346)
        tmpdata(EEG.etc.clean_channel_mask,EEG.etc.clean_sample_mask) = EEG.data;
Error in vis_artifacts (line 103)
    new = expand_rejections(to_continuous(new));

3) Cut file in half (from 10 min to 5 min) EEG = pop_select(EEG,'point',[1 78000]);

4) clean_rawdata and vis_artifacts as before, but now it works

--> This suggests it is a memory issue or something related to file size? However, we've processed bigger files before. And here is another example with a 2-min Muse file (37000 samples, so smaller than the 5-min file above that has 78000 samples).

CASE 2

1) Load sampleEDF.set 2) Run pop_clean_rawdata and vis_artifacts with including highpass filter, remove bad channels with only channel criterion, and clean data.

cleanEEG = pop_clean_rawdata(EEG, 'FlatlineCriterion','off','ChannelCriterion',0.8,...
    'LineNoiseCriterion','off','Highpass',[0.25 0.75] ,'BurstCriterion',20,...
    'WindowCriterion',0.25,'BurstRejection','on','Distance','Euclidian',...
    'WindowCriterionTolerances',[-Inf 7]);
vis_artifacts(cleanEEG, EEG);

--> it works after removing 2 channels

3) Turn channelCriterion to 'off' and run again --> the error appears

Now I did the same ting to the first sample file above, and it replicates. So it wasn't about cutting in half, but maybe it removed enough of the bad channels to make it work. So it crashes when there are some bad channels that remain in the data. I tried removing first channel with 'off', and it didn't work as before, so it's not about removing any channel, but specific to some bad channels still in data.

I cannot find why some channels with bad data would cause this issue for vis_artifacts. Somehow it causes a length issue in the EEG.etc.clean_sample_mask in vis_artifacts/expand_rejections (line 346)

CASE 3

1) load the same EDF file as in CASE 2 2) downsample to 128 Hz EEG = pop_resample(EEG, 128); 3) launch pop_clean_rawdata with line that worked above (after removing channels)

cleanEEG = pop_clean_rawdata(EEG, 'FlatlineCriterion','off','ChannelCriterion',.8,...
    'LineNoiseCriterion','off','Highpass',[0.25 0.75] ,'BurstCriterion',20,...
    'WindowCriterion',0.25,'BurstRejection','on','Distance','Euclidian',...
    'WindowCriterionTolerances',[-Inf 7]);

--> same error comes back again Note that in CASE 2, both AF7 and AF8 were removed, whereas in CASE 3 after downsampling, only AF8 was removed. Somehow, this leads to the error.

Cedric

arnodelorme commented 1 year ago

Good and clear report. Fixed both issues in https://github.com/sccn/clean_rawdata/commit/6ae4f729682360300c1ff7b591f7b2d2cffec256