search

MariusKlug / zapline-plus

Improvements of the ZapLine function to remove line noise from EEG/MEG data. Adds automatic detection of the number of components to remove, and chunks the data into segments to account for nonstationarities.
GNU General Public License v3.0
25 stars 9 forks source link

Flat channels result in error #3

Closed behinger closed 2 years ago

behinger commented 2 years ago

Hi Marius! If there is a flat channel (or flat chunk), the function errors because the power spectrum is -Inf (understandably so ;)).

You could add an assert to stop this from happening. For me, the error appeared in L421, the "searchIndividualNoise" part. I stumbled over it, now I have to remove flat channels, then run zapline, then remove bad channels again sigh.

It would be interesting, btw. to get your opinion, when exactly zapline+ should be run. Before cleaning data? After removing big spikes etc.? After channel cleaning?

Cheers, Bene

MariusKlug commented 2 years ago

thanks! i'll check this and add a workaround. indeed, I run this function at the very beginning in my processing, before bad channel removal, and before average referencing, just after resampling. sidenote: i don't remove samples of big spikes nowadays, only breaks and pre/post-experiment parts, because AMICA has a great working auto clean function which in our experience works better than time-domain cleaning (paper incoming ;) )

MariusKlug commented 2 years ago

could you make the dataset with which you had the error available to me? that way i can easily test it and make sure it works.

it is a tricky thing though, even in principle. is this a chunk that is flat on all channels? i just tried setting one single channel flat and in that case it would not work either, but not crash, instead it just didn't clean at all because the spectrum was unusable (righly so). since i want to use zapline on the raw data i would have to find a way to ignore flat channels but still use zapline on the rest...

i'll have to think about this, thanks for pointing me to the issue! do you have a straightforward idea on how to do that conceptually? what about removing them temporarily, then do the cleaning, then add them back in in the end?

behinger commented 2 years ago

dataset is not so easy to share, but imho it was just a completly flat channel.

that case it would not work either, but not crash, I don't understand, it doesnt work => but not a crash? For me it ran into that error

I solved it by running the flat-channel detection of clean_rawdata first, then zapline, then channel removal - probably I could run the clean_rawdata for bad channels first with all criterions :)

MariusKlug commented 2 years ago

Ok a single flat channel cause a crash for you? Interesting. For me it just gave no noise peaks in the search and then crashed in the plot, but it didn't actually go into the cleaning.

I would prefer to have a way to make zap+ compatible with flat channels in the first place, I don't want to have flat channel detection as a mandatory step before... I'll see if it is possible to just ignore it and add it back in later after the cleaning.

I think zap+ can recover a few broken channels if they arent that broken, so that's why I would like to run it first, and only then do average referencing and bad channel detection.

MariusKlug commented 2 years ago

pushed a new update :) zapline-plus now ignores flat channels (removes them before processing and adds them back in after processing) both of the whole data set and also of single chunks (if channel gets flat after a while).

i tested on a dataset where i set several entire channels to zero and also only the second half of a few more channels. i think this should work for you, let me know!