Closed mouadriyad closed 5 years ago
So the first thing is to try EEGNet-8,2, with kernLength = 32. This should get you around 66-68% overall accuracy with 4-fold blockwise cross-validation as reported in the paper for within-subject classification. The performance of cross-subject classification should be about 40% overall accuracy. I also used braindecode's preprocessing for BCI IV 2A so the code above should be OK (although I haven't looked at it closely).
So the first thing is to try EEGNet-8,2, with kernLength = 32. This should get you around 66-68% overall accuracy with 4-fold blockwise cross-validation as reported in the paper for within-subject classification. The performance of cross-subject classification should be about 40% overall accuracy. I also used braindecode's preprocessing for BCI IV 2A so the code above should be OK (although I haven't looked at it closely).
I didn't understand the way you used in within-subject, did you mix the training set with the testing set (288x2) then split it into 4 blocks (288x2/4)?
I did 4-fold CV with the BCI IV 2A competition test set always being the test set. So take the training set, divide it into three equal contiguous partitions, select 2 of those 3 to be the training, 1 to be the validation, then set the test set to be the competition test set. Repeat this for all combinations of training/validation.
Here's a Dropbox link with the data I used for the paper, as well as some quick code to run the EEGNet model. The data was pre-processed and filtered according to the code in the braindecode repository, and I've already epoched the data at [0.5, 2.5]s. I've also included one run of the within-subject results. Note that due to the small sample size of BCI Competition IV you should expect variation from run-to-run.
https://www.dropbox.com/s/cxwyigcx4j6h8t5/EEGNet_BCI_IV.zip?dl=0
Hope this helps.
Hi,
Thanks for your help, I use your data now and I merged a part of your code with mine for still for cross-subject. But, I got an average of 53% for almost the third of the 90 fold. For the within-subject, It works well and I got a similar result.
Could it come from the Tensorflow version? (I use Google Colab so the current version is 1.12.0 and I couldn't install the 1.9.0 due to library)
EDIT : I run a simulation with 30 fold (3 per subject to check) and from the first 16 fold (I use colab), it seems to me that the version 1.9.0 give low accuracy than the 1.12.0. Many trial doesn't learn (25% accuracy)
I recheck and if it's ok I'll close the issue.
Here's a Dropbox link with the data I used for the paper, as well as some quick code to run the EEGNet model. The data was pre-processed and filtered according to the code in the braindecode repository, and I've already epoched the data at [0.5, 2.5]s. I've also included one run of the within-subject results. Note that due to the small sample size of BCI Competition IV you should expect variation from run-to-run.
https://www.dropbox.com/s/cxwyigcx4j6h8t5/EEGNet_BCI_IV.zip?dl=0
Hope this helps.
Hi, Which batch size did you use on cross-subject? It's the last parameter I need to test. I expect a high than 64
I used batchsize 64 for all experiments, both within- and cross- subject.
I found a bug with my tensorflow configuration.Your result are correct.
Thank you for your attention and sorry for the inconvenience.
Can I use your data for my further work?
Glad to help. Feel free to use the data I sent you, with the appropriate citation for the data source:
Tangermann, Michael, et al. "Review of the BCI competition IV." Frontiers in neuroscience 6 (2012): 55.
@vlawhern can the above code be added as part of demo code/demo data as per issue #6 Also, the link https://www.dropbox.com/s/cxwyigcx4j6h8t5/EEGNet_BCI_IV.zip?dl=0 isn't working
My main concern with hosting that particular dataset was that I don't own it (property of the Berlin BCI group). I only temporarily put up my code to analyze that dataset to help another user (issue #6 as you've referenced).
I've instead written a sample script using MNE's auto-download tools to automatically download another dataset (in this case, a 4-class ERP EEG classification dataset) to illustrate how to use and train the EEGNet model. Hopefully this helps.
Hi,
I tried to recreate the result for EEGNET-4-1 to use it as a baseline in cross-subject case. The problem is that I couldn't get the same result where simulation gets higher Accuracy around 50 and 60 (in the paper 40).
I reused completely the code from braindecode as data preprocessing code and I add the resampling function as in the paper and It not working.
If someone can check the code below and look for something I missed it would be great.
The Preprocessing part: