search

NeuroTechX / moabb

Mother of All BCI Benchmarks
https://neurotechx.github.io/moabb/
BSD 3-Clause "New" or "Revised" License
693 stars 177 forks source link

From Binary classification performance to BCI Application level #281

Open jsosulski opened 2 years ago

jsosulski commented 2 years ago

OT: Sorry for the flood of issues, I am just cleaning out my personal todo / issue log to the official one on Github :)

Currently moabb only allows the evaluation of binary classification performance, which is an important consideration for practical BCI usage, however there are application level metrics that could be interesting as well. E.g. for an ERP/P300 speller, you are mostly interested in correctly spelled letters.

However, enabling this requires more annotations of the epochs / ground truths etc. I have the necessary information for the Huebner datasets, and potentially Lee_ERP, but implementing this would be a larger task imo, so this issue is mostly a note for future work... :)

P.S.: Having sub-class information available (i.e. which letters were flashed for each epoch) could enable unsupervised learning / evaluations.

sylvchev commented 2 years ago

This is related with the idea of @qbarthelemy and #186 There are some dataset (BNCI 2014 008 and 009) that also include the character information. At least I could retrieved it. This is a very good idea, but I have no clear opinion on how to do it.