FingerFlex: Inferring Finger Trajectories from ECoG signals

Vladislav Lomtev · Alexander Kovalev · Alex Timchenko

Paper

We propose FingerFlex, a new state of the art model for prediction finger movements from brain activity (ECoG).

Abstract

Motor brain-computer interface (BCI) development relies critically on neural time series decoding algorithms. Recent advances in deep learning architectures allow for automatic feature selection to approximate higher-order dependencies in data. This article presents the FingerFlex model - a convolutional encoder-decoder architecture adapted for finger movement regression on electrocorticographic (ECoG) brain data. State-of-the-art performance was achieved on a publicly available BCI competition IV dataset 4 with a correlation coefficient between true and predicted trajectories up to 0.74. The presented method provides the opportunity for developing fully-functional high-precision cortical motor brain-computer interfaces.

Model

Figure 1. Model architecture. The proposed model is designed in the same way as the convolutional autoencoder. It allows to capture local and global features from brain signals. Our model is fully convolutional so we can use different time window during real time prediction and tune this parameter for reducing temporal delay.

Results

We test our FingerFlex on multiple datasets BCI Competition IV and Stanford which covers various subjects and different ECoG positions.

Video demonstration on BCI Competition IV dataset.

Graphics on the same dataset.

Figure 2. An example of true and decoded finger trajectories time series on BCI Competition IV dataset.

Metrics

Figure 3. Performance comparison on BCI Competition IV dataset: finger trajectory decoding task.

How to check

Citation

@article{fingerflex2022,
  title={FingerFlex: Inferring Finger Trajectories from ECoG signals},
  author={Lomtev, Vladislav and Kovalev, Alexander and Timchenko, Alexey},
  journal={arXiv preprint arXiv:2211.01960},
  year={2022}
}