Creating embeddings

[SamGijsen]

Hello, thanks so much for making your code available!

I'd like to embed EEG data using the model and wanted to check whether the following makes sense to you. Given a pretrained model, e.g. your provided base model, I'd avoid adding a classification head to NeuralTransformer and catch the output of self.forward_features(...). Given input signals of [batch_size, channels, samples] and default repo parameters, the model would then provide embeddings [batch_size, embed_dim].

Does that seem OK or would you recommend another approach?

Thanks for your help!

[935963004]

Thanks for your interest. Given input data of [batch_size, channels, samples], e.g., [128, 64, 2000], they will be transformed into [batch_size, num_patches, patch_length] in the code, in this example, [128, 640, 200], where we define the patch length as 1 second with sampling rate 200 Hz. Finally, the shape of the output embeddings will be [batch_size, num_patches, embed_dim] as well. In the base model, embed_dim is 200. You can use these embeddings as you need.

[SamGijsen]

Right, but if one aims to obtain crop/epoch-level embeddings then one option would be to average across the patch-dimension (as implemented in self.fc_norm(t.mean(1) as far as I can see), correct? Are these averaged representations also what you used as input for your classification head during finetuning in your paper, which seemed to work well?

A bit of an unrelated question, but was a seed set for the train:validation split for TUAB? Or some other record of which subjects ultimately constituted your training set?

Appreciate your help!

[935963004]

Correct, we use the average embeddings for classification.

For the second question, the train:val split set was randomly splitted and unfortunately we have no record of which subjects included in the training set. I think this is not very important because the test set is the same no matter what training set is.