About the result of "EEGClip"

[xuxiran]

Thank you for your excellent work and patient answers before. I tried many times but I failed, so I came again. In the "EEGClip", I have two questions. The first is about the result. I have run the "main.py" for 2785 epochs but the result is still lower than 0.05. The second is about evaluating. I could only find "main.py" to train the model but I did not find "evaluate.py" to get the result of testing data. Perhaps I overlooked something? Thank you very much for your attention and patience!

[prajwalsingh]

Hi @xuxiran , check this issue [ link ]. If it doesn't work for you, please let us know.

For evaluation this is code we have used [ topkrecall.py ]

[xuxiran]

Thank you very much for your timely response. I have read the closed issue #2 and #7. I made sure that the dataset is "5-95Hz". However, I could not get satisfactory answer. I doubt that maybe I miss some pre-programs? Do I need to prepare any other files before running "main. py" in "EEGClip"? Just like before running "train. py" for "EEGStyleGAN-ADA-CVPR40", I need to prepare eegbestckpt and imageckpt. Or, as I understand it, "EEGClip" should be an end-to-end architecture, and I should simply run "main. py"?

In addition, I evaluated this code by running "topkrecall.py", but I still could not get a good result. After modifying some problems, I successfully ran the "topkrecall.py", but the result was still lower than I expected. When I try to use the pre-trained parameters, it seems that the model of pre-trained parameters is not the model we got in "EEGClip". Thank you very much for your attention and patience!

[prajwalsingh]

Hi @xuxiran , thank you for raising the issue. I am adding Dwip in the loop @dwipddalal . He will address the problem soon.

[xuxiran]

Thank you very much! You are the best repository owner I have seen!

[prajwalsingh]

Hi @xuxiran @zihang-xu, thank you for your valuable feedback. We re-run the code and observed there was a checkpoint mismatched as well, as the code was overwritten.

We have fixed the code along with and updated the checkpoint on the one drive. You can now download the updated code as well as checkpoint to test EEGClip.

A few things to note:

1. When we were training the clip model, we were calculating K-means, which follows the linear classification, but we observed that the feature we get using EEGClip gives a poor performance in case of linear classification (linear probing), but when fine-tuning it shows good performance (we have mentioned the same in WACV24 paper also or may be skipped from our mind to mention that, similar observation is discussed in Masked Autoencoders Are Scalable Vision Learners [link] paper.).
2. This is the reason for the updated code: while training EEGClip using the "main.py" file, we removed the linear evaluation code. We trained the EEGClip for 2048 epochs and have presented the study for the same in our paper.
3. After training EEGClip using main.py for evaluation of EEG features, run the "classification_eeg.py" code.
4. You can try EEGClip on both raw and filtered eeg data. We have shared the checkpoint for raw EEG only as filter one checkpoint is not available to me.

Let us know if you have any questions.

[xuxiran]

I have finished running main.py and are running classification.py now. It seems works. Thank you very much!

[zihang-xu]

Congratulations! You seem to be almost successful. Do you use filtered EEG data? I'm still running main.py. Can you share the clip_2048.pth file?

[prajwalsingh]

Hi @zihang-xu , we are also running the experiments in parallel. We will upload the clip_2048.pth once it is done. We made a few updates to the code and after making the final updates on Git Hub. I will let you know.

For filtered EEG data, we have to check again. We have used and reported the results in the paper.

P.S. The initial experiment code got deleted due to some confusion while cleaning the drive, so we have to re-run and check which one worked earlier 😅

[zihang-xu]

Thanks for your patience and explanation! I look forward to your updates.

Experiment results can be deleted by mistake. I've done the same thing 🤣

[xuxiran]

@zihang-xu I used the unfiltered data. I will use the filtered data to try again today. @prajwalsingh I have one more question about the "EEGClip". I want to know what "classification.py" and "classification_image.py" do. To my understand, the "classification.py" should decode the class name of EEG, the chance level is 2.5%. But what the "classification_image.py" do? Thank you very much!

[prajwalsingh]

Hi @xuxiran , that in general doesn't mean much we were just fine-tuning the image embedding of clip. The only goal of this work is to check how good EEG representations are learning. We have removed the code of image classification in latest update, only kept classification_eeg.py

For filter one we are figuring out what parameters and networks were used for paper as initial code got deleted.

Thank you.

[zihang-xu]

If you have new results, look forward to sharing and updating them. :) Thanks again！

[weipipione]

Is EEGCLIP also a zero sample recognition task?

[prajwalsingh]

Hi @weipipione , if you are asking about zero-shot classification. We have performed that experiment with a triplet-loss-based feature extraction method. We haven't explored it for EEGCLIP.

[Applicant-233]

@prajwalsingh Hello, may I ask where the original data was obtained? I did not find any relevant connections, like EEGCVPR40 Dataset

[prajwalsingh]

Hi @Applicant-233 , the data is obtained from here [link]

[Applicant-233]

Hi @Applicant-233 , the data is obtained from here [link]

Thank you! The link offers raw data, but the preprocessed dataset used in the paper is not included.

[prajwalsingh]

@Applicant-233 Both raw and preprocess data are there with name 5_95...

[Applicant-233]

@Applicant-233 Both raw and preprocess data are there with name 5_95...

Thanks!

[aohodo]

这个代码从哪里运行呢