search

ChiShengChen / MUSE_EEG

The official implement of Mind's eye: image recognition by EEG via multimodal similarity-keeping contrastive learning.
15 stars 0 forks source link

Question: about SK Loss #1

Closed gyuseok0917 closed 2 months ago

gyuseok0917 commented 2 months ago

Hello. I have a question about SK Loss. Looking at the algorithm for the loss in the paper, it calculates the cosine similarity matrix between EEG features and the cosine similarity matrix between image features, calculates the cosine similarity between the two matrices, and then subtracts the average value from 1, which is defined as SK Loss. However, in the code, it measures the similarity of the flattened EEG data, not the EEG features, and calculates the similarity of each EEG and image as an inner product, and then uses F.cosine_similarity to calculate the similarity between the two modalities. I wonder why it was done this way.

ChiShengChen commented 2 months ago

Hello, thank you for identifying this issue. Please consider the code as the primary reference for now. The approach of calculating the distance between EEG data is based on the methods outlined in several papers, where the cosine similarity is computed directly on the EEG data rather than on the features. As for the distance calculation using EEG features mentioned in the original paper, I will conduct the relevant experiments and provide an update within the next week. Once again, thank you for reviewing and using our work!