MedARC-AI / fMRI-reconstruction-NSD

fMRI-to-image reconstruction on the NSD dataset.
MIT License
294 stars 39 forks source link

Mapping to new embedding spaces or using ensembles #16

Open PaulScotti opened 1 year ago

PaulScotti commented 1 year ago

MindEye can map to any multimodal latent space like CLIP, OpenCLIP, AlexNet, ImageBind, etc. Can you improve our retrieval and/or reconstruction evaluation metrics by either (1) mapping to a different embedding space or (2) combining multiple models mapped to different embedding spaces to improve final metrics?