git clone https://github.com/MedARC-AI/MindEyeV2.git
cd MindEyeV2
git clone https://huggingface.co/datasets/pscotti/mindeyev2 .
or for specifically downloading only parts of the dataset (will need to edit depending on what you want to download):
from huggingface_hub import snapshot_download, hf_hub_download
snapshot_download(repo_id="pscotti/mindeyev2", repo_type = "dataset", revision="main", allow_patterns="*.tar",
local_dir= "your_local_dir", local_dir_use_symlinks = False, resume_download = True)
hf_hub_download(repo_id="pscotti/mindeyev2", filename="coco_images_224_float16.hdf5", repo_type="dataset")
. src/setup.sh
to install a new "fmri" virtual environment. Make sure the virtual environment is activated with "source fmri/bin/activate".src/Train.ipynb
trains models (both single-subject and multi-subject). Check the argparser arguments to specify how you want to train the model (e.g., --num_sessions=1
to train with 1-hour of data).
--no-blurry_recon
), and lowering the batch size.--no-multi_subject
and --subj=#
where # is the subject from NSD you wish to train--multi_subject
and --subj=#
where # is the one subject out of 8 NSD subjects to not include in the pretraining.--no-multi_subject
and --multisubject_ckpt=path_to_your_pretrained_ckpt_folder
src/recon_inference.ipynb
will run inference on a pretrained model, outputting tensors of reconstructions/predicted captions/etc.src/final_evaluations.ipynb
will visualize reconstructions output from src/recon_inference
and compute quantitative metrics.