mamba - Githubissues

anhhuyalex / mind-reader

[NeurIPS2022] Mind Reader: Reconstructing complex images from brain activities

0 stars 0 forks source link

mamba #5

Open anhhuyalex opened 3 months ago

anhhuyalex commented 3 months ago

downstream evals try scaling, e.g. reconstruct full volume

anhhuyalex commented 3 months ago

to do: try varying the amount of input patches fed (from 25% of full to close to 100%) because we want the encoder to be robust to the number of patches fed as input

anhhuyalex commented 3 months ago

/weka/home-alexnguyen/mamba_fmri/ckpts/mar24_encoder32_decoder1_mamba64_lr3e-4/last.pth failed

anhhuyalex commented 2 months ago

encoder->decoder (1M input embeddings are too much)
unfreeze mae model during downstream

anhhuyalex commented 2 months ago

working runs: apr22_encoder32_decoder32_equalseqlen_learnableposemb_encoder_outdim_512_lr4e-4

random tube mask: apr19_encoder32_decoder1_randomtubemask_fixdecodemask_learnableposemb_encoder_outdim_512_lr4e-6