The idea behind this is that, generally speaking, retrieval is a lot easier than reconstruction. And MindEye's retrieval results are astounding (in my opinion)! Whereas MindEye's reconstructions are only a little bit better than Brain-Diffuser.
To get working results using retrieval-augmented diffusion, see the following repos:
You don't need to outperform the current MindEye's reconstructions, just implement retrieval-augmented diffusion and get results that look coherent to test the possibility of using this approach.
The idea behind this is that, generally speaking, retrieval is a lot easier than reconstruction. And MindEye's retrieval results are astounding (in my opinion)! Whereas MindEye's reconstructions are only a little bit better than Brain-Diffuser.
To get working results using retrieval-augmented diffusion, see the following repos:
https://github.com/CompVis/retrieval-augmented-diffusion-models
https://github.com/afiaka87/retrieval-augmented-diffusion
You don't need to outperform the current MindEye's reconstructions, just implement retrieval-augmented diffusion and get results that look coherent to test the possibility of using this approach.