search

ozcelikfu / brain-diffuser

Official repository for the paper "Brain-Diffuser: Natural scene reconstruction from fMRI signals using generative latent diffusion" by Furkan Ozcelik and Rufin VanRullen.
MIT License
111 stars 19 forks source link

Analysis of reconstruction results #14

Open spicybody opened 1 year ago

spicybody commented 1 year ago

Thank you very much for your contribution to this project. I also wish the subject better and better on the road in the future. For the reproduction of this project, I have a problem, in the first stage of the reconstruction result I reproduced successfully, but in the second stage, I followed the operation of the readme file step by step, but the final picture generation result is only two kinds of color noise and black picture, which makes me very confused, I hope that kind people can help me, thank you very much 85e9b1ffc930bb1bcdedea8e03edd7e

ozcelikfu commented 1 year ago

I think I have solved the problem. It is the version of the transformers lib. When I use v4.33.2 the extracted clipvision features are different from v4.19.2. Using the latter one I can replicate your results. Thanks a lot!

Can it be a problem similar to this one in #9 ? Please check and let me know.

spicybody commented 1 year ago

I'm sad it's not the same as problem 9, I confirm I have my transformers==v4.19.2 and it still outputs black images or colored noise, which confuses me.

HaoyyLi commented 12 months ago

Excuse me, could you please tell me the value of alpha (the author's alpha=50000) was used when fitting the regression model in the first stage? I can't achieve excellent results by reusing the author's alpha=50000 in my project.