search

tudelft / ssl_e2vid

MIT License
68 stars 11 forks source link

Unable to reproduce the metrics in the paper #2

Open MrFisher97 opened 2 years ago

MrFisher97 commented 2 years ago

Since there is no code to evaluate the effect of reconstruction in this project, we refer to the implementation of the evaluation function of WarranWeng/ET-Net. However, we find that the metrics (the model is downloaded from there and the setting complies with the value given in the project) are different from the metrics published in the paper. For example, on the HQF data-set, we test E2VID model, the MSE value is only 0.0930 and the LPIPs is 0.5390, which are quite different from the 0.0611 and 0.4740 in the paper.

So, would you provide the test code or give a more detailed test configuration?

midofalasol commented 2 years ago

Hi, @wanzengy !

Do you use a fixed number of events or take the number of events per frame interval when generating the voxel grid?

MrFisher97 commented 2 years ago

Hi, @wanzengy !

Do you use a fixed number of events or take the number of events per frame interval when generating the voxel grid?

I fix the number of events when training, while testing, I take the events between frame interval to generate the voxel grid.

ercanburak commented 1 year ago

For anyone seeking quantitative evaluation, you can utilize EVREAL, our library designed to evaluate and analyze PyTorch-based event-based video reconstruction methods (including SSL-E2VID).