search

yilundu / irem_code_release

ICML 2022: Learning Iterative Reasoning through Energy Minimization
https://energy-based-model.github.io/iterative-reasoning-as-energy-minimization/
MIT License
42 stars 6 forks source link

Graph_train.py --mem flag #4

Open mcleish7 opened 1 year ago

mcleish7 commented 1 year ago

Good Afternoon,

The --mem flag in graph_train.py, does not appear to be used. Only being called: 1) When declaring iterativeFC 2) When declaring GraphEBM

However, it is not used in either constructors: IterativeFC nor EBM.

My testing is showing a large difference in loss when the --mem flag is turned on. What does the flag do please.

Thanks, Sean

yilundu commented 1 year ago

Hi,

In the model, the --mem flag should add a computational scratchpad to models, so that that not only is an output optimization, but also an intermediate latent "scratchpad".

Best, Yilun

mcleish7 commented 1 year ago

Thank you for the quick response. Just to clarify, the scratchpad is not currently implemented for the graph models, so the --mem flag should have no impact currently?

Thanks, Sean

yilundu commented 1 year ago

I believe so -- it should only do something on the MLP models

mcleish7 commented 1 year ago

Thank you so much for the help.

I have another question, if that is okay. The edge weights for the shortest path data set are drawn, independently, from a uniform distribution and then added to their own transpose as the graph is undirected so the matrix should be symmetric. However, in 'Appendix C-Graphical Algorithmic Computation' of the paper for the models, it states edge weights are drawn from a uniform distribution on [0,1], not the the addition of two. Could you confirm which one was used please.

Thanks, Sean