anthony-wang / CrabNet

Predict materials properties using only the composition information!
https://doi.org/10.1038/s41524-021-00545-1
MIT License
92 stars 28 forks source link

attention-heads as samples from posterior distribution in a Bayesian sense #22

Open sgbaird opened 2 years ago

sgbaird commented 2 years ago

https://aclanthology.org/2020.emnlp-main.17.pdf

Though I think CrabNet might need to be refitted for new samples (i.e. if you specify N=10, then you only get 10 samples from the posterior, to get more would probably require refitting, and not sure if these would be directly comparable to the 10 from the first run). Also not exactly sure how this could be converted to individual predictions. Maybe just some basic plumbing in and after:

https://github.com/anthony-wang/CrabNet/blob/9e0d79c5bff56ceae0600015942c54214d78152f/crabnet/kingcrab.py#L151-L157