search

yliess86 / BayeFormers

General API for Deep Bayesian Variational Inference by Backpropagation. The repository has been designed to work with Transformers like architectures. Compatible with the HuggingFace Transformers models.
MIT License
41 stars 3 forks source link

Converting CrabNet using BayesFormers #1

Open sgbaird opened 2 years ago

sgbaird commented 2 years ago

Hi @yliess86 @WIN32GG,

My repository of interest is (my refactor of) CrabNet, which uses transformers to predict materials properties based on chemical formula (i.e. composition). I'm interested in modifying CrabNet to work with Ax, a Bayesian adaptive design tool, but this requires being able to sample from the posterior as mentioned in https://github.com/facebook/Ax/issues/748.

I take it the first step would probably be to convert CrabNet to a HuggingFace Transformer model? I appreciate any suggestions or help you can offer.

sgbaird commented 2 years ago

bump

sgbaird commented 2 years ago

@yliess86 @WIN32GG happy to provide more context as well.

sgbaird commented 2 years ago

For example, the CrabNet class subclasses nn.Module. Is this typically sufficient? Or does it need to follow a HuggingFace API?