Use nn.Dropout for BertModel - Githubissues

huggingface / candle

Minimalist ML framework for Rust

Apache License 2.0

15.88k stars 962 forks source link

Use nn.Dropout for BertModel #2600

Closed DimitriTimoz closed 2 weeks ago

DimitriTimoz commented 2 weeks ago

Implement the following todo to be able to train the bert model with real dropout layers. https://github.com/huggingface/candle/blob/e2b6b367fa852ed30ac532f8d77cd8479c7ed092/candle-transformers/src/models/bert.rs#L126-L132