search

BaderLab / saber

Saber is a deep-learning based tool for information extraction in the biomedical domain. Pull requests are welcome! Note: this is a work in progress. Many things are broken, and the codebase is not stable.
https://baderlab.github.io/saber/
MIT License
102 stars 17 forks source link

Implement mixed precision training #160

Closed JohnGiorgi closed 5 years ago

JohnGiorgi commented 5 years ago

The BERT models from the PyTorch Transformers library we are using support mixed-precision training.

We should implement this ASAP, as we are currently having memory issues with our BERT based models. This should reduce memory usage and speed up training times with no effect on task-specific performance. The API could be like

config file

"model": {
    "args": {
        "fp16": true,
}

command line

python -m saber.cli.train --fp16