Saber is a deep-learning based tool for information extraction in the biomedical domain. Pull requests are welcome! Note: this is a work in progress. Many things are broken, and the codebase is not stable.
This pull request adds the missing transformer encoder layer to our joint NER + RE model ✨.
Right now, the number of attention heads and the number of encoder layers are both hardcoded at 2. In future pull requests, these will be added to the config file and we will include them in our hyperparameter search to determine the best default values.
Additional Changes
Trying to train on a CPU with Apex caused an error. This has been fixed by requiring both Apex and an available GPU before defaulting to mixed-precision training.
Overview
This pull request adds the missing transformer encoder layer to our joint NER + RE model ✨.
Right now, the number of attention heads and the number of encoder layers are both hardcoded at 2. In future pull requests, these will be added to the config file and we will include them in our hyperparameter search to determine the best default values.
Additional Changes
Trying to train on a CPU with Apex caused an error. This has been fixed by requiring both Apex and an available GPU before defaulting to mixed-precision training.
Closes
Closes #167.