This repository is the central location for the demos the ET data science team is developing within the OS-Climate project. This demo shows how to use the tools provided by Open Data Hub (ODH) running on the Operate First cluster to perform ETL, create training and inference pipelines.
Apache License 2.0
11
stars
25
forks
source link
Use sparsified neural models for relevance task #222
The existing transformer model use distilbert but we want to see if there are other models in the sparse zoo that we can train for a better performance. Use the existing notebooks (transformer_relevance, transformer_inference) as a reference and create new benchmarks for this task.
Acceptance
[ ] Replicate the training and inference snippets used for the distilbert model to the sparse model
[ ] Get the benchmarks for the sparse model (note: add model size comparison to benchmarks)
[ ] Explore other models in the sparse zoo and select a few to add to the benchmarks
The existing transformer model use distilbert but we want to see if there are other models in the sparse zoo that we can train for a better performance. Use the existing notebooks (transformer_relevance, transformer_inference) as a reference and create new benchmarks for this task.
Acceptance