os-climate / aicoe-osc-demo

This repository is the central location for the demos the ET data science team is developing within the OS-Climate project. This demo shows how to use the tools provided by Open Data Hub (ODH) running on the Operate First cluster to perform ETL, create training and inference pipelines.
Apache License 2.0
10 stars 24 forks source link

[EPIC] Sparsification for KPI extraction question answering task #231

Open Shreyanand opened 1 year ago

Shreyanand commented 1 year ago

The current KPI extraction question answering model is huge (1.7gb) and it takes around a total of 7mins to infer for a pdf. We want to find smaller version of the models that get similar performance but using a smaller and faster model.

Overall, we want to investigate model pruning and test the effects of tools such as NeuralMagic to measure the performance impact of different levels of pruning. This EPIC is the first step for this overall goal.