os-climate / aicoe-osc-demo

This repository is the central location for the demos the ET data science team is developing within the OS-Climate project. This demo shows how to use the tools provided by Open Data Hub (ODH) running on the Operate First cluster to perform ETL, create training and inference pipelines.
Apache License 2.0
10 stars 24 forks source link

Explore onnx runtime for inference #239

Closed Shreyanand closed 1 year ago

Shreyanand commented 1 year ago

Onnx runtime is a format used by neural magic for compressed memory sized models. We want to explore that and the quantization features that it comes with. Add a notebook that uses this runtime with the relevance text classification task.