search

opensearch-project / opensearch-py-ml

Apache License 2.0
32 stars 62 forks source link

[FEATURE] Trace Question Answering models to TorchScript and Onnx format #304

Open dhrubo-os opened 11 months ago

dhrubo-os commented 11 months ago

We are planning to add more model supports in ML-Commons. https://github.com/opensearch-project/ml-commons/issues/1164

The target of this issue is to trace 3 popular pre-trained Question answering models to TorchScript and Onnx format. In this repo we traced pre-trained sentence embedding models into torchscript and onnx

We need to build the similar method to trace summarization models. Primarily we can target these models:

  1. distilbert-base-cased-distilled-squad
  2. distilbert-base-uncased-distilled-squad
  3. bert-large-uncased-whole-word-masking-finetuned-squad

I created a feature branch : feature/summarization_model_conversation/. All the development of this issue should be done in that branch.

faradawn commented 11 months ago

[2023-10-11] Hi, I would like to tackle this issue.

I plan to create a similiar script called distilbert model besides Sentence Transformer:

opensearch-py-ml/opensearch_py_ml/ml_models/sentencetransformermodel.py
opensearch-py-ml/opensearch_py_ml/ml_models/distilbertmodel.py

Specifically, I plan to

  1. Download a Distlled Bert model.
  2. Create a save_to_pt and to_onnx function utilizing torch.jit.trace.
  3. In the future, create a train function if needed.

Should I push directly to feature/summarization_model_conversation/ branch? Since the assignee to issue #303 is also pushing to this branch.

If there is anything I can do, please let me know.

[2023-10-12] I would like to select the "cased" model.

Among the three models, only the "cased" model answered my questions correctly.

Test 1

Context: I like hot drinks. Tea is hot. Coke is cold.
Question: Which one will I pick?

distilbert-base-cased-distilled-squad -> Tea.
distilbert-base-uncased-distilled-squad -> Coke.
bert-large-uncased-whole-word-masking-finetuned-squad -> Coke.

Test 2

Context: I live in Beijing, China. People in China speak Chinese. People in U.S. speak English. 
Question: What language do I speak?

distilbert-base-cased-distilled-squad -> Chinese.
distilbert-base-uncased-distilled-squad -> english.
bert-large-uncased-whole-word-masking-finetuned-squad -> english.
dhrubo-os commented 11 months ago

Sure assigning to you.

dhrubo-os commented 11 months ago

May be we can create a question answering model class and work in there? distilbertmodel is one type of model, so I don't think we should create separate classes for distilbertmodel

dhrubo-os commented 11 months ago

Yeah, you can raise the PR in feature/summarization_model_conversation/ branch too.

faradawn commented 11 months ago

Hi @dhrubo-os,

Got it -- make sense! Will 1) create a question_answering.py model, and 2) raise PR in that branch!

Thanks, Faradawn