search

tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
Apache License 2.0
436 stars 63 forks source link

DistilBert gs demo benchmarking #4482

Closed Sudharsan-V closed 9 months ago

Sudharsan-V commented 10 months ago

GS e2e Demo

test uses batched real inputs and produces batched outputs

this test is looped (takes several batches of inputs, produces several batches of outputs)

test evaluates results of TT model against validation data in selected data set

test is documented

how to run test (commands to execute) what does the model do (high-level explanation in a couple of sentences) description of what is input and what is output this demo should load inputs and expected outputs from a data set on Weka (TBD which one)

kkeerthana0573 commented 9 months ago

This commit contains benchmarking of Distilbert for Question and Answering using SQUAD dataset.

Corresponding commit - commit Corresponding PR - #4510

Sudharsan-V commented 9 months ago

Distilbert GS demo Benchmarking is completed and it is merged with mail. All the CI tests passed.