search

tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
Apache License 2.0
387 stars 47 forks source link

MNIST GS demo Benchmarking #4431

Closed Sudharsan-V closed 7 months ago

Sudharsan-V commented 8 months ago

GS e2e Demo

test uses batched real inputs and produces batched outputs

this test is looped (takes several batches of inputs, produces several batches of outputs)

test evaluates results of TT model against validation data in selected data set

test is documented

how to run test (commands to execute) what does the model do (high-level explanation in a couple of sentences) description of what is input and what is output this demo should load inputs and expected outputs from a data set on Weka (TBD which one)

Sudharsan-V commented 8 months ago

The Benchmarking of GS MNIST model is completed. Data-set: MNIST Metric: Accuracy

Corresponding: commit

jayasuryamaganuru commented 8 months ago

PR for MNIST GS demo benchmarking , model is tested with 1000 datapoints extracted from the MNIST handwritten digits dataset , the accuracy is listed below:

Metric: Accuracy Accuracy: 62.2

PR - #4502

saichandax commented 8 months ago
saichandax commented 7 months ago

PR is approved. CI is pending

saichandax commented 7 months ago

PR is merged. Hence closing the ticket.