search

tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
https://docs.tenstorrent.com/ttnn/latest/index.html
Apache License 2.0
485 stars 79 forks source link

Port functionality to n300 card (single device) #13406

Open saichandax opened 1 month ago

saichandax commented 1 month ago

Enable support for BERT-Tiny model using TTNN ops and port functionality to n300 card (single device).

vigneshkeerthivasanx commented 1 month ago

Draft PR is created for the Bert-Tiny Corresponding PR: #13471 Demo results in n300 is recorded here : Link

vigneshkeerthivasanx commented 1 month ago

@mbahnasTT , Bert-Tiny model is completed and the PCC for all sub-modules > 0.99. But the PCC of the entire model has dropped after the final linear layer to ~0.96 and ~0.97 after splitting start_logits and end_logits respectively. The models accuracy for 100 samples are as follows:

Accuracy between TTNN model and SQUADv2 reference: 

Eval: exact: 14.423076923076923 
F1: 22.086671341479036

Accuracy between PyTorch model and SQUADv2 reference: 

exact: 19.23076923076923 
F1: 24.28325947556717

Accuracy between TTNN and PyTorch model: 

exact: 58.65384615384615
F1: 0.5875457875457875

Pytorch results does not align with the dataset for most of the cases. However, Pytorch and TTNN results match in most cases, and the results can be found here: Link Corresponding Draft PR : #13471 Can we proceed with this model with the above results?

mbahnasTT commented 1 month ago

@vigneshkeerthivasanx please proceed with this model, and file an issue with these details, and keep track of the issue # i the main model status sheet. Thanks!