Porting RoBERTa model to n300 - Githubissues

tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.

Apache License 2.0

471 stars 73 forks source link

Porting RoBERTa model to n300 #14424

Open Sudharsan-V opened 2 weeks ago

Sudharsan-V commented 2 weeks ago

[ ] Measure and record current performance.
[ ] Rebase the model to main, ensure the PCC = 0.99
[ ] Port functionality to n300 card (single device)
[ ] Provide Op Report
[ ] Check Model into CI
[ ] Implement Demo
[ ] https://github.com/tenstorrent/tt-metal/issues/14423
[ ] #14729

Executive summary (as of Nov 14):

Single Device implementation is complete: (BLOCKED) due to pending approvals and CIs for #8896
- PCC = 0.94
- BS = 8
- Perf numbers
  - E2E perf is 117.168 samples/sec
  - Device perf is 121.932 samples/sec
Data parallel implementation is complete: (BLOCKED) due to pending approvals and CIs for #15042
- PCC = 0.94
- BS = 16
- Perf numbers
  - E2E perf is 229.79 samples/sec
  - Device perf is 193.188 samples/sec
Trace_2cqs implementation (In progress)
- Draft PR - #14839

ToDo:

Try higher bs, record perf numbers and merge once we have PR approvals and passing CIs.
Perf with trace_2cqs (in progress)
- Draft PR: #14839

kkeerthana0573 commented 1 week ago

@mbahnasTT,

We're calculating the Sen/Sec metric for the RoBERTa model which is brought up using ttnn_optimized_bert. I couldn’t reproduce the mentioned Sen/Sec for BERT-Large using test_perf_bert11.py. Could you provide details on how Sen/Sec is calculated as available on tt-metal homepage here?