search

tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
Apache License 2.0
380 stars 47 forks source link

TrOcr: Enable TTNN support #9287

Open Sudharsan-V opened 3 months ago

Sudharsan-V commented 3 months ago

Enable support for TrOcr model using ttnn ops.

jayasuryamaganuru commented 2 months ago

Sub module Tests for TrocrAttention , TrocrDecoderLayer is done - commit

Sudharsan-V commented 2 months ago

The ttnn pipeline for the trocr Decoder sub-module is enabled with PyTorch Embedding layer. The pcc for the sub-modules are updated in the sheet Corresponding WIP-commit

jayasuryamaganuru commented 2 months ago

The reference implementation for TrOCRForCausalLM is completed, PCC for all the sub-modules are all 1.0 corresponding commit

jayasuryamaganuru commented 2 months ago

pipeline for reference implementation vision encoder decoder model is in progress, corresponding - WIP-commit

jayasuryamaganuru commented 1 month ago

Reference implementation for trocr is completed. corresponding PR - link

vigneshkeerthivasanx commented 1 month ago

Commit for ttnn functional implementation for trocr and demo is created. corresponding commit

vigneshkeerthivasanx commented 1 month ago

ttnn trocr demo results are recorded in docs: Link ttnn trocr PCC results are recorded in sheets: Link updated commit : Link

jayasuryamaganuru commented 1 month ago

w.r.t device_performance script for trocr, facing the warning related to DRAM buffers multiple times WARNING  | Profiler DRAM buffers were full, markers were dropped! device 0, worker core 1, 1, Risc NCRISC, bufferEndIndex = 15996

followed by assertion error while appending device data Traceback (most recent call last): File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/ubuntu/jayasurya/fresh/tt-metal/tt_eager/tracy.py", line 346, in <module> main() File "/home/ubuntu/jayasurya/fresh/tt-metal/tt_eager/tracy.py", line 330, in main generate_report(options.output_folder, options.name_append, options.child_functions) File "/home/ubuntu/jayasurya/fresh/tt-metal/tt_eager/tracy.py", line 166, in generate_report process_ops(outFolder, nameAppend, True) File "/home/ubuntu/jayasurya/fresh/tt-metal/tt_metal/tools/profiler/process_ops_logs.py", line 526, in process_ops deviceOps = append_device_data(ops, PROFILER_LOGS_DIR) File "/home/ubuntu/jayasurya/fresh/tt-metal/tt_metal/tools/profiler/process_ops_logs.py", line 193, in append_device_data assert len(deviceOps[device]) == len( AssertionError: Device data mismatch. Expected 5277 but received 2839 ops on device 0

Steps to reproduce:

vigneshkeerthivasanx commented 1 month ago

ttnn implementation and demo file for functional trocr is completed. corresponding PR : #10394