tenstorrent / tt-buda

Tenstorrent TT-BUDA Repository
Other
162 stars 21 forks source link

[bug report] BUDA failing to compile Waifu2X (and other models) with error `Unimplemented narrow in buda` #29

Open marty1885 opened 1 month ago

marty1885 commented 1 month ago

I'm trying to bring up early image processing models like Waifu2X and SRCNN, among models in other categories like RWKVv4 on BUDA. I would expect these are relatively easy as they only use standard and old-fashioned design. However, I often running into the error Unimplemented narrow in buda.

I have uploaded the code to my GitHub. Please let me know if you can't replicate the issue/anything I can help. https://github.com/marty1885/waifu2x-buda

OS: Arch Linux Card: Grayskull e75 Python: 3.10 BUDA: v0.15.0-alpha

The error log:

2024-06-06 09:16:50.632 | INFO     | pybuda.compile:pybuda_compile_from_context:240 - Running compile stage buda_graph_pre_placer
2024-06-06 09:26:57.256 | ERROR    | pybuda.device:run_next_command:469 - Compile error: Unimplemented narrow in buda
Traceback (most recent call last):
  File "/home/marty/micromamba/envs/buda/lib/python3.10/site-packages/pybuda/device.py", line 458, in run_next_command
    ret = self.compile_for(
  File "/home/marty/micromamba/envs/buda/lib/python3.10/site-packages/pybuda/ttdevice.py", line 820, in compile_for
    self._compile_output = pybuda_compile_from_context(compile_context)
  File "/home/marty/micromamba/envs/buda/lib/python3.10/site-packages/pybuda/compile.py", line 248, in pybuda_compile_from_context
    next_stage = stage_to_func[current_stage](context)
  File "/home/marty/micromamba/envs/buda/lib/python3.10/site-packages/pybuda/compile.py", line 858, in run_pre_placer_pass
    context.lowered_graph, context.placer_config_update = run_pre_placer_buda_passes(
  File "/home/marty/micromamba/envs/buda/lib/python3.10/site-packages/pybuda/op/eval/pybuda/__init__.py", line 228, in <lambda>
    return lambda *inputs : module_or_class.lower(op_type.op, op_type.attr, *inputs)
  File "/home/marty/micromamba/envs/buda/lib/python3.10/site-packages/pybuda/op/eval/pybuda/tm.py", line 844, in lower
    raise NotImplementedError("Unimplemented narrow in buda")
NotImplementedError: Unimplemented narrow in buda

Traceback (most recent call last):
  File "/home/marty/Documents/waifu2x-ttbuda/main.py", line 24, in <module>
    outqueue = pybuda.run_inference()
  File "/home/marty/micromamba/envs/buda/lib/python3.10/site-packages/pybuda/run/api.py", line 90, in run_inference
    return _run_inference(module, inputs, input_count, output_queue, _sequential, _perf_trace, _verify_cfg)
  File "/home/marty/micromamba/envs/buda/lib/python3.10/site-packages/pybuda/run/impl.py", line 277, in _run_inference
    return _run_devices_inference(
  File "/home/marty/micromamba/envs/buda/lib/python3.10/site-packages/pybuda/run/impl.py", line 467, in _run_devices_inference
    output_queue = _initialize_pipeline(False, output_queue, sequential=sequential, verify_cfg=verify_cfg)
  File "/home/marty/micromamba/envs/buda/lib/python3.10/site-packages/pybuda/run/impl.py", line 414, in _initialize_pipeline
    _compile_devices(sequential, training=training, sample_inputs=sample_inputs, sample_targets=sample_targets, microbatch_count=microbatch_count, verify_cfg=verify_cfg)
  File "/home/marty/micromamba/envs/buda/lib/python3.10/site-packages/pybuda/run/impl.py", line 1248, in _compile_devices
    raise ret
NotImplementedError: Unimplemented narrow in buda
staylorTT commented 1 month ago

@nvukobratTT Can you help me find someone that can look at this?