Closed JushBJJ closed 1 month ago
Hi There, thanks for filing this issue. I will do my best to get someone to take a look at this from the TT side.
To clarify @JushBJJ are the inputs to the op constant (even if they're call nodes). That is, would we be able to extract them using _infer_value
?
To clarify @JushBJJ are the inputs to the op constant (even if they're call nodes). That is, would we be able to extract them using
_infer_value
?
I will get into this very soon later today, my e75 just arrived yesterday
@JushBJJ Do you have any updates here?
@JushBJJ Do you have any updates here?
Hi, incredibly sorry for forgetting about this. Since I don't have much to do atm this is full priority now. Currently trying to pass some Buda hurdles atm with the more recent updates but so far I haven't reached this issue again...yet
Appreciate the update. If you hit this again please report back.
@staylorTT Yup this can stay closed, can confirm that this is no longer an issue
Hi I'm doing the bounties with @marty1885 and @JonathanALevine.
Context
When attempting to implement Qwen 1.5 (0.5B) https://github.com/tenstorrent/tt-buda-demos/issues/20, we encountered an issue where we had this error:
Qwen 1.5 (0.5B) implementation demo can be found in https://github.com/tenstorrent/tt-buda-demos/pull/37
The OP code in this case for triu would be:
*Requires transformers >=v4.37.0
Root Cause
The
triu
function in the PyTorchOpConverter class intvm/relay/frontend/pytorch.py
uses np.triu to handle these inputs.mask = np.triu(np.ones(x_shape), inputs[1]).astype(np.bool)
However, it is not possible to handle these input types when the
inputs
parameter is a list containing nested functions. In this case,inputs[1]
would be:Workaround
In draft PR https://github.com/tenstorrent/tt-tvm/pull/2, I used
self.trilu(inputs, input_types, mode="triu")
simply because_op.trilu
can do triangular upper instead of triangular bottom ifupper=True
.This workaround successfully compiles the Qwen 1.5 (0.5B) model but was still unable to properly run.
See https://github.com/tenstorrent/tt-tvm/pull/2 to see the issues of this workaround.
Environment
OS: Ubuntu 20.04 Pybuda Version: v0.10.5.gs.240315 TVM Version (from latest Pybuda): 0.14.0
Steps to reproduce
See https://github.com/tenstorrent/tt-buda-demos/pull/37
Triage
frontend:pytorch