dvartaniansTT
commented
4 months ago note the pre-written prompts test for falcon7b passes. the error happens when providing my own prompt.
PASSING: pytest --disable-warnings -q -s --input-method=json --input-path='models/demos/falcon7b/demo/input_data.json' models/demos/wormhole/falcon7b/demo_wormhole.py::test_demo[user_input0-default_mode_stochastic]
FAILING: pytest --disable-warnings -q -s --input-method=cli --cli-input="Tell me about computer architecture" models/demos/wormhole/falcon7b/demo_wormhole.py::test_demo[user_input0-default_mode_stochastic]
error: `models/demos/wormhole/falcon7b/demo_wormhole.py:37:
models/demos/falcon7b/demo/demo.py:432: in run_falcon_demo_kv tt_logits, kv_cache = tt_FalconCausalLM( python_env/lib/python3.8/site-packages/torch/nn/modules/module.py:1511: in _wrapped_call_impl return self._call_impl(*args, kwargs) python_env/lib/python3.8/site-packages/torch/nn/modules/module.py:1520: in _call_impl return forward_call(*args, *kwargs) models/demos/falcon7b/tt/falcon_causallm.py:98: in forward hidden_states, presents = super().forward( models/demos/falcon7b/tt/falcon_model.py:283: in forward layer_output = layer( python_env/lib/python3.8/site-packages/torch/nn/modules/module.py:1511: in _wrapped_call_impl return self._call_impl(args, kwargs) python_env/lib/python3.8/site-packages/torch/nn/modules/module.py:1520: in _call_impl return forward_call(*args, kwargs) models/demos/falcon7b/tt/falcon_decoder.py:178: in forward attn_outputs = self.self_attn_decode( python_env/lib/python3.8/site-packages/torch/nn/modules/module.py:1511: in _wrapped_call_impl return self._call_impl(*args, *kwargs) python_env/lib/python3.8/site-packages/torch/nn/modules/module.py:1520: in _call_impl return forward_call(args, kwargs) models/demos/falcon7b/tt/falcon_attention.py:675: in forward self.query_key_value_weights[i],
input_tensor = ttnn.Tensor([[[[ 0.02100, -0.02197, ..., -0.03271, 0.01025], [ 0.00488, 0.00879, ..., 0.00977, 0....0.00977, ..., -0.00903, 0.00537]]]], shape=Shape([1, 1, 4544, 4672]), dtype=DataType::BFLOAT8_B, layout=Layout::TILE) slices = (slice(None, 0, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
@ttnn.register_operation(
name="ttnn.Tensor.__getitem__",
validate_input_tensors=_getitem_validate_input_tensors,
is_method=True,
golden_function=_golden_function,
)
def __getitem__(input_tensor: ttnn.Tensor, slices) -> ttnn.Tensor:
input_rank = len(input_tensor.shape)
input_layout = input_tensor.layout
if isinstance(slices, int):
slices = (slice(None, slices, None),)
elif isinstance(slices, slice):
slices = (slices,)
elif isinstance(slices, type(...)):
raise RuntimeError("Ellipsis is not supported!")
normalized_slices = []
for s in slices:
if isinstance(s, int):
normalized_slices.append(slice(None, s, None))
elif isinstance(s, slice):
normalized_slices.append(s)
else:
raise RuntimeError("Invalid slice type!")
slices = tuple(normalized_slices)
while len(slices) != input_rank:
slices = slices + (slice(None, None, None),)
if isinstance(slices, tuple):
if len(slices) > input_rank:
raise RuntimeError(f"Too many slices for tensor of rank {input_rank}")
if input_rank <= 4:
input_tensor = ttnn.unsqueeze_to_4D(input_tensor)
while len(slices) != 4:
slices = (slice(None, None, None),) + slices
slice_start = [_slice.start if _slice.start is not None else 0 for _slice in slices]
slice_end = [
(_slice.stop if _slice.stop is not None else input_tensor.shape[index])
for index, _slice in enumerate(slices)
]
padded_slice_end = list(slice_end)
if input_layout == ttnn.TILE_LAYOUT:
padded_slice_end[-1] = int(math.ceil((slice_end[-1]) / ttnn.TILE_SIZE)) * ttnn.TILE_SIZE
padded_slice_end[-2] = int(math.ceil((slice_end[-2]) / ttnn.TILE_SIZE)) * ttnn.TILE_SIZE
if list(padded_slice_end) == list(input_tensor.shape.with_tile_padding()):
output = input_tensor
else:
padded_slice_end_minus_1 = [x - 1 for x in padded_slice_end]
if any([x < 0 for x in padded_slice_end_minus_1]):raise RuntimeError("ttnn.Tensor.__getitem__: cannot return a scalar!")E RuntimeError: ttnn.Tensor.getitem: cannot return a scalar!
ttnn/ttnn/operations/core.py:92: RuntimeError ================================================================= short test summary info ================================================================== FAILED models/demos/wormhole/falcon7b/demo_wormhole.py::test_demo[user_input0-default_mode_stochastic] - RuntimeError: ttnn.Tensor.getitem: cannot return a scalar! ========================================================= 1 failed, 1 warning in 368.28s (0:06:08) ========================================================= Device | INFO | Closing user mode device drivers `
tt-rkim
skhorasganiTT
Describe the bug I am testing demos for WH on N150. And encountering errors.
To Reproduce Steps to reproduce the behavior:
Expected behavior demos and any suggested pytest in the models' README should pass from main. And, the README's should be regularly updated to reflect the changes in main, be it about build flow, required env variables or anything else that is changed in main.
Please complete the following environment information:
Additional context I will be adding more to this issue as I keep testing.