NoumanAmir657
commented
14 hours ago @ammarhd Not really related to your issue but did you run into this error?
error: "<eval_with_key>.0 from /home/nouman-10x/shark-model-dev/turbine_venv/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py:551 in wrapped":95:0: 'torch.aten.copy_' op operand #0 must be Multi-dimensional array modeling Torch's Tensor type, but got '!torch.vtensor<[?,?],f32>'Also you might want to try
iree-compiler: 20240828.999
iree-runtime: 20240828.999
transformers: 4.37.1
torch: 2.3.0+cpu
shark-turbine: 2.4.1
we actually have had this working and it was still working up till last week stateless_llama.py it was working using shark_turbine and all of a sudden we start getting an issue and unfortunately we had some libraries not locked with specific version and I guess there few updates released last week and it did not work anymore .... I tried to use older versions of iree-turbine and torch but could not make it work any more.
now we are getting the following error
... Traceback (most recent call last): File "/app/app/services/../pipelines/llama_2_pipeline/run_pipeline.py", line 92, in slice_up_to_step\n sliced = IREE.tensor_slice(\n ...... File "/usr/local/lib/python3.10/dist-packages/iree/turbine/aot/support/procedural/iree_emitter.py", line 285, in tensor_slice\n result.set_dynamic_dim_values(result_dynamic_dims)\n ...... File "/usr/local/lib/python3.10/dist-packages/iree/turbine/aot/support/procedural/iree_emitter.py", line 285, in tensor_slice\n result.set_dynamic_dim_values(result_dynamic_dims)\n ...... File "/usr/local/lib/python3.10/dist-packages/iree/turbine/aot/support/procedural/primitives.py", line 168, in set_dynamic_dim_values\n assert len(values) == 0, "Dynamic dims not currently supported"\nAssertionError: Dynamic dims not currently supported\n'} ...I have used the following and tested different versions
torch 2.5.1 also tested with 2.4.1 iree-base-compiler 2.9.0 iree-base-runtime 2.9.0 iree-turbine 2.9.0 also tried with 2.5 and 2.3 python 3.10 and 3.11 rocm/dev-ubuntu-22.04:6.0.2 and 6.1