Open mjmikulski opened 2 months ago
(as requested by @tfogal)
If the line model = thunder.jit(model)
in the above repro script is replaced with the following lines:
from thunder.dynamo import ThunderCompiler
executors = list(thunder.get_default_executors())
backend = ThunderCompiler(executors=executors)
model = torch._dynamo.optimize(backend=backend)(model)
The same error occurs:
RuntimeError: Advanced indexing currently only supports zero or one-dimensional integer tensors, but found a tensor with dtype thunder.dtypes.int64 and 2 dimensions
I also checked a few different models from Qwen family and all failed:
# Qwen 2
Qwen/Qwen2-1.5B-Instruct (originally reported) - fail
Qwen/Qwen2-0.5B-Instruct - fail
Qwen/Qwen2-7B - fail
# Qwen 1.5
Qwen/Qwen1.5-4B - fail
# Qwen 1
Qwen/Qwen-1.8B-Chat - fail (but different error connected to DispatchKeySet)
So fixing this issue has potential to enable qwen 1.5 and qwen 2 models to work with thunder.
Triage review 9/16/24: Problem acknowledged. We welcome the community to update the meta function to solve the problem.
@riccardofelluga, could you please take this one?
🚀 Model / language coverage
I encountered an issue while attempting to use
thunder.jit
with models outside of the lit-gpt universe, specifically the Hugging Face model Qwen2-1.5B-Instruct. The following error is thrown:The shape of the tensor in question is actually (1, 1024), which could potentially be handled with squeeze().
Pitch
Supporting this case could enable compatibility with Qwen2-1.5B-Instruct and possibly with other models from Qwen family.
Alternatives / Potential work-arounds
Adding the following code to the function
_advanced_indexing
inthunder/clang/__init__.py
resolves the issue temporarily:However, the same issue re-emerges in the prims.take_meta function:
Minimal Repro
results in