search

Lightning-AI / lightning-thunder

Make PyTorch models up to 40% faster! Thunder is a source to source compiler for PyTorch. It enables using different hardware executors at once; across one or thousands of GPUs.
Apache License 2.0
1.17k stars 77 forks source link

Non-supported diffusion transformer operators #133

Open odegeasslbc opened 6 months ago

odegeasslbc commented 6 months ago 
TensorBase.bfloat16
_set_grad_enabled of torch._C
_VariableFunctionsClass.empty of torch
TensorBase.long
TensorBase.type
TensorBase.__setitem__
_VariableFunctionsClass.lerp of torch
device of torch
TensorBase.clone
TensorBase.masked_fill_
TensorBase.get_device
TensorBase.grad_fn
_VariableFunctionsClass.linspace of torch

cc @apaz-cli

carmocca commented 6 months ago

Adding more context for other viewers: the above is the output of examine for a diffusion transformer

mruberry commented 6 months ago

Thank you for filing this issue, @odegeasslbc!

We'll look at prioritizing these operators soon. We should probably create a larger operator issue with operators that are good first issues, ones that are more complicated, and ones that are too complicated to handle at this time. From the above list, the inplace operations, like setitem and maskedfill probably won't be supported in the near future. A workaround is to use thunder around a smaller portion of the model that doesn't include these operators, or revise that part of the model to use out-of-place operations.