Closed rasbt closed 3 months ago
Hey Seb! @nikitaved Just merged a PR to improve the messaging here: #78
The TLDR is that you want to run examine
on the model to get a report of what's not working:
from thunder.examine import examine
x = ...
model = ...
examine(model, x)
It would be useful if you can include here what it reports for those models.
This is nice, thanks! The report is
Files already downloaded and verified
Found 18 distinct operations, of which 15 (83.3%) are supported
Please file an issue requesting the following operators here: https://github.com/Lightning-AI/lightning-thunder/issues/new
TensorBase.is_nested
multi_head_attention_forward of torch.nn.functional
_assert of torch
So the culprit seems to be https://github.com/pytorch/pytorch/blob/1e8d4b389b5f03cea191ed558051f036fe04f92d/torch/nn/functional.py#L5163
triage review:
We think there are three issues here:
First issue:
Second issue:
is_nested
to False and ensuring programs passed nested tensors failThird issue:
Can we break this issue up into those three, @rasbt?
This sounds totally reasonable, please feel free to break it up into these three.
Re first issue: Not sure if that's feasible, but perhaps even automatically calling examine
upon failure could not be a bad thing for users.
We do seem to be able to run ResNet as of today and vit_b_16 (at least), thanks to #584 and #633 . :tada:
🐛 Bug
I was trying to run a simple torchvision ViT and am getting the following error:
Not sure how to go about debugging this. I thought that sharing this may help improving thunder in terms of supporting more models and edge cases
To Reproduce
Steps to reproduce the behavior:
I attached self-contained code in the zip.
Code sample
See zip attached
Expected behavior
Either a clearer error message or ideally it should work :)
Environment
Same as Zero to Thunder studio.
Archive.zip
cc @apaz-cli