search

Lightning-AI / lightning-thunder

Make PyTorch models up to 40% faster! Thunder is a source to source compiler for PyTorch. It enables using different hardware executors at once; across one or thousands of GPUs.
Apache License 2.0
1.15k stars 77 forks source link

Dyanmic Shape TensorProxy #1039

Open jjsjann123 opened 1 month ago

jjsjann123 commented 1 month ago

🚀 Feature

This is a continuation of the work in #262 . The phase 2 proposal targets the re-use of compiled thunder program by supporting TensorProxy with dynamic shape and adjusting caching policy accordingly.

This is an umbrella issue to track progress. I would like to leave detailed technical discussion in separate issues and link them. This section will be gradually populated as we start picking up speed:

jjsjann123 commented 2 weeks ago

Linking feature request #581

I think a better design should make this a constraint, rather than opaquely treat it as a cache miss by backend. Will target this when we proceed to static constraints.