AI-Hypercomputer / jetstream-pytorch

PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"
Apache License 2.0
42 stars 15 forks source link

Fix: correct quantization name filtering #196

Closed tengomucho closed 1 month ago

tengomucho commented 1 month ago

The quantization filter based on layer names did not work, because modules walk is done with the Module.apply method, that resolves names locally, so the "absolute" naming does not work. The fix just prepares a list out of the names before entering the loop, so the correct reference is captured.