microsoft / BitNet

Official inference framework for 1-bit LLMs
MIT License
11.43k stars 773 forks source link

Refactor TL2 codegen #84

Open JCGoran opened 1 month ago

JCGoran commented 1 month ago

The codegen for TL2 was a bit difficult to reason about since the C++ code was directly embedded as a Python string. With this PR, I've added a simple Jinja2 template which contains the whole file, making it much easier to follow as well as make changes.

Other notable changes:

The generated bitnet-lut-kernels.h is more or less identical (with some slight whitespace differences).

Note that I didn't do the same for TL1 codegen since I'd first like to get feedback on whether this is a step in the right direction.