search

octoml / mlc-llm

Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.
https://mlc.ai/mlc-llm
Apache License 2.0
5 stars 8 forks source link

[FP8][PTQ] Support packing fp8 into uint8 #246

Closed csullivan closed 5 months ago

csullivan commented 5 months ago

Simplify packing to allow packing fp8 into uint8 storage