Open moonlightian opened 5 months ago
Hi I think we already support that https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/mpt#16-fp8-post-training-quantization-with-ammo
This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 15 days."
System Info
None
Who can help?
@Tracin https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/mpt
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
None
Expected behavior
None
actual behavior
None
additional notes
None