Dear authors of TinyChatEngine, I was looking at your code here and would love to discuss how we can hook the Metal and AVX kernels into Python such that they can be reused in other frameworks like AutoAWQ. I am most interested in the W4A16 part since that is what AutoAWQ focuses on for now.
What do you think the next steps are to create a Python extension and would you be willing to help give me pointers on how to do it in AutoAWQ?
https://github.com/casper-hansen/AutoAWQ
Dear authors of TinyChatEngine, I was looking at your code here and would love to discuss how we can hook the Metal and AVX kernels into Python such that they can be reused in other frameworks like AutoAWQ. I am most interested in the W4A16 part since that is what AutoAWQ focuses on for now.
What do you think the next steps are to create a Python extension and would you be willing to help give me pointers on how to do it in AutoAWQ? https://github.com/casper-hansen/AutoAWQ