search

mit-han-lab / TinyChatEngine

TinyChatEngine: On-Device LLM Inference Library
https://mit-han-lab.github.io/TinyChatEngine/
MIT License
751 stars 73 forks source link

Porting TinyChatEngine to FPGA #117

Open ztachip opened 2 months ago

ztachip commented 2 months ago

I would like to port TinyChatEngine to a proprietary hardware accelerator platform based on FPGA. I would like to run VILA model. Your implementation seems to be optimized for CUDA and NEON. But I would like a reference implementation with just plain C/C++ code as a reference for me to port to new accelerator. Is this possible? Thanks