search

mlc-ai / mlc-llm

Universal LLM Deployment Engine with ML Compilation
https://llm.mlc.ai/
Apache License 2.0
18.59k stars 1.5k forks source link

[Feature Request] vulkan backend on android #285

Open rhjdvsgsgks opened 1 year ago

rhjdvsgsgks commented 1 year ago

🚀 Feature

use vulkan as a fallback backend on android

Motivation

some android device may not have proper opencl support. add vulkan fallback will add compatibility to these device

Alternatives

maybe also cpu(neon) backend?

Additional context

the best solution should be nnapi. but it not avaliable on tvm yet. https://discuss.tvm.apache.org/t/rfc-byoc-android-nnapi-integration/9072 https://github.com/apache/tvm/pull/8076

Pevernow commented 1 year ago

This is a good idea.

Why didn't developers pay attention?