The Llama 7B model works on my Android phone, but other models do not.

[siz0001]

Hello, I've been asking a lot of questions today. After building the Android phone app I created as an example and installing it on a Galaxy S22 model with 12GB of memory, I found that only the Llama 7B model works properly, while other models either cause the app to crash or only respond with repeated <unk><unk><unk><unk><unk><unk><unk><unk>.... The models I tested were Gemma-2b-Q4_K and Mistral-7b-Q4_K, both of which worked well in the PC demo. Is there any variable or setting I need to adjust to apply models other than Llama on Android? Below is the code I modified while switching models.

update : I have noted in the documentation that the current Android app is in a very early stage and only supports the Llama2 and Fuyu models. Do you have any plans to support other models like Gemma? Also, I am curious if it would be possible for Llama2 to work with models that have been fine-tuned for specific languages, such as Llama2-ko. I had many questions, but I sincerely thank you in advance for your prompt response.

[LibHelper.hpp]

    enum PreDefinedModel {
        LLAMA = 0,
        FUYU,
    };

[MainActivity.kt]

    fun initStatus(context: Context,modelType:Int=_modelType.value?:0){
        if (_isExternalStorageManager.value != true) return;
        val modelPath = when(modelType){
         0->"model/llama-2-7b-chat-q4_k.mllm"

//           0->"model/gemma-2b-q4_k.mllm"
//            0->"model/model_q4_k.mllm"
//            0->"model/model.mllm"
//            0->"model/mistral-7b-q4_k.mllm"
            1->"model/llama_2.mllm"
            2->"model/fuyu.mllm"
            else->"model/llama"
        }
        val vacabPath = when(modelType){
          0->"model/llama_vocab.mllm"
//          0->"model/gemma_vocab.mllm"
//          0->"model/vocab.mllm"
//            0->"model/mistral_vocab.mllm"
            1->"model/vocab_uni.mllm"
            else->"model/vocab.mllm"
        }
        var downloadsPath = "/storage/emulated/0/Download/"
        if (!downloadsPath.endsWith("/")){
            downloadsPath = downloadsPath.plus("/")

[chenghuaWang]

cc @lx200916

[lx200916]

Hi siz,

1. Updating the Android App to support more models (such as Gemma) is indeed on our roadmap, but it will have to wait until the JNI interface refactoring is completed, which may take a few months.

2. The LLaMA2 model for finetuning should theoretically be able to replace the current model seamlessly, but some testing may be required. You can refer to the documentation at https://github.com/UbiquitousLearning/mllm?tab=readme-ov-file#customization to attempt replacing the weight and vocabulary files.