mybigday / llama.rn

React Native binding of llama.cpp
MIT License
300 stars 24 forks source link

Unable to import model #76

Open Prasad-178 opened 2 months ago

Prasad-178 commented 2 months ago

Hello, sorry I am very new to this and it might be a very basic problem. I get the following error while trying to load my model: [TypeError: Cannot read property 'initContext' of null]

This is the model I am using currently: tinyllama-1.1b-chat-v0.3.Q3_K_L.gguf

This is my code:

` import { StyleSheet, TextInput, Button } from "react-native"; import { Text, View } from "react-native"; import { initLlama } from "llama.rn"; import { useState } from "react";

export default function TabOneScreen() { const [input, setInput] = useState(""); const [result, setResult] = useState("");

const llama = async (userInput) => {
console.log("input is : ", userInput)

let context
try {
    context = await initLlama({
      model: "C:/My_Files/Learning/Appdev/expo-demo-example/app/models/tinyllama-1.1b-chat-v0.3.Q3_K_L.gguf",
      use_mlock: true,
      n_gpu_layers: 0
    });
} catch (err) {
    console.log("err is : ", err)
}

console.log("context is : ", context)

const stopWords = [
  "</s>",
  "<|end|>",
  "<|eot_id|>",
  "<|end_of_text|>",
  "<|im_end|>",
  "<|EOT|>",
  "<|END_OF_TURN_TOKEN|>",
  "<|end_of_turn|>",
  "<|endoftext|>",
];

let textResult
try {
    textResult = await context!.completion(
      {
        prompt: `This is a conversation between user and llama, a friendly chatbot. respond in simple markdown.\n\nUser: ${userInput}\nLlama:`,
        n_predict: 100,
        stop: [...stopWords, "Llama:", "User:"],
      }
    );
} catch (err) {
    console.log(err)
}
console.log("text result is : ", textResult)

setResult(textResult!.text);

};

const handlePress = () => { llama(input); };

return (

Chat with Llama
Arnob1998 commented 1 month ago

Any solutions yet?