Open gitknu opened 9 months ago
Hello @gitknu,
You can find the source code for the playground and other examples over here: https://github.com/rahuldshetty/ggml.js-examples
You just need to provide the relative path to the model file in your local machine.
E.g code:
<script type="module">
import {LLM} from "../dist/llm.js";
let submitButton = document.getElementById('submitBtn');
let outputElement = document.getElementById('output');
const on_loaded = () => {
submitButton.disabled = false;
submitButton.innerText = "Generate";
}
const write_result = (line) => {
outputElement.textContent += line + "\n";
}
const run_model = () => {
const text = document.getElementById("textInput").value;
submitButton.innerText = "Running...";
submitButton.disabled = true;
outputElement.textContent = ""; // clead old content
app.run({
prompt: text,
max_token_len: 64,
top_k: 1,
});
}
const run_complete = () => {
submitButton.innerText = "Generate";
submitButton.disabled = false;
}
const app = new LLM(
'LLAMA',
'../models/llama2_xs_460m_experimental_evol_instruct.q4_k_m.gguf',
on_loaded,
write_result,
run_complete
);
app.load_worker();
submitButton.addEventListener("click", run_model);
</script>
This works with local live-server: https://marketplace.visualstudio.com/items?itemName=ritwickdey.LiveServer
Hello, sorry for stupid question, but can you upload the project with Playground it seelf and the manual how to run model locally, just by picking downloaded GGUF file? I would like to run offline webserver and try running models on isolated machine.
Thank you for understanding and such a great job you`ve done!