Simplified llama2.c.dll

[JohnClaw]

Llama.dll that can be downloaded from llama.cpp repo is suitable mostly for programming languages that have abilities to work with rather difficult (for novice coder) concepts like pointers, structures etc. The same thing applies to this repo. So the question is: does anybody know any simplified implementation of llama2.c.dll or llama.cpp.dll? For example, it would be wonderful if it will be possible to work with this dll in such way:

CONSOLE

Dim LLama_DLL as Integer

Load "llama.dll" as LLama_DLL

Call function Load_Local_Gguf ("Llama-2-7b-q8.0.gguf") from LLama_DLL

Call function Set_Context_Size_For_Loaded_Gguf (4096) from LLama_DLL

Dim Answer$ As String

Dim Question$ As String

Label #Begin

Input Question$

Answer$ = Call function SentPromptToLoadedGguf (Question$) from LLama_DLL

Print Answer$

Goto #Begin

[jameswdelancey]

I have done this to call from PHP for a web app but I haven't shared it on github yet. It's not a big modification, especially since the refactor on this repo. It consists of passing a pointer to the main function with what would be arguments for a CLI, putting the pointer in transformer.data, replacing the fread with memcpy in the read_configuration, and I think that's it. I can upload if there's nothing out there right now.

[jameswdelancey]

https://github.com/jameswdelancey/llama3.c/blob/master/runqdll.c#L1054

If you look at the bottom, you can see how quick and dirty replicated the CLI args with the build_main call. This works with the quantized version of LLaMA 3B base or instruct when exported by export.py as a version-2 (quantized ak42 magic)

[jameswdelancey]

It doesn't work with LLaMA2 or tinyshakespear or tinystories though, because the overhaul to Llama3 has taken place, but can be backported.

[JohnClaw]

It doesn't work with LLaMA2 or tinyshakespear or tinystories though, because the overhaul to Llama3 has taken place, but can be backported.

I found dll that can chat with llm-s using one simple function without pointers and structures: https://github.com/tinyBigGAMES/Dllama It's ideal for novice coders like me and can be used from any programming language. For example, here's AutoIt's (https://www.autoitscript.com/site/) implementation:

Local $answer = DllCall($hDLL, "str:cdecl", "Dllama_Simple_Inference", "str", "config.json", "str", "llama3", "str", $question, "uint", 1024)

Could you add such simple function to your dll, please?