fix ctransformers default arguments

[cartazio]

currently

https://github.com/anarchy-ai/LLM-VM/blob/main/src/llm_vm/onsite_llm.py#L305-L331

the model file and model uri should be keyword args for the init, and stuff like that needs cleanup

[bilal-aamer]

Hey @cartazio,

Is this something you mean?

def __init__(self, model_uri=None, model_file=None, tokenizer_kw_args={}, model_kw_args={}):
    if model_uri is not None:
        self.model_uri = model_uri
    if model_uri is None and self.model_uri is None:
        raise ValueError('A very specific bad thing happened.')

    self.model_file = model_file
.
.
.

[cartazio]

the keyword arguments should be set with a REPO and a filename by default,

like https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML and llama-2-7b-chat.ggmlv3.q2_K.bin https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/blob/main/llama-2-7b-chat.ggmlv3.q2_K.bin or something like that, i'm probably getting the uri slightly wrong :)

On Tue, Sep 19, 2023 at 4:12 PM Bilal Aamer @.***> wrote:

Hey @cartazio https://github.com/cartazio,

Is this something you mean?

def init(self, model_uri=None, model_file=None, tokenizer_kw_args={}, model_kw_args={}): if model_uri is not None: self.model_uri = model_uri if model_uri is None and self.model_uri is None: raise ValueError('A very specific bad thing happened.')

self.model_file = model_file

. . .

— Reply to this email directly, view it on GitHub https://github.com/anarchy-ai/LLM-VM/issues/281#issuecomment-1726404591, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAABBQT2TI6LYY2T7HKYA7LX3H4DTANCNFSM6AAAAAA44777QE . You are receiving this because you were mentioned.Message ID: @.***>

[cartazio]

looking at the example docs in pypi for ctransformers

from ctransformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("marella/gpt-2-ggml", hf=True)
tokenizer = AutoTokenizer.from_pretrained(model)

looks like we're not quite doing that ..

[bilal-aamer]

@cartazio Got a PR out on this

[cartazio]

Looking more closely I was just dumb, the code is correct