Open Ademsk1 opened 6 months ago
Nice. Have you tested it?
@mishushakov tested for the openai section, doesn't crash the server anymore :D
I haven't tested the Llama one, but I think it should work the same. I started working on one last night, so will let you know of the results later on this evening.
Very good! I will run some tests too. Looking forward to get it merged 😄
@mishushakov Been trying to test using this model as my model, but my machine cannot handle it. What model are you using?
TinyLlama
I can test for you, no worries
Tested this with TinyLlama and no crashes :D
that being said, there weren't any crashes without the try/catch block for the local model, because we don't reference any tool_calls or the like. Still I think it's practical to have it there just in case.
While I have you @mishushakov , maybe you have more experience, with these models, I seem to get this error when trying to scrape data from a fairly large website.
GGML_ASSERT: ... n_tokens_all <= cparams.n_batch
Did you have this error at all? Any advice?
I believe you're hitting the input token limit. We need to think about a way of splitting websites into multiple chunks, maybe. Can you open an issue for this?
Hi
This might help with issue #20 . I'm hoping to test it for openai later on today, but I don't have access to a local language model right now, are there any models you recommend I use?
Thanks