search

SciSharp / LLamaSharp

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
https://scisharp.github.io/LLamaSharp
MIT License
2.56k stars 334 forks source link

Stateless executor doesn't work with LlamaSharp 0.10 & NET 8.0 #578

Open pugafran opened 7 months ago

pugafran commented 7 months ago

I'm missing something? When I execute stateless executor example in LlamaSharp project it works, but I replicated exactly the same code in my own project using last versions and it crashes.

image

martindevans commented 7 months ago

Did you use the same model when you tested it in the Examples project and when you tested it in your own app?

pugafran commented 7 months ago

Yes I did, Ironically I have fixed it by commenting out "await Task.Delay(75);" in the spinner method. What I am unable to understand is that the "stateless executor" does not work without the spinner, and with the spinner only if you modify the line I said.

martindevans commented 7 months ago

Well that is bizarre! I've used the stateless executor without a spinner myself (e.g. in the unit tests).

Is there any more detail on what the crash is? It looks like exit code zero from your screenshot, but I'm not certain.

pugafran commented 7 months ago

Yes, ends with code zero. Debugging leads to "ex.InferAsync(prompt, inferenceParams)" and then ends. If I put the spinner method it ends inside in "await Task.Delay(75)" line.

Any other information you need, let me know and I will be happy to provide you with it, the truth is that logically speaking the situation kills me. At least it works hahahaha

AsakusaRinne commented 6 months ago

Hi, does this error appear every time you run it? I couldn't reproduce it on my PC.

pugafran commented 6 months ago

Yes, still happens everytime if I don't comment "await Task.Delay(75)" line inside spinner method, and if I don't use the spinner method it also gives error. The only difference that I detect is that we don't use Task for our run(), it returns void, but it shouldn't affect in that way, I have tried several times to change the whole structure of the project to use Task and it still didn't work. It's not a very important problem because doing what I said makes it work, but I don't know if it doesn't work because of my ignorance or because something is really wrong.