search

eastriverlee / LLM.swift

LLM.swift is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.
MIT License
326 stars 34 forks source link

Race condition #10

Closed djmango closed 7 months ago

djmango commented 7 months ago
image

Just gonna submit a PR for this real quick adding a conditional

djmango commented 7 months ago
image

I think its a race condition

djmango commented 7 months ago

Or maybe sending in an empty array + content causes this? going to just stop sending empty arrays lol but something to consider

eastriverlee commented 7 months ago

okay i should consider switching main logic to actor. thank you for reporting.

djmango commented 7 months ago 
    ###Instructions###

    Provide exactly two words for a title for the above chat. YOU WILL BE PENALIZED for any additional words, print exactly two words

For some reason this prompt causes a crash - this is the cause of the Int32 crash i posted above

eastriverlee commented 7 months ago

if you give me the model, the seed, and the template you've used, it will be easier for me to check later. can you provide me those details?

eastriverlee commented 7 months ago

give me any minimal reproducible case if you can please.

djmango commented 7 months ago

Running mistral-7b-instruct-v0.2.Q4_K_M.gguf - not sure about seed, how could that be determined? It replicates every time running that string on .respond (using the .mistral template)