search

eastriverlee / LLM.swift

LLM.swift is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.
MIT License
382 stars 40 forks source link

getting 'noFilteredURL' thrown #19

Closed nimrodbens closed 5 months ago

nimrodbens commented 7 months ago

i'm trying to initialize 'deepset/roberta-base-squad2' model, and i get this error.

let systemPrompt = "I want you to analyze an answer to a question."
let bot = try await LLM(from: HuggingFaceModel("deepset/roberta-base-squad2", template: Template(systemPrompt: systemPrompt)))

and the error is being thrown in the init. to my understanding there is no link in the html file to download the model directly.

are the only two methods to interact with a model is to have it as a file locally or to download it?

i saw that some model weigh a few GB so either way is not feasible for my app.

i'm new with LLM so sorry if i asked obvious questions.

eastriverlee commented 5 months ago

sorry for the late reply. the code that you wrote is not working because the repository you chose has no gguf file. it has to be supported by LLM.swift's backend which is llama.cpp. "deepset/roberta-base-squad2" is not supported in llama.cpp. if you are looking for a small model, i recommend you using "bartowski/h2o-danube2-1.8b-chat-GGUF" or tinyllama.