Improvement: API support for Groq, ooba, exllama2, kobold.cpp

rmusser01 / tldw

Too Long, Didn't Watch(TL/DW): Your Personal Research Multi-Tool - Open Source NotebookLM

Apache License 2.0

45 stars 2 forks source link

Improvement: API support for Groq, ooba, exllama2, kobold.cpp #14

Closed rmusser01 closed 1 month ago

rmusser01 commented 1 month ago

A user should be able to specify Groq as an API endpoint to be used for summarization.

rmusser01 commented 1 month ago

Also, oobabooga & exllama2

rmusser01 commented 1 month ago

Also kobold.cpp

rmusser01 commented 1 month ago

Groq works. Code for ooba and kobold are in. Token/text issue with kobold, but otherwise fine. Prompt is parsed, but not correctly, or somethings off.

rmusser01 commented 1 month ago

ooba now works.

rmusser01 commented 1 month ago

Kobold works.

rmusser01 commented 1 month ago

exllama2 doesn't have an API turns out.