search

context-labs / autodoc

Experimental toolkit for auto-generating codebase documentation using LLMs
MIT License
1.98k stars 117 forks source link

Support Alpaca and Llama models #8

Open samheutmaker opened 1 year ago

samheutmaker commented 1 year ago

Autodoc is currently reliant on OpenAI for access to cutting-edge language models. Going forward, we would like to support models running locally or at providers other than OpenAI, like Llama, or Alpaca. This gives developers more control over how their code is indexed, and allows indexing of private code that cannot be shared with OpenAI.

This is a big undertaking that will be an on-going process. A few thoughts for someone who wants to get starting hacking on this.

  1. It would be nice to be able to configure Autodoc with a LangChain LLM via the Autodoc config file. This would allow for complete control over how an LLM is configured.
  2. It seems like a lot of people are using llamma.cpp to run llamma locally. It may be worth using this as a starting point to support other models.

This issue is high priority. If you're interesting in working on it, please reach out.

dahifi commented 1 year ago

Ideally the model configuration stuff should be abstracted out to something like langchain, it's too bad there no TS port for it yet.

samheutmaker commented 1 year ago

Yes there is, we already use it in Autodoc. https://github.com/hwchase17/langchainjs

katopz commented 1 year ago

Maybe consider use https://github.com/rustformers/llama-rs in that case? and also https://github.com/sobelio/llm-chain for langchain.

quicoli commented 1 year ago

have you seen this project? https://github.com/microsoft/semantic-kernel It might help with this task....

Semantic Kernel (SK) is a lightweight SDK enabling integration of AI Large Language Models (LLMs) with conventional programming languages. The SK extensible programming model combines natural language semantic functions, traditional code native functions, and embeddings-based memory unlocking new potential and adding value to applications with AI.

SK supports prompt templating, function chaining, vectorized memory, and intelligent planning capabilities out of the box.

cognitivetech commented 3 months ago

ollama supports openai api