search

GreyDGL / PentestGPT

A GPT-empowered penetration testing tool
MIT License
6.67k stars 796 forks source link

expand support to Claude, Cohere, Llama2 and Azure #142

Open krrishdholakia opened 11 months ago

krrishdholakia commented 11 months ago

Hi @GreyDGL,

Saw you expanded support to LocalLLMs - and wanted to see how to add support for other LLM providers. I'm working on litellm (simple library to standardize LLM API Calls - https://github.com/BerriAI/litellm) and was wondering if we could be helpful.

Added support for Anthropic, Llama2, and Cohere by replacing the raw openai.chatCompletion.create function with completion. The code is pretty similar to the OpenAI class - as litellm follows the same pattern as the openai-python sdk.

Would love to know if this helps.

krrishdholakia commented 11 months ago

Happy to make any relevant changes if required

vmayoral commented 11 months ago

@krrishdholakia did you tested this out? can you report on any experiments conducted with it?

krrishdholakia commented 11 months ago

Hey @vmayoral tested it with Anthropic (claude-instant-1).

Screenshot 2023-08-07 at 12 22 51 PM
krrishdholakia commented 11 months ago

The PR checks for environment keys for anthropic/cohere/replicate in addition to the openai keys.

Depending on which model is passed in by the user it routes to the correct spec.

ishaan-jaff commented 9 months ago

@vmayoral any updates on this PR ?

Are you exploring building a hosted version of Pentestgpt ?

krrishdholakia commented 9 months ago

Bump on this? @vmayoral @GreyDGL