search

microsoft / graphrag

A modular graph-based Retrieval-Augmented Generation (RAG) system
https://microsoft.github.io/graphrag/
MIT License
8.65k stars 681 forks source link

Which LLM models are supported？ #341

Open yyyhainan opened 6 days ago

yyyhainan commented 6 days ago

Whether other LLM models are supported, such as ChatGLM and QWEN？

Lbaiall commented 6 days ago

i got with same question ....... where can i see the main source code ....

M00N-3 commented 6 days ago

+1

andysingal commented 6 days ago

+1

young169 commented 5 days ago

And also can we use locally deployed LLMs other than via api keys?

zzk2021 commented 5 days ago

same question

gallypette commented 5 days ago

+1

AlonsoGuevara commented 4 days ago

Hi! During our research we got the most quality out of gpt-4, gpt-4-turbo and gpt-4o, that's why out of the box we include support for these in both OpenAI and Azure environments.

Regarding local hosting there's a very interesting conversation going on in this thread #339

bmaltais commented 4 days ago

I have tested gemma2 and llama3 with success. The only thing that does not work locally is the embeddings. There need to be a fix to accept the style of response coming from ollama when quering embeddings... Once that is fixed you will be able to run this 100% local on a personal computer... but probably need a NVidia with 24GB of VRAM like a 3090 or a Mx Mac with 32GB RAM.

zzk2021 commented 4 days ago

I have tested gemma2 and llama3 with success. The only thing that does not work locally is the embeddings. There need to be a fix to accept the style of response coming from ollama when quering embeddings... Once that is fixed you will be able to run this 100% local on a personal computer... but probably need a NVidia with 24GB of VRAM like a 3090 or a Mx Mac with 32GB RAM.

can we use local embedding?

vamshi-rvk commented 4 days ago

I have tested gemma2 and llama3 with success. The only thing that does not work locally is the embeddings. There need to be a fix to accept the style of response coming from ollama when quering embeddings... Once that is fixed you will be able to run this 100% local on a personal computer... but probably need a NVidia with 24GB of VRAM like a 3090 or a Mx Mac with 32GB RAM.

Can you help me with running llama 3 from the local please..

ishotoli commented 2 days ago

I have tested gemma2 and llama3 with success. The only thing that does not work locally is the embeddings. There need to be a fix to accept the style of response coming from ollama when quering embeddings... Once that is fixed you will be able to run this 100% local on a personal computer... but probably need a NVidia with 24GB of VRAM like a 3090 or a Mx Mac with 32GB RAM.

Can you help me with running llama 3 from the local please..

Here's my .env file, put it under ./ragtest dir, hope this can help you: ''' GRAPHRAG_LLM_API_KEY=DEFAULTS GRAPHRAG_LLM_TYPE=openai_chat GRAPHRAG_LLM_API_BASE=http://127.0.0.1:5081/v1 GRAPHRAG_LLM_MODEL=Hermes-2-Pro-Llama-3-Instruct-Merged-DPO GRAPHRAG_LLM_REQUEST_TIMEOUT=700 GRAPHRAG_LLM_MODEL_SUPPORTS_JSON=True GRAPHRAG_LLM_THREAD_COUNT=16 GRAPHRAG_LLM_CONCURRENT_REQUESTS=16 GRAPHRAG_EMBEDDING_TYPE=openai_embedding GRAPHRAG_EMBEDDING_API_BASE=http://127.0.0.1:9997/v1 GRAPHRAG_EMBEDDING_MODEL=bce-embedding-base_v1 GRAPHRAG_EMBEDDING_BATCH_SIZE=64 GRAPHRAG_EMBEDDING_BATCH_MAX_TOKENS=512 GRAPHRAG_EMBEDDING_THREAD_COUNT=16 GRAPHRAG_EMBEDDING_CONCURRENT_REQUESTS=16 GRAPHRAG_INPUT_FILE_PATTERN=".*.txt$" '''

vamshi-rvk commented 15 hours ago

this worked for me

https://github.com/TheAiSingularity/graphrag-local-ollama