search

microsoft / graphrag

A modular graph-based Retrieval-Augmented Generation (RAG) system
https://microsoft.github.io/graphrag/
MIT License
16.22k stars 1.52k forks source link

huge openai API usage for very small amount of text for indexing #431

Closed hemangjoshi37a closed 1 month ago

hemangjoshi37a commented 1 month ago

it just used $15 for a very small amount of text to index on openai API. Is there any solution to reduce cost for this ? I was just testing initially and it just wiped out $15 with just one indexing command. I believe this is good way to earn money for Microsoft and openai and rob it's users.

stevetru1 commented 1 month ago

Adding an additional warning here: https://github.com/microsoft/graphrag/pull/439

kdawgwilk commented 1 month ago

You can also use different models to make things cheaper. currently the default is gpt-4-turbo which is more expensive than gpt-4o. You could also use tools like https://ollama.com to run open source models locally and use them

KylinMountain commented 1 month ago

it is due to the token of prompt is too large when extracting entity...you can try to delete some examples.

ngcheeyuan commented 1 month ago

it just used $15 for a very small amount of text to index on openai API. Is there any solution to reduce cost for this ? I was just testing initially and it just wiped out $15 with just one indexing command. I believe this is good way to earn money for Microsoft and openai and rob it's users.

They did run a demo showing that they used $12 indexing a book.

hemangjoshi37a commented 1 month ago

@kdawgwilk how to use it with ollama . any documentation or any tips for this ?

natoverse commented 1 month ago

@kdawgwilk how to use it with ollama . any documentation or any tips for this ?

Lots of conversation going on around ollama, I've been closing issues and pointing to a consolidated issue #657 .

I'm closing this otherwise, given that we've put a warning in that it can be expensive and we strongly recommend starting with a small sample to evaluate the quality and cost before you throw a lot of data at it. You can also run much cheaper models than gpt-4-turbo on OpenAI, as mentioned by @kdawgwilk