search

mpaepper / content-chatbot

Build a chatbot or Q&A bot of your website's content
https://www.paepper.com/blog/posts/build-q-and-a-bot-of-your-website-using-langchain/
522 stars 56 forks source link

Large scale Costs #7

Closed kaiqueask closed 1 year ago

kaiqueask commented 1 year ago

The cost of this on a large scale ends up being a little high, what do you suggest for these cases?

Because we have a messaging product which ends up increasing the cost of the high number of messages we receive.

Would you have any suggestions?

mpaepper commented 1 year ago

What costs are you referring to?

The embedding costs are quite small and you can store them locally as illustrated, so you only have costs for the requests.

In case you refer to costs for the requests, then I'd recommend to store the questions you get with their embeddings as well and before sending new requests, retrieve the related embedded questions and answers and show them to users first.

pavei commented 1 year ago

Do you think it would be possible to use gtp-turbo.3.5?