search

hwchase17 / langchain-hub

3.26k stars 267 forks source link

CSV Agent - OpenAI Context limit #26

Open infa-bhbalaji opened 1 year ago

infa-bhbalaji commented 1 year ago

I am trying to load a large CSV with create_csv_agent function. I get the error " This model's maximum context length is 4097 tokens, however you requested 6595 tokens" when I do agent.run() for some commands. I tried looking for ways to use document splitter here. But it is not clear from the documentation about how to integrate a split document with agents directly.

Adorp94 commented 1 year ago

I'm running through the same issue. What's odd for me is that I was able to run it some weeks ago without this issue. Have found the way to solve it? Or have you applied an strategy that worked (i.e. map reduce, refining, etc).

Thanks. I will be working on how to solve it. If I come with an asnwer I'll respond here.

TMoneyBidness commented 1 year ago

I have the same issue. It's infuriating. Sometimes it works, others it doesn't, I have a CSV with 100 rows.