zabirauf / AutoGPT.js

Auto-GPT on the browser
https://autogptjs.com
MIT License
1.02k stars 177 forks source link

Summarize Website on long Websites not handled #18

Open PhillipRt opened 1 year ago

PhillipRt commented 1 year ago

Long Websites should be chunked and summaized in parts or at least cut off. Currently I am getting this issue: Website Content Summary:

Error calling API with status code 400 and message "{

"error": {

"message": "This model's maximum context length is 4097 tokens. However, your messages resulted in 45439 tokens. Please reduce the length of the messages.",

"type": "invalid_request_error",

"param": "messages",

"code": "context_length_exceeded"

}

}

shamoons commented 1 year ago

Maybe we can add a chunked summarization?