gpt token 수 부족

[caboodles]

gpt token 수 부족으로 response 안돌아옴

`/workspace/app/node_modules/openai/error.js:44 return new BadRequestError(status, error, message, headers); ^

BadRequestError: 400 This model's maximum context length is 4097 tokens. However, your messages resulted in 9132 tokens. Please reduce the length of the messages. at APIError.generate (/workspace/app/node_modules/openai/error.js:44:20) at OpenAI.makeStatusError (/workspace/app/node_modules/openai/core.js:255:33) at OpenAI.makeRequest (/workspace/app/node_modules/openai/core.js:294:30) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async /workspace/app/node_modules/@langchain/openai/dist/chat_models.cjs:649:29 at async RetryOperation._fn (/workspace/app/node_modules/p-retry/index.js:50:12) { status: 400, headers: { 'access-control-allow-origin': '*', 'alt-svc': 'h3=":443"; ma=86400', 'cf-cache-status': 'DYNAMIC', 'cf-ray': '84e1aaba69f2edfd-ICN', connection: 'keep-alive', 'content-length': '281', 'content-type': 'application/json', date: 'Wed, 31 Jan 2024 11:38:11 GMT', 'openai-organization': 'user-8ajoec35ib7hi96vqkdktor7', 'openai-processing-ms': '24', 'openai-version': '2020-10-01', server: 'cloudflare', 'set-cookie': '__cf_bm=kNxVjphN92oG5koQyhZv.UKfI4d7vpyqbzn9QL86bfU-1706701091-1-AdrsP8/+Ds72nNnfufoeQydJJov+PB5hyP5mQjbBqaEUzbC7CDeW/xKO35ZBSAPywhEvV7OeTk0KKbXRCZQgWj8=; path=/; expires=Wed, 31-Jan-24 12:08:11 GMT; domain=.api.openai.com; HttpOnly; Secure; SameSite=None, _cfuvid=0v3CnWu5zimAnHbZydxUNf6iaT2oIMsFhp3S9gIPngc-1706701091305-0-604800000; path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None', 'strict-transport-security': 'max-age=15724800; includeSubDomains', 'x-ratelimit-limit-requests': '10000', 'x-ratelimit-limit-tokens': '60000', 'x-ratelimit-remaining-requests': '9999', 'x-ratelimit-remaining-tokens': '54738', 'x-ratelimit-reset-requests': '8.64s', 'x-ratelimit-reset-tokens': '5.262s', 'x-request-id': '921a6b6911e568f7a7919cbea1832160' }, error: { message: "This model's maximum context length is 4097 tokens. However, your messages resulted in 9132 tokens. Please reduce the length of the messages.", type: 'invalid_request_error', param: 'messages', code: 'context_length_exceeded' }, code: 'context_length_exceeded', param: 'messages', type: 'invalid_request_error', attemptNumber: 1, retriesLeft: 6 }`

[caboodles]

prompting의 경우

const model = new ChatOpenAI({
  modelName: "gpt-3.5-turbo-0125",
  apiKey: apiKey,
});

와 같이 token window가 큰 모델을 사용하면 됨

embedding의 경우

openAI에는 max input token이 더 큰 모델은 없는 듯함

• 해결법 1 : openAI가 아닌 다른 embedder 사용
• 해결법 2: model을 시켜서 summerize후 embedding
• 해결법 3: txt파일을 분할 후 저장

[jnnkk]

혹시 이 이슈 언제 해결 가능할까요?