search

kakaotech-bootcamp-11 / kakaotech-bootcamp-11-ktb-11-project-1-chatbot-nlp-server

0 stars 0 forks source link

[Error] Rate limit error #57

Open jieun-lim opened 2 months ago

jieun-lim commented 2 months ago

Problem

gpt-4o-mini 모델의 tokens per min (TPM) 초과 (Limit 200000, Used 199072, Requested 2415)

에러 메시지


openai.RateLimitError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o-mini in organization org-on tokens per min (TPM): Limit 200000, Used 199072, Requested 2415. Please try again in 446ms. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}
2024-09-23 10:53:47 - ERROR - Error on request:
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/werkzeug/serving.py", line 363, in run_wsgi
    execute(self.server.app)
  File "/usr/local/lib/python3.12/site-packages/werkzeug/serving.py", line 326, in execute
    for data in application_iter:
                ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/werkzeug/wsgi.py", line 256, in __next__
    return self._next()
           ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/werkzeug/wrappers/response.py", line 32, in _iter_encoded
    for item in iterable:
                ^^^^^^^^^
  File "/app/utils.py", line 27, in stream_message
    for char in text:
  File "/app/app.py", line 72, in generate_response_stream
    for chunk in retriever.stream(user_input):  # stream을 사용하여 스트리밍 처리
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 3261, in stream
    yield from self.transform(iter([input]), config, **kwargs)
  File "/usr/local/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 3248, in transform
    yield from self._transform_stream_with_config(
  File "/usr/local/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 2054, in _transform_stream_with_config
    chunk: Output = context.run(next, iterator)  # type: ignore
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 3211, in _transform
    yield from final_pipeline
  File "/usr/local/lib/python3.12/site-packages/langchain_core/output_parsers/transform.py", line 65, in transform
    yield from self._transform_stream_with_config(
  File "/usr/local/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 2018, in _transform_stream_with_config
    final_input: Optional[Input] = next(input_for_tracing, None)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 1290, in transform
    yield from self.stream(final, config, **kwargs)
  File "/usr/local/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py", line 410, in stream
    raise e
  File "/usr/local/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py", line 390, in stream
    for chunk in self._stream(messages, stop=stop, **kwargs):
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/langchain_openai/chat_models/base.py", line 558, in _stream
    response = self.client.create(**payload)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/openai/_utils/_utils.py", line 274, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/openai/resources/chat/completions.py", line 668, in create
    return self._post(
           ^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/openai/_base_client.py", line 1260, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/openai/_base_client.py", line 937, in request
    return self._request(
           ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/openai/_base_client.py", line 1026, in _request
    return self._retry_request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/openai/_base_client.py", line 1075, in _retry_request
    return self._request(
           ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/openai/_base_client.py", line 1026, in _request
    return self._retry_request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/openai/_base_client.py", line 1075, in _retry_request
    return self._request(
           ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/openai/_base_client.py", line 1041, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.RateLimitError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o-mini in organization on tokens per min (TPM): Limit 200000, Used 198811, Requested 2095. Please try again in 271ms. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}```
jieun-lim commented 2 months ago

해결 방안

RAG 출력 토큰 150으로 제한함

document_retriever.py

llm = ChatOpenAI(model_name="gpt-4o-mini", temperature=0.2, streaming=True, max_tokens = 150)