search

datadreamer-dev / DataDreamer

DataDreamer: Prompt. Generate Synthetic Data. Train & Align Models.   🤖💤
https://datadreamer.dev
MIT License
724 stars 39 forks source link

Together 524 Server Error Exception #9

Closed preemware closed 4 months ago

preemware commented 4 months ago

Uncaught exception when using Together API. I'm using the model Phind/Phind-CodeLlama-34B-v2. 

   return self(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/root/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 379, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 314, in iter
    return fut.result()
           ^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/root/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/root/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 289, in wrapped_f
    return self(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/root/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 379, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 314, in iter
    return fut.result()
           ^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/root/venv/lib/python3.11/site-packages/tenacity/__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/root/venv/lib/python3.11/site-packages/datadreamer/llms/together.py", line 116, in _retry_wrapper
    return func(**kwargs)
           ^^^^^^^^^^^^^^
  File "/root/venv/lib/python3.11/site-packages/together/complete.py", line 48, in create
    response = create_post_request(
               ^^^^^^^^^^^^^^^^^^^^
  File "/root/venv/lib/python3.11/site-packages/together/utils.py", line 119, in create_post_request
    response_status_exception(response)
  File "/root/venv/lib/python3.11/site-packages/together/utils.py", line 87, in response_status_exception
    response.raise_for_status()
  File "/root/venv/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 524 Server Error:  for url: https://api.together.xyz/api/inference
AjayP13 commented 4 months ago

Patched here on version 0.22.0 to retry on this exception: https://github.com/datadreamer-dev/DataDreamer/commit/6829c6dd630be3ddb22f82e49d8cd523b5356542