UnboundLocalError: local variable 'result' referenced before assignment

[vedtam]

Hi,

I'm often getting this error when using the library:

2024-06-12T12:41:03.049891604Z     return articleController.analyse(request.json)
2024-06-12T12:41:03.049895461Z   File "/home/site/wwwroot/app/controllers/article_controller.py", line 92, in analyse
2024-06-12T12:41:03.049899479Z     relevancy_result = self.llmRelevancyCheck(article)
2024-06-12T12:41:03.049903356Z   File "/home/site/wwwroot/app/controllers/article_controller.py", line 142, in llmRelevancyCheck
2024-06-12T12:41:03.049907323Z     data = self.llamaChat(system_prompt, user_prompt)
2024-06-12T12:41:03.049911150Z   File "/home/site/wwwroot/app/controllers/article_controller.py", line 127, in llamaChat
2024-06-12T12:41:03.049915118Z     response = client.chat.completions.create(
2024-06-12T12:41:03.049918925Z   File "/home/site/wwwroot/antenv/lib/python3.10/site-packages/together/resources/chat/completions.py", line 136, in create
2024-06-12T12:41:03.049922892Z     response, _, _ = requestor.request(
2024-06-12T12:41:03.049926700Z   File "/home/site/wwwroot/antenv/lib/python3.10/site-packages/together/abstract/api_requestor.py", line 242, in request
2024-06-12T12:41:03.049935837Z     result = self.request_raw(
2024-06-12T12:41:03.049939814Z   File "/home/site/wwwroot/antenv/lib/python3.10/site-packages/together/abstract/api_requestor.py", line 520, in request_raw
2024-06-12T12:41:03.049943862Z     response_headers=dict(result.headers),
2024-06-12T12:41:03.049947709Z UnboundLocalError: local variable 'result' referenced before assignment

The relevant function: "llamaChat" is:

  def llamaChat(self, system_prompt: str, user_prompt: str)-> Optional[Dict]:
    messages = [{
      'role': 'system',
      'content': system_prompt
      },
      {
      'role': 'user',
      'content': user_prompt
    }]

    client = Together(api_key='xxx')
    response = client.chat.completions.create(
        model='meta-llama/Llama-3-8b-chat-hf',
        messages=messages,
    )
    content = response.choices[0].message.content

    try:
      return json.loads(content)
    except Exception as e:
      sentry_sdk.capture_message(f'Error parsing JSON in answer! Content: {content}, Exception: {e}')
      return None

Would really appreciate your feedback, thanks!

[cgrx]

@vedtam Were you able identify any solution ? I am getting the same error.

Edit : My stop gap solution is to use openai-python. I am able to get inferences without any issue.

[vedtam]

@cgrx well it seems like a basic error in the library. However, it doesn't happen every time, only at random intervals. Will have to see what the devs will have to say about it.

[jdf-prog]

Also encounter this problem

[M-Ali-ML]

I'm facing this issue with every call, I can't use together AI at all