search

mistralai / client-python

Python client library for Mistral AI platform
Apache License 2.0
477 stars 103 forks source link

Client exception when working with pytorch. #90

Closed saurabhmahra91 closed 3 months ago

saurabhmahra91 commented 6 months ago 
from dotenv import load_dotenv
import torch
from mistralai.client import MistralClient

print(torch.tensor([1, 2, 3]))

load_dotenv()
mistral = MistralClient()

print("hello world")

The code above would raise an exception. The program is executed successfully but at the end the program meets an exception. I assume that this exception is coming during python's internal memory management.

Here is the output of the above program I get

tensor([1, 2, 3])
hello world
Exception ignored in: <function MistralClient.__del__ at 0x11015cc20>
Traceback (most recent call last):
  File "/Users/saurabhmahra/Experiments/pii_mapper/venv/lib/python3.12/site-packages/mistralai/client.py", line 46, in __del__
  File "/Users/saurabhmahra/Experiments/pii_mapper/venv/lib/python3.12/site-packages/httpx/_client.py", line 1258, in close
  File "/Users/saurabhmahra/Experiments/pii_mapper/venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 240, in close
  File "/Users/saurabhmahra/Experiments/pii_mapper/venv/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 313, in close
  File "/Users/saurabhmahra/Experiments/pii_mapper/venv/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 303, in _close_connections
TypeError: 'NoneType' object is not callable

I can get rid of the issue by just removing the pytorch part of the code.

from dotenv import load_dotenv
# import torch
from mistralai.client import MistralClient

# print(torch.tensor([1, 2, 3]))

load_dotenv()
mistral = MistralClient()

print("hello world")

My understanding of the error is that becuase of pytorch, the httpx client somehow become None and then the error appears because the during the memory management mistral_client object's _client attribute (httpx client) is None.

More info about my system Python version 3.12.3 mistralai==0.1.8 torch==2.3.0 I am running a macbook air M2 2022 - 8GB with macOS version Sonoma 14.4.1

sophiamyang commented 3 months ago

Can't reproduce. Could you try mistralai 1.0.0 and see if you still see the issue? Thanks so much!