search

eth-sri / lmql

A language for constraint-guided and efficient LLM programming.
https://lmql.ai
Apache License 2.0
3.48k stars 191 forks source link

python AttributeError: 'NoneType' object has no attribute 'get_event_loop', lmql == 0.7.3 #349

Open TioeAre opened 2 months ago

TioeAre commented 2 months ago

I'm trying a simple test in https://towardsdatascience.com/lmql-sql-for-language-models-d7486d88c541, and code like this

import lmql

query_string = """
"Q: What is the sentiment of the following review: ```The food was very good.```?n"
"A: [SENTIMENT]" where (SENTIMENT in ['positive', 'negative', 'neutral'])
"""
print(lmql.run_sync(
    query_string,
    model=lmql.model("local:llama.cpp:/home/lmql/llama.cpp/models/zephyr-7b-beta.Q4_K_M.gguf",
                     tokenizer='/home/zephyr-7b-beta', cuda=True)).variables['SENTIMENT'])

however there comes error,

Exception ignored in: <function lmtp_model.__del__ at 0x7fd63813c680>
Traceback (most recent call last):
  File "/home/miniconda3/envs/lmql/lib/python3.11/site-packages/lmql/models/lmtp/lmtp_dcmodel.py", line 532, in __del__
AttributeError: 'NoneType' object has no attribute 'get_event_loop'

my environment is ubuntu20.04, python 3.11.8

llama_cpp_python              0.2.61
lmql                          0.7.3
MarkupSafe                    2.1.3
message-filters               1.16.0
mpmath                        1.3.0
multidict                     6.0.5
networkx                      3.2.1
numpy                         1.24.4
nvidia-cublas-cu11            11.11.3.6
nvidia-cublas-cu12            12.1.3.1
nvidia-cuda-cupti-cu11        11.8.87
nvidia-cuda-cupti-cu12        12.1.105
nvidia-cuda-nvrtc-cu11        11.8.89
nvidia-cuda-nvrtc-cu12        12.1.105
nvidia-cuda-runtime-cu11      11.8.89
nvidia-cuda-runtime-cu12      12.1.105
nvidia-cudnn-cu11             8.7.0.84
nvidia-cudnn-cu12             8.9.2.26
nvidia-cufft-cu11             10.9.0.58
nvidia-cufft-cu12             11.0.2.54
nvidia-curand-cu11            10.3.0.86
nvidia-curand-cu12            10.3.2.106
nvidia-cusolver-cu11          11.4.1.48
nvidia-cusolver-cu12          11.4.5.107
nvidia-cusparse-cu11          11.7.5.86
nvidia-cusparse-cu12          12.1.0.106
nvidia-nccl-cu11              2.19.3
nvidia-nccl-cu12              2.18.1
nvidia-nvjitlink-cu12         12.4.127
nvidia-nvtx-cu11              11.8.86
nvidia-nvtx-cu12              12.1.105
openai                        1.19.0
packaging                     24.0
pillow                        10.2.0
pip                           23.3.2
torch                         2.1.2
torchaudio                    2.2.2+cu118
torchvision                   0.17.2+cu118
tqdm                          4.66.2
transformers                  4.39.3
triton                        2.1.0
typing_extensions             4.8.0
urllib3                       2.2.1
wheel                         0.42.0
xacro                         1.14.17
yarl                          1.9.4

I found it seems like eth-sri/lmql#6, looking forward to your reply.

ag2435 commented 2 months ago

I got the same error message when I tried running the following query via lmql run:

argmax 
    "Say 'this is a test':[RESPONSE] \n"
from
    lmql.model("local:meta-llama/Meta-Llama-3-8B-Instruct", cuda=True)
where
    len(RESPONSE) < 50

And the error message:

Say 'this is a test': The art of the emergency broadcast system
The sE 

 valid=True, final=var
Exception ignored in: <function lmtp_model.__del__ at 0x7f8db3d4d9e0>
Traceback (most recent call last):
  File "/home/ag2435/.conda/envs/qasper-gpu/lib/python3.12/site-packages/lmql/models/lmtp/lmtp_dcmodel.py", line 532, in __del__
AttributeError: 'NoneType' object has no attribute 'get_event_loop'