[Bug] - Githubissues

hitzhu commented 2 months ago

Checklist

[X] 1. I have searched related issues but cannot get the expected help.
[X] 2. The bug has not been fixed in the latest version.
[X] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

Task exception was never retrieved future: <Task finished name='Task-4' coro=<AsyncEngine.stop_session() done, defined at /root/.local/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py:312> exception=AttributeError("'numpy.ndarray' object has no attribute 'dlpack'")> Traceback (most recent call last): File "/root/.local/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 315, in stop_session await self.id2generator[str(session_id)].async_cancel(session_id) File "/root/.local/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 515, in async_cancel self.cancel(session_id) File "/root/.local/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 505, in cancel for outputs in stop_generator.stream_infer(session_id, File "/root/.local/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 823, in stream_infer tm_inputs = _np_dict_to_tm_dict(inputs) File "/root/.local/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 51, in _np_dict_to_tm_dict ret[k] = _tm.from_dlpack(v) AttributeError: 'numpy.ndarray' object has no attribute 'dlpack' Traceback (most recent call last): File "/checkpoint/binary/train_package/./test.py", line 19, in res=pipe((question, load_image(url))) File "/root/.local/lib/python3.10/site-packages/lmdeploy/serve/vl_async_engine.py", line 123, in call return super().call(prompts, kwargs) File "/root/.local/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 305, in call return self.batch_infer(prompts, File "/root/.local/lib/python3.10/site-packages/lmdeploy/serve/vl_async_engine.py", line 109, in batch_infer return super().batch_infer(prompts, kwargs) File "/root/.local/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 429, in batch_infer _get_event_loop().run_until_complete(gather()) File "/opt/conda/envs/python3.10.13/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete return future.result() File "/root/.local/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 426, in gather await asyncio.gather( File "/root/.local/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 411, in _inner_call async for out in generator: File "/root/.local/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 620, in generate async for outputs in generator.async_stream_infer( File "/root/.local/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 709, in async_stream_infer tm_inputs = _np_dict_to_tm_dict(inputs) File "/root/.local/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 51, in _np_dict_to_tm_dict ret[k] = _tm.from_dlpack(v) AttributeError: 'numpy.ndarray' object has no attribute 'dlpack'

Reproduction

from lmdeploy import pipeline, TurbomindEngineConfig, GenerationConfig, ChatTemplateConfig, VisionConfig from lmdeploy.vl import load_image import os os.environ["CUDA_LAUNCH_BLOCKING"] = "1"

model_path='/data/oss_bucket_0/jihuai/pretrained/InternVL2-40B'

system_prompt = '我是书生·万象，英文名是InternVL，是由上海人工智能实验室及多家合作单位联合开发的多模态大语言模型。' chat_template_config = ChatTemplateConfig('internvl-zh-hermes2') chat_template_config.meta_instruction = system_prompt pipe = pipeline(model_path, chat_template_config=chat_template_config, backend_config=TurbomindEngineConfig(tp=2, session_len=16384),vision_config=VisionConfig(thread_safe=True))

question='请描述一下这张图片,请特别关注图片中商品的品牌和货号信息,其他信息可以忽略' url='https://img.alicdn.com/imgextra/i3/683416656/O1CN01ZYyInb1z2V57OFAPK_!!683416656.jpg' res=pipe((question, load_image(url))) print(res)

Environment

a100*2 lmd==0.5.1

Error traceback

Task exception was never retrieved
future: <Task finished name='Task-4' coro=<AsyncEngine.stop_session() done, defined at /root/.local/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py:312> exception=AttributeError("'numpy.ndarray' object has no attribute '__dlpack__'")>
Traceback (most recent call last):
  File "/root/.local/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 315, in stop_session
    await self.id2generator[str(session_id)].async_cancel(session_id)
  File "/root/.local/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 515, in async_cancel
    self.cancel(session_id)
  File "/root/.local/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 505, in cancel
    for outputs in stop_generator.stream_infer(session_id,
  File "/root/.local/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 823, in stream_infer
    tm_inputs = _np_dict_to_tm_dict(inputs)
  File "/root/.local/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 51, in _np_dict_to_tm_dict
    ret[k] = _tm.from_dlpack(v)
AttributeError: 'numpy.ndarray' object has no attribute '__dlpack__'
Traceback (most recent call last):
  File "/checkpoint/binary/train_package/./test.py", line 19, in <module>
    res=pipe((question, load_image(url)))
  File "/root/.local/lib/python3.10/site-packages/lmdeploy/serve/vl_async_engine.py", line 123, in __call__
    return super().__call__(prompts, **kwargs)
  File "/root/.local/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 305, in __call__
    return self.batch_infer(prompts,
  File "/root/.local/lib/python3.10/site-packages/lmdeploy/serve/vl_async_engine.py", line 109, in batch_infer
    return super().batch_infer(prompts, **kwargs)
  File "/root/.local/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 429, in batch_infer
    _get_event_loop().run_until_complete(gather())
  File "/opt/conda/envs/python3.10.13/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/root/.local/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 426, in gather
    await asyncio.gather(
  File "/root/.local/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 411, in _inner_call
    async for out in generator:
  File "/root/.local/lib/python3.10/site-packages/lmdeploy/serve/async_engine.py", line 620, in generate
    async for outputs in generator.async_stream_infer(
  File "/root/.local/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 709, in async_stream_infer
    tm_inputs = _np_dict_to_tm_dict(inputs)
  File "/root/.local/lib/python3.10/site-packages/lmdeploy/turbomind/turbomind.py", line 51, in _np_dict_to_tm_dict
    ret[k] = _tm.from_dlpack(v)
AttributeError: 'numpy.ndarray' object has no attribute '__dlpack__'

hitzhu commented 2 months ago

cc

JohnYan98 commented 2 months ago

我也遇到了类似的问题，请问是如何解决的呢？ @hitzhu

colorfulandcjy0806 commented 2 months ago

pip install numpy==1.24.4 我通过这个命令来解决，希望可以给大家做参考。

InternLM / lmdeploy

[Bug] #2270

Checklist

Describe the bug

Reproduction

Environment

Error traceback