search

lxtGH / OMG-Seg

OMG-LLaVA and OMG-Seg codebase [CVPR-24 and NeurIPS-24]
Other
1.28k stars 48 forks source link

Inference error #28

Closed echo840 closed 2 months ago

echo840 commented 3 months ago

When I test the model I trained on the app, I encountered the following error. How can I solve this? However in the training stage, there is no error.

Traceback (most recent call last):
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/gradio/queueing.py", line 541, in process_events
    response = await route_utils.call_process_api(
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/gradio/route_utils.py", line 276, in call_process_api
    output = await app.get_blocks().process_api(
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/gradio/blocks.py", line 1928, in process_api
    result = await self.call_function(
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/gradio/blocks.py", line 1514, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/gradio/utils.py", line 833, in wrapper
    response = f(*args, **kwargs)
  File "/home/kas/lz_new/ref_monkey/OMG-Seg/omg_llava/omg_llava/tools/app.py", line 309, in inference
    generate_output = llm.generate(
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/peft/peft_model.py", line 1638, in generate
    outputs = self.base_model.generate(*args, **kwargs)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/transformers/generation/utils.py", line 1764, in generate
    return self.sample(
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/transformers/generation/utils.py", line 2861, in sample
    outputs = self(
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = module._old_forward(*args, **kwargs)
  File "/home/kas/.cache/huggingface/modules/transformers_modules/internlm2-chat-7b/modeling_internlm2.py", line 1047, in forward
    outputs = self.model(
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = module._old_forward(*args, **kwargs)
  File "/home/kas/.cache/huggingface/modules/transformers_modules/internlm2-chat-7b/modeling_internlm2.py", line 932, in forward
    layer_outputs = decoder_layer(
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = module._old_forward(*args, **kwargs)
  File "/home/kas/.cache/huggingface/modules/transformers_modules/internlm2-chat-7b/modeling_internlm2.py", line 639, in forward
    hidden_states, self_attn_weights, present_key_value = self.attention(
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = module._old_forward(*args, **kwargs)
  File "/home/kas/.cache/huggingface/modules/transformers_modules/internlm2-chat-7b/modeling_internlm2.py", line 344, in forward
    qkv_states = self.wqkv(hidden_states)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/peft/tuners/lora/bnb.py", line 467, in forward
    result = self.base_layer(x, *args, **kwargs)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = module._old_forward(*args, **kwargs)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/bitsandbytes/nn/modules.py", line 256, in forward
    out = bnb.matmul_4bit(x, self.weight.t(), bias=bias, quant_state=self.weight.quant_state)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py", line 577, in matmul_4bit
    return MatMul4Bit.apply(A, B, out, bias, quant_state)
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/torch/autograd/function.py", line 539, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "/home/kas/.conda/envs/omgllava/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py", line 516, in forward
    output = torch.nn.functional.linear(A, F.dequantize_4bit(B, quant_state).to(A.dtype).t(), bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (282x4096 and 1x12582912)
lxtGH commented 2 months ago

@echo840 Which command do you use?

echo840 commented 2 months ago

image

I use this command.

zhang-tao-whu commented 2 months ago

Could you provide information about your environment, such as the output of 'pip list'? Also, from which repository did you download internlm2-7b? It's possible that they have been updated, causing a conflict with your version of transformers.

echo840 commented 2 months ago 
absl-py                       2.0.0
accelerate                    0.25.0
addict                        2.4.0
aiofiles                      23.2.1
aiohttp                       3.9.1
aiosignal                     1.3.1
aliyun-python-sdk-core        2.14.0
aliyun-python-sdk-kms         2.16.2
altair                        5.2.0
annotated-types               0.6.0
antlr4-python3-runtime        4.9.3
anyio                         3.7.1
appdirs                       1.4.4
asttokens                     2.4.1
async-timeout                 4.0.3
attrs                         23.2.0
av                            11.0.0
bitsandbytes                  0.42.0
boto3                         1.34.14
botocore                      1.34.14
braceexpand                   0.1.7
cachetools                    5.3.2
certifi                       2023.11.17
cffi                          1.16.0
charset-normalizer            3.3.2
click                         8.1.7
cmake                         3.28.1
colorama                      0.4.6
contourpy                     1.2.0
cpm-kernels                   1.0.11
crcmod                        1.7
cryptography                  42.0.3
cycler                        0.12.1
dashscope                     1.13.6
datasets                      2.16.1
decorator                     5.1.1
decord                        0.6.0
deepspeed                     0.13.5
dill                          0.3.7
distro                        1.9.0
docker-pycreds                0.4.0
einops                        0.7.0
einops-exts                   0.0.4
et-xmlfile                    1.1.0
exceptiongroup                1.2.0
executing                     2.0.1
fastapi                       0.108.0
ffmpy                         0.3.1
filelock                      3.13.1
flash-attn                    2.3.6
fonttools                     4.47.0
frozenlist                    1.4.1
fsspec                        2023.10.0
ftfy                          6.2.0
func-timeout                  4.3.5
fvcore                        0.1.5.post20221221
gitdb                         4.0.11
GitPython                     3.1.40
google-ai-generativelanguage  0.4.0
google-api-core               2.15.0
google-auth                   2.26.1
google-auth-oauthlib          1.2.0
google-generativeai           0.3.2
googleapis-common-protos      1.62.0
gradio                        4.37.2
gradio_client                 1.0.2
gradio_image_prompter         0.1.0
grpcio                        1.60.0
grpcio-status                 1.60.0
h11                           0.14.0
hjson                         3.1.0
httpcore                      1.0.2
httpx                         0.26.0
huggingface-hub               0.20.2
idna                          3.6
imageio                       2.34.2
importlib-metadata            7.0.1
importlib-resources           6.1.1
iopath                        0.1.10
ipdb                          0.13.13
ipython                       8.19.0
jedi                          0.19.1
Jinja2                        3.1.2
jmespath                      0.10.0
jsonschema                    4.20.0
jsonschema-specifications     2023.12.1
kiwisolver                    1.4.5
kornia                        0.7.3
kornia_rs                     0.1.5
lagent                        0.1.2
lazy_loader                   0.4
lit                           17.0.6
Markdown                      3.5.1
markdown-it-py                3.0.0
MarkupSafe                    2.1.3
mat4py                        0.6.0
matplotlib                    3.8.2
matplotlib-inline             0.1.6
mdurl                         0.1.2
mmcv                          2.0.1
mmdet                         3.1.0
mmengine                      0.10.4
mmpretrain                    1.0.1
mmsegmentation                1.1.1
model-index                   0.1.11
modelindex                    0.0.2
modelscope                    1.16.0
mpi4py_mpich                  3.1.5
mpmath                        1.3.0
multidict                     6.0.4
multiprocess                  0.70.15
networkx                      3.2.1
ninja                         1.11.1.1
numpy                         1.26.3
nvidia-cublas-cu11            11.10.3.66
nvidia-cublas-cu12            12.1.3.1
nvidia-cuda-cupti-cu11        11.7.101
nvidia-cuda-cupti-cu12        12.1.105
nvidia-cuda-nvrtc-cu11        11.7.99
nvidia-cuda-nvrtc-cu12        12.1.105
nvidia-cuda-runtime-cu11      11.7.99
nvidia-cuda-runtime-cu12      12.1.105
nvidia-cudnn-cu11             8.5.0.96
nvidia-cudnn-cu12             8.9.2.26
nvidia-cufft-cu11             10.9.0.58
nvidia-cufft-cu12             11.0.2.54
nvidia-curand-cu11            10.2.10.91
nvidia-curand-cu12            10.3.2.106
nvidia-cusolver-cu11          11.4.0.1
nvidia-cusolver-cu12          11.4.5.107
nvidia-cusparse-cu11          11.7.4.91
nvidia-cusparse-cu12          12.1.0.106
nvidia-nccl-cu11              2.14.3
nvidia-nccl-cu12              2.18.1
nvidia-nvjitlink-cu12         12.3.101
nvidia-nvtx-cu11              11.7.91
nvidia-nvtx-cu12              12.1.105
oauthlib                      3.2.2
omegaconf                     2.3.0
openai                        1.3.5
opencv-python                 4.9.0.80
openpyxl                      3.1.2
openxlab                      0.0.34
ordered-set                   4.1.0
orjson                        3.9.10
oss2                          2.17.0
packaging                     23.2
pandas                        2.1.4
parameterized                 0.9.0
parso                         0.8.3
peft                          0.7.1
pexpect                       4.9.0
pillow                        10.2.0
pip                           23.3.1
platformdirs                  4.1.0
portalocker                   2.8.2
prettytable                   3.10.2
progressbar2                  4.3.2
prompt-toolkit                3.0.43
proto-plus                    1.23.0
protobuf                      4.23.4
psutil                        5.9.7
ptyprocess                    0.7.0
pure-eval                     0.2.2
py-cpuinfo                    9.0.0
pyarrow                       14.0.2
pyarrow-hotfix                0.6
pyasn1                        0.5.1
pyasn1-modules                0.3.0
pycocoevalcap                 1.2
pycocotools                   2.0.7
pycparser                     2.21
pycryptodome                  3.20.0
pydantic                      2.5.3
pydantic_core                 2.14.6
pydub                         0.25.1
Pygments                      2.17.2
pynvml                        11.5.0
pyparsing                     3.1.1
python-dateutil               2.8.2
python-multipart              0.0.9
python-utils                  3.8.1
pytorchvideo                  0.1.5
pytz                          2023.3.post1
PyYAML                        6.0.1
referencing                   0.32.1
regex                         2023.12.25
requests                      2.32.3
requests-oauthlib             1.3.1
rich                          13.4.2
rpds-py                       0.16.2
rsa                           4.9
ruff                          0.5.2
s3transfer                    0.10.0
safetensors                   0.4.1
scikit-image                  0.24.0
scipy                         1.11.4
seaborn                       0.13.1
semantic-version              2.10.0
sentencepiece                 0.1.99
sentry-sdk                    1.39.1
setproctitle                  1.3.3
setuptools                    60.2.0
shapely                       2.0.5
shellingham                   1.5.4
six                           1.16.0
smmap                         5.0.1
sniffio                       1.3.0
stack-data                    0.6.3
starlette                     0.32.0.post1
sty                           1.0.6
SwissArmyTransformer          0.4.9
sympy                         1.12
tabulate                      0.9.0
tensorboard                   2.15.1
tensorboard-data-server       0.7.2
tensorboardX                  2.6.2.2
termcolor                     2.4.0
terminaltables                3.1.10
tifffile                      2024.7.2
tiktoken                      0.5.2
timeout-decorator             0.5.0
timm                          0.9.12
tokenizers                    0.15.2
tomli                         2.0.1
tomlkit                       0.12.0
toolz                         0.12.0
torch                         2.1.2+cu118
torchaudio                    2.1.2+cu118
torchvision                   0.16.2+cu118
tqdm                          4.65.2
traitlets                     5.14.1
transformers                  4.36.0
transformers-stream-generator 0.0.4
triton                        2.1.0
typer                         0.12.3
typing_extensions             4.9.0
tzdata                        2023.4
urllib3                       2.2.2
uvicorn                       0.25.0
visual-genome                 1.1.1
vlmeval                       0.1.0 
wandb                         0.16.1
wcwidth                       0.2.13
webdataset                    0.2.86
websockets                    11.0.3
Werkzeug                      3.0.1
wheel                         0.41.2
xformers                      0.0.23.post1
XlsxWriter                    3.1.9
xtuner                        0.1.21 
xxhash                        3.4.1
yacs                          0.1.8
yapf                          0.40.2
yarl                          1.9.4
zipp                          3.17.0

Here is my pip list and I download internlm2 from "internlm/internlm2-chat-7b".

zhang-tao-whu commented 2 months ago

Hi, you can try reload the modeling_internlm2.py from here and put it into "internlm/internlm2-chat-7b".

zhang-tao-whu commented 2 months ago

This error seems to be caused by the update of the official huggingface repository of the internlm2

echo840 commented 2 months ago

This error seems to be caused by the update of the official huggingface repository of the internlm2

Thank you for your response. Could you please let me know what changes in the internlm2 led to this error? And why doesn't this error occur during training?

lxtGH commented 2 months ago

@echo840 You can earlier commit of internlm2 to use the code and, in particular, use the version in 6. 2024.