search

yisol / IDM-VTON

[ECCV2024] IDM-VTON : Improving Diffusion Models for Authentic Virtual Try-on in the Wild
https://idm-vton.github.io/
4.01k stars 629 forks source link

Load model from C:\tut\IDM-VTON\ckpt/humanparsing/parsing_atr.onnx failed:Protobuf parsing failed #42

Closed nitinmukesh closed 7 months ago

nitinmukesh commented 7 months ago

Any suggestions please.

I noticed it is only using RAM and not VRAM, not sure why?

(venv) C:\tut\IDM-VTON>python gradio_demo/app.py
The config attributes {'decay': 0.9999, 'inv_gamma': 1.0, 'min_decay': 0.0, 'optimization_step': 37000, 'power': 0.6666666666666666, 'update_after_step': 0, 'use_ema_warmup': False} were passed to UNet2DConditionModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Some weights of the model checkpoint were not used when initializing UNet2DConditionModel:
 ['add_embedding.linear_1.bias, add_embedding.linear_1.weight, add_embedding.linear_2.bias, add_embedding.linear_2.weight']
Traceback (most recent call last):
  File "C:\tut\IDM-VTON\gradio_demo\app.py", line 95, in <module>
    parsing_model = Parsing(0)
  File "C:\tut\IDM-VTON\.\preprocess\humanparsing\run_parsing.py", line 20, in __init__
    self.session = ort.InferenceSession(os.path.join(Path(__file__).absolute().parents[2].absolute(), 'ckpt/humanparsing/parsing_atr.onnx'),
  File "C:\tut\IDM-VTON\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 419, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "C:\tut\IDM-VTON\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 452, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidProtobuf: [ONNXRuntimeError] : 7 : INVALID_PROTOBUF : Load model from C:\tut\IDM-VTON\ckpt/humanparsing/parsing_atr.onnx failed:Protobuf parsing failed.

(venv) C:\tut\IDM-VTON>pip list

Package                   Version
------------------------- ------------------
absl-py                   2.1.0
accelerate                0.25.0
addict                    2.4.0
aiofiles                  23.2.1
altair                    5.3.0
annotated-types           0.6.0
antlr4-python3-runtime    4.9.3
anyio                     4.3.0
attrs                     23.2.0
av                        12.0.0
basicsr                   1.4.2
bitsandbytes              0.39.0
certifi                   2024.2.2
charset-normalizer        3.3.2
click                     8.1.7
cloudpickle               3.0.0
colorama                  0.4.6
coloredlogs               15.0.1
contourpy                 1.2.1
cycler                    0.12.1
diffusers                 0.25.0
einops                    0.7.0
exceptiongroup            1.2.1
fastapi                   0.110.3
ffmpy                     0.3.2
filelock                  3.14.0
flatbuffers               24.3.25
fonttools                 4.51.0
fsspec                    2024.3.1
future                    1.0.0
fvcore                    0.1.5.post20221221
gradio                    4.24.0
gradio_client             0.14.0
grpcio                    1.63.0
h11                       0.14.0
httpcore                  1.0.5
httpx                     0.27.0
huggingface-hub           0.22.2
humanfriendly             10.0
idna                      3.7
imageio                   2.34.1
importlib_metadata        7.1.0
importlib_resources       6.4.0
iopath                    0.1.10
Jinja2                    3.1.3
jsonschema                4.22.0
jsonschema-specifications 2023.12.1
kiwisolver                1.4.5
lazy_loader               0.4
lightning-utilities       0.11.2
lmdb                      1.4.1
Markdown                  3.6
markdown-it-py            3.0.0
MarkupSafe                2.1.5
matplotlib                3.8.4
mdurl                     0.1.2
mpmath                    1.3.0
networkx                  3.3
numpy                     1.26.4
omegaconf                 2.3.0
onnxruntime               1.16.2
opencv-python             4.9.0.80
orjson                    3.10.2
packaging                 24.0
pandas                    2.2.2
pillow                    10.3.0
pip                       24.0
platformdirs              4.2.1
portalocker               2.8.2
protobuf                  4.25.3
psutil                    5.9.8
pycocotools               2.0.7
pydantic                  2.7.1
pydantic_core             2.18.2
pydub                     0.25.1
Pygments                  2.17.2
pyparsing                 3.1.2
pyreadline3               3.4.1
python-dateutil           2.9.0.post0
python-multipart          0.0.9
pytorch-triton            0.0.1
pytz                      2024.1
pywin32                   306
PyYAML                    6.0.1
referencing               0.35.0
regex                     2024.4.28
requests                  2.31.0
rich                      13.7.1
rpds-py                   0.18.0
ruff                      0.4.2
safetensors               0.4.3
scikit-image              0.23.2
scipy                     1.11.1
semantic-version          2.10.0
setuptools                63.2.0
shellingham               1.5.4
six                       1.16.0
sniffio                   1.3.1
starlette                 0.37.2
sympy                     1.12
tabulate                  0.9.0
tb-nightly                2.17.0a20240501
tensorboard-data-server   0.7.2
termcolor                 2.4.0
tifffile                  2024.4.24
tokenizers                0.15.2
tomli                     2.0.1
tomlkit                   0.12.0
toolz                     0.12.1
torch                     2.0.1+cu118
torchaudio                2.0.2+cu118
torchmetrics              1.2.1
torchvision               0.15.2+cu118
tqdm                      4.66.1
transformers              4.36.2
typer                     0.12.3
typing_extensions         4.11.0
tzdata                    2024.1
urllib3                   2.2.1
uvicorn                   0.29.0
websockets                11.0.3
Werkzeug                  3.0.2
yacs                      0.1.8
yapf                      0.40.2
zipp                      3.18.1
nitinmukesh commented 7 months ago

Sorry, needed to download the models

luckierya commented 7 months ago

您好，我也遇到这个相同问题了，想问下在哪里下载模型呀？

XinPeiHou commented 6 months ago

您好，我也遇到相同的问题，请问您是咋样解决的啊？

ipfans commented 5 months ago

It looks like the sha1 checksum is not the same

samsara-ku commented 2 months ago

For who has same problem; download all the models from OOTDiffusion.

added) densepose model came from vid2densepose

NatanFreeman commented 2 months ago

@samsara-ku I downloaded the models from the location you've provided but still get the same error:

(idm) root@ubuntu:/home/IDM-VTON# python gradio_demo/app.py
/root/miniconda3/envs/idm/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
The config attributes {'decay': 0.9999, 'inv_gamma': 1.0, 'min_decay': 0.0, 'optimization_step': 37000, 'power': 0.6666666666666666, 'update_after_step': 0, 'use_ema_warmup': False} were passed to UNet2DConditionModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Some weights of the model checkpoint were not used when initializing UNet2DConditionModel:
 ['add_embedding.linear_1.bias, add_embedding.linear_1.weight, add_embedding.linear_2.bias, add_embedding.linear_2.weight']
Traceback (most recent call last):
  File "/home/IDM-VTON/gradio_demo/app.py", line 95, in <module>
    parsing_model = Parsing(0)
  File "/home/IDM-VTON/./preprocess/humanparsing/run_parsing.py", line 20, in __init__
    self.session = ort.InferenceSession(os.path.join(Path(__file__).absolute().parents[2].absolute(), 'ckpt/humanparsing/parsing_atr.onnx'),
  File "/root/miniconda3/envs/idm/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 419, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/root/miniconda3/envs/idm/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 452, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidProtobuf: [ONNXRuntimeError] : 7 : INVALID_PROTOBUF : Load model from /home/IDM-VTON/ckpt/humanparsing/parsing_atr.onnx failed:Protobuf parsing failed.

Here is the file tree at /ckpt:

.
├── densepose
│   └── model_final_162be9.pkl
├── humanparsing
│   ├── exp-schp-201908261155-lip.pth
│   ├── exp-schp-201908301523-atr.pth
│   ├── parsing_atr.onnx
│   └── parsing_lip.onnx
├── image_encoder
├── ip_adapter
└── openpose
    └── ckpts
        └── body_pose_model.pth

And here are the sha256 sums:

02aead67b765229259f8309e19c45e7e3994bc36122001032c12160e0be7d43a  ./humanparsing/exp-schp-201908301523-atr.pth
49b2acae17dd1fffcfe4c8c7689b0cfd7ae7d73460219da68005f0d0807b6499  ./humanparsing/exp-schp-201908261155-lip.pth
b8a7382001b16e453bad95ca9dbc68ae8f2b839b304cf90eaf5c27fbdb4dae91  ./densepose/model_final_162be9.pkl
d14e9b77d55c87dfdb4c3ea0d6fc05be2d9defc49030dc86aded06abcb855653  ./openpose/ckpts/body_pose_model.pth
e953475b1378e1d0566f8ad8de20077ce8610ae23fb2b5f8bfe57104aca8e911  ./openpose/.DS_Store
ee89ec53434fcf4c7cb189ff1588f391b8958cda75090e29b330a9e1411be89c  ./humanparsing/parsing_atr.onnx
fb71093030b61205347b712f82ace42ede5cd4a006a737fe92f0f6b06e23984d  ./humanparsing/parsing_lip.onnx
samsara-ku commented 2 months ago

@NatanFreeman Sorry for late replying due to our national holiday.

First, my current working directory is little bit different with you. (some files are missing)

.
├── densepose
│   └── model_final_162be9.pkl
├── humanparsing
│   ├── parsing_atr.onnx
│   └── parsing_lip.onnx
├── image_encoder
├── ip_adapter
└── openpose
    └── ckpts
        └── body_pose_model.pth

Here is my sha256 sums:

b8a7382001b16e453bad95ca9dbc68ae8f2b839b304cf90eaf5c27fbdb4dae91  ./model_final_162be9.pkl
25a948c16078b0f08e236bda51a385d855ef4c153598947c28c0d47ed94bb746  ./body_pose_model.pth
04c7d1d070d0e0ae943d86b18cb5aaaea9e278d97462e9cfb270cbbe4cd977f4  ./parsing_atr.onnx
8436e1dae96e2601c373d1ace29c8f0978b16357d9038c17a8ba756cca376dbc  ./parsing_lip.onnx

It is clearly different with your and my body_pose_model.pth, parsing_atr.onnx, and parsing_lip.onnx sha256sum.

ADDED) I tried to re-download all the checkpoints in the above link to check about sha256sum, and I cannot find any problem with that. Could you try again?

NatanFreeman commented 2 months ago

I tried again and got the same error:

/root/miniconda3/envs/idm/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
The config attributes {'decay': 0.9999, 'inv_gamma': 1.0, 'min_decay': 0.0, 'optimization_step': 37000, 'power': 0.6666666666666666, 'update_after_step': 0, 'use_ema_warmup': False} were passed to UNet2DConditionModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Some weights of the model checkpoint were not used when initializing UNet2DConditionModel:
 ['add_embedding.linear_1.bias, add_embedding.linear_1.weight, add_embedding.linear_2.bias, add_embedding.linear_2.weight']
Traceback (most recent call last):
  File "/home/IDM-VTON/gradio_demo/app.py", line 95, in <module>
    parsing_model = Parsing(0)
  File "/home/IDM-VTON/./preprocess/humanparsing/run_parsing.py", line 20, in __init__
    self.session = ort.InferenceSession(os.path.join(Path(__file__).absolute().parents[2].absolute(), 'ckpt/humanparsing/parsing_atr.onnx'),
  File "/root/miniconda3/envs/idm/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 419, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/root/miniconda3/envs/idm/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 452, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidProtobuf: [ONNXRuntimeError] : 7 : INVALID_PROTOBUF : Load model from /home/IDM-VTON/ckpt/humanparsing/parsing_atr.onnx failed:Protobuf parsing failed.

I also tried removing the extra files but it made no difference. I am running this on Ubuntu WSL2 and I have an NVIDIA GPU. Are there any log files or additional information which may be usefull?

samsara-ku commented 2 months ago

I'm already told you that there is difference between sha256sum your and mine. Did you download files from below link, right?

parsing_atr.onnx: https://huggingface.co/levihsu/OOTDiffusion/resolve/main/checkpoints/humanparsing/parsing_atr.onnx?download=true parsing_lip.onnx: https://huggingface.co/levihsu/OOTDiffusion/resolve/main/checkpoints/humanparsing/parsing_lip.onnx?download=true body_pose_model.pth: https://huggingface.co/levihsu/OOTDiffusion/resolve/main/checkpoints/openpose/ckpts/body_pose_model.pth?download=true

NatanFreeman commented 1 month ago

@samsara-ku downloading the weights from those links fixed that issue. Thank you for your patience!