search

warmshao / FasterLivePortrait

Bring portraits to life in Real Time！onnx/tensorrt support！实时肖像驱动！
492 stars 47 forks source link

onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running GridSample node. Name:'/dense_motion_network/GridSample' Status Message: Only 4-D tensor is supported #36

Closed linhcentrio closed 2 months ago

linhcentrio commented 2 months ago

(/home/Efficient-Live-Portrait/venv) [root@aiclip-dfl FasterLivePortrait]# python run.py \

--src_image assets/examples/source/s10.jpg \ --dri_video assets/examples/driving/d14.mp4 \ --cfg configs/onnx_infer.yaml INFO:albumentations.check_version:A new version of Albumentations is available: 1.4.11 (you have 1.4.10). Upgrade using: pip install --upgrade albumentations loading model: warping_spade {'name': 'WarpingSpadeModel', 'predict_type': 'ort', 'model_path': './checkpoints/liveportrait_onnx/warping_spade.onnx'} OnnxRuntime use ['CUDAExecutionProvider', 'CoreMLExecutionProvider', 'CPUExecutionProvider'] /home/Efficient-Live-Portrait/venv/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'CoreMLExecutionProvider' is not in available provider names.Available providers: 'CUDAExecutionProvider, CPUExecutionProvider' warnings.warn( ort loading model: motion_extractor {'name': 'MotionExtractorModel', 'predict_type': 'ort', 'model_path': './checkpoints/liveportrait_onnx/motion_extractor.onnx'} OnnxRuntime use ['CUDAExecutionProvider', 'CoreMLExecutionProvider', 'CPUExecutionProvider'] ort loading model: landmark {'name': 'LandmarkModel', 'predict_type': 'ort', 'model_path': './checkpoints/liveportrait_onnx/landmark.onnx'} OnnxRuntime use ['CUDAExecutionProvider', 'CoreMLExecutionProvider', 'CPUExecutionProvider'] ort loading model: face_analysis {'name': 'FaceAnalysisModel', 'predict_type': 'ort', 'model_path': ['./checkpoints/liveportrait_onnx/retinaface_det_static.onnx', './checkpoints/liveportrait_onnx/face_2dpose_106_static.onnx']} OnnxRuntime use ['CUDAExecutionProvider', 'CoreMLExecutionProvider', 'CPUExecutionProvider'] OnnxRuntime use ['CUDAExecutionProvider', 'CoreMLExecutionProvider', 'CPUExecutionProvider'] loading model: app_feat_extractor {'name': 'AppearanceFeatureExtractorModel', 'predict_type': 'ort', 'model_path': './checkpoints/liveportrait_onnx/appearance_feature_extractor.onnx'} OnnxRuntime use ['CUDAExecutionProvider', 'CoreMLExecutionProvider', 'CPUExecutionProvider'] ort loading model: stitching {'name': 'StitchingModel', 'predict_type': 'ort', 'model_path': './checkpoints/liveportrait_onnx/stitching.onnx'} OnnxRuntime use ['CUDAExecutionProvider', 'CoreMLExecutionProvider', 'CPUExecutionProvider'] ort loading model: stitching_eye_retarget {'name': 'StitchingModel', 'predict_type': 'ort', 'model_path': './checkpoints/liveportrait_onnx/stitching_eye.onnx'} OnnxRuntime use ['CUDAExecutionProvider', 'CoreMLExecutionProvider', 'CPUExecutionProvider'] ort loading model: stitching_lip_retarget {'name': 'StitchingModel', 'predict_type': 'ort', 'model_path': './checkpoints/liveportrait_onnx/stitching_lip.onnx'} OnnxRuntime use ['CUDAExecutionProvider', 'CoreMLExecutionProvider', 'CPUExecutionProvider'] ort 2024-07-25 09:11:33.094898950 [E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running GridSample node. Name:'/dense_motion_network/GridSample' Status Message: Only 4-D tensor is supported Traceback (most recent call last): File "/home/FasterLivePortrait/run.py", line 64, in dri_crop, out_crop, out_org = pipe.run(frame, img_src) File "/home/FasterLivePortrait/src/pipelines/faster_live_portrait_pipeline.py", line 329, in run out_crop = self.model_dict["warping_spade"].predict(f_s, x_s, x_d_i_new) File "/home/FasterLivePortrait/src/models/warping_spade_model.py", line 34, in predict preds = self.predictor.predict(feature_3d, kp_driving, kp_source) File "/home/FasterLivePortrait/src/models/predictor.py", line 211, in predict results = self.onnx_model.run(None, input_feeds) File "/home/Efficient-Live-Portrait/venv/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 220, in run return self._sess.run(output_names, input_feed, run_options) onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running GridSample node. Name:'/dense_motion_network/GridSample' Status Message: Only 4-D tensor is supported

linhcentrio commented 2 months ago

(/home/Efficient-Live-Portrait/venv) [root@aiclip-dfl FasterLivePortrait]# conda list

aiofiles 23.2.1 pypi_0 pypi albucore 0.0.12 pypi_0 pypi albumentations 1.4.10 pypi_0 pypi altair 5.3.0 pypi_0 pypi annotated-types 0.7.0 pypi_0 pypi antlr4-python3-runtime 4.9.3 pypi_0 pypi anyio 4.4.0 pypi_0 pypi attrs 23.2.0 pypi_0 pypi blas 1.0 openblas bzip2 1.0.8 h5eee18b_6 ca-certificates 2024.7.4 hbcca054_0 conda-forge certifi 2024.7.4 pypi_0 pypi charset-normalizer 3.3.2 pypi_0 pypi click 8.1.7 pypi_0 pypi cmake 3.30.1 pypi_0 pypi coloredlogs 15.0.1 py310h06a4308_2 contourpy 1.2.1 pypi_0 pypi cuda-cccl 11.8.89 0 nvidia/label/cuda-11.8.0 cuda-command-line-tools 11.8.0 0 nvidia/label/cuda-11.8.0 cuda-compiler 11.8.0 0 nvidia/label/cuda-11.8.0 cuda-cudart 11.8.89 0 nvidia/label/cuda-11.8.0 cuda-cudart-dev 11.8.89 0 nvidia/label/cuda-11.8.0 cuda-cuobjdump 11.8.86 0 nvidia/label/cuda-11.8.0 cuda-cupti 11.8.87 0 nvidia/label/cuda-11.8.0 cuda-cuxxfilt 11.8.86 0 nvidia/label/cuda-11.8.0 cuda-documentation 11.8.86 0 nvidia/label/cuda-11.8.0 cuda-driver-dev 11.8.89 0 nvidia/label/cuda-11.8.0 cuda-gdb 11.8.86 0 nvidia/label/cuda-11.8.0 cuda-libraries 11.8.0 0 nvidia/label/cuda-11.8.0 cuda-libraries-dev 11.8.0 0 nvidia/label/cuda-11.8.0 cuda-memcheck 11.8.86 0 nvidia/label/cuda-11.8.0 cuda-nsight 11.8.86 0 nvidia/label/cuda-11.8.0 cuda-nsight-compute 11.8.0 0 nvidia/label/cuda-11.8.0 cuda-nvcc 11.8.89 0 nvidia/label/cuda-11.8.0 cuda-nvdisasm 11.8.86 0 nvidia/label/cuda-11.8.0 cuda-nvml-dev 11.8.86 0 nvidia/label/cuda-11.8.0 cuda-nvprof 11.8.87 0 nvidia/label/cuda-11.8.0 cuda-nvprune 11.8.86 0 nvidia/label/cuda-11.8.0 cuda-nvrtc 11.8.89 0 nvidia/label/cuda-11.8.0 cuda-nvrtc-dev 11.8.89 0 nvidia/label/cuda-11.8.0 cuda-nvtx 11.8.86 0 nvidia/label/cuda-11.8.0 cuda-nvvp 11.8.87 0 nvidia/label/cuda-11.8.0 cuda-profiler-api 11.8.86 0 nvidia/label/cuda-11.8.0 cuda-sanitizer-api 11.8.86 0 nvidia/label/cuda-11.8.0 cuda-toolkit 11.8.0 0 nvidia/label/cuda-11.8.0 cuda-tools 11.8.0 0 nvidia/label/cuda-11.8.0 cuda-version 11.8 hcce14f8_3 cuda-visual-tools 11.8.0 0 nvidia/label/cuda-11.8.0 cudatoolkit exceptiongroup 1.2.2 pypi_0 pypi fastapi 0.111.1 pypi_0 pypi fastapi-cli 0.0.4 pypi_0 pypi ffmpeg-python 0.2.0 pypi_0 pypi ffmpy 0.3.2 pypi_0 pypi filelock 3.15.4 pypi_0 pypi fonttools 4.53.1 pypi_0 pypi fsspec 2024.6.1 pypi_0 pypi future 1.0.0 pypi_0 pypi gds-tools 1.4.0.31 0 nvidia/label/cuda-11.8.0 gmp 6.2.1 h295c915_3 gmpy2 2.1.2 py310heeb90bb_0 gradio 4.37.1 pypi_0 pypi gradio-client 1.0.2 pypi_0 pypi h11 0.14.0 pypi_0 pypi httpcore 1.0.5 pypi_0 pypi httptools 0.6.1 pypi_0 pypi httpx 0.27.0 pypi_0 pypi huggingface-hub 0.24.2 pypi_0 pypi humanfriendly 10.0 py310h06a4308_1 idna 3.7 pypi_0 pypi imageio 2.34.2 pypi_0 pypi imageio-ffmpeg 0.5.1 pypi_0 pypi importlib-resources 6.4.0 pypi_0 pypi inquirerpy 0.3.4 pypi_0 pypi insightface 0.7.3 pypi_0 pypi jinja2 3.1.4 pypi_0 pypi joblib 1.4.2 pypi_0 pypi jsonschema 4.23.0 pypi_0 pypi jsonschema-specifications 2023.12.1 pypi_0 pypi kiwisolver 1.4.5 pypi_0 pypi lazy-loader 0.4 pypi_0 pypi ld_impl_linux-64 2.38 h1181459_1 libabseil 20240116.2 cxx17_h6a678d5_0 libcublas 11.11.3.6 0 nvidia/label/cuda-11.8.0 libcublas-dev 11.11.3.6 0 nvidia/label/cuda-11.8.0 libcufft 10.9.0.58 0 nvidia/label/cuda-11.8.0 libcufft-dev 10.9.0.58 0 nvidia/label/cuda-11.8.0 libcufile 1.4.0.31 0 nvidia/label/cuda-11.8.0 libcufile-dev 1.4.0.31 0 nvidia/label/cuda-11.8.0 libcurand 10.3.0.86 0 nvidia/label/cuda-11.8.0 libcurand-dev 10.3.0.86 0 nvidia/label/cuda-11.8.0 libcusolver 11.4.1.48 0 nvidia/label/cuda-11.8.0 libcusolver-dev 11.4.1.48 0 nvidia/label/cuda-11.8.0 libcusparse 11.7.5.86 0 nvidia/label/cuda-11.8.0 libcusparse-dev 11.7.5.86 0 nvidia/label/cuda-11.8.0 libffi 3.4.4 h6a678d5_1 libgcc-ng 14.1.0 h77fa898_0 conda-forge libgfortran-ng 11.2.0 h00389a lmdb 1.4.1 pypi_0 pypi mako 1.3.5 pypi_0 pypi markdown-it-py 3.0.0 pypi_0 pypi markupsafe 2.1.5 pypi_0 pypi matplotlib 3.9.0 pypi_0 pypi mdurl 0.1.2 pypi_0 pypi mpc 1.1.0 h10f8cd9_1 mpfr 4.0.2 hb69a4c5_1 mpmath 1.3.0 py310h06a4308_0 ncurses 6.4 h6a678d5_0 networkx 3.3 pypi_0 pypi nsight-compute 2022.3.0.22 0 nvidia/label/cuda-11.8.0 numpy 1.26.4 py310heeff2f4_0 numpy-base 1.26.4 py310h8a23956_0 nvidia-cublas-cu11 11.11.3.6 pypi_0 pypi nvidia-cublas-cu12 12.5.3.2 pypi_0 pypi nvidia-cuda-cupti-cu11 11.8.87 pypi_0 pypi nvidia-cuda-nvrtc-cu11 11.8.89 pypi_0 pypi nvidia-cuda-runtime-cu11 11.8.89 pypi_0 pypi nvidia-cuda-runtime-cu12 12.5.82 pypi_0 pypi nvidia-cudnn-cu11 8.7.0.84 pypi_0 pypi nvidia-cudnn-cu12 9.2.1.18 pypi_0 pypi nvidia-cufft-cu11 10.9.0.58 pypi_0 pypi nvidia-curand-cu11 10.3.0.86 pypi_0 pypi nvidia-cusolver-cu11 11.4.1.48 pypi_0 pypi nvidia-cusparse-cu11 11.7.5.86 pypi_0 pypi nvidia-nccl-cu11 2.20.5 pypi_0 pypi nvidia-nvtx-cu11 11.8.86 pypi_0 pypi nvidia-tensorrt 99.0.0 pypi_0 pypi omegaconf 2.3.0 pypi_0 pypi onnx 1.16.1 pypi_0 pypi onnxruntime 1.18.1 py310hefa0614_200_cuda conda-forge opencv-python 4.10.0.84 pypi_0 pypi opencv-python-headless 4.10.0.84 pypi_0 pypi openssl 3.3.1 h4bc722e_2 conda-forge orjson 3.10.6 pypi_0 pypi packaging 24.1 py310h06a4308_0 pandas 2.2.2 pypi_0 pypi pfzy 0.3.4 pypi_0 pypi pillow 10.4.0 pypi_0 pypi pip 24.0 py310h06a4308_0 platformdirs 4.2.2 pypi_0 pypi prettytable 3.10.2 pypi_0 pypi prompt-toolkit 3.0.47 pypi_0 pypi protobuf 4.25.3 py310h12ddb61_0 pycuda 2024.1.1 pypi_0 pypi pydantic 2.8.2 pypi_0 pypi pydantic-core 2.20.1 pypi_0 pypi pydu referencing 0.35.1 pypi_0 pypi requests 2.32.3 pypi_0 pypi rich 13.7.1 pypi_0 pypi rpds-py 0.19.1 pypi_0 pypi ruff 0.5.4 pypi_0 pypi scikit-image 0.24.0 pypi_0 pypi scikit-learn 1.5.1 pypi_0 pypi scipy 1.13.1 pypi_0 pypi semantic-version 2.10.0 pypi_0 pypi setuptools 69.5.1 py310h06a4308_0 shellingham 1.5.4 pypi_0 pypi shtab 1.7.1 pypi_0 pypi six 1.16.0 pypi_0 pypi sniffio 1.3.1 pypi_0 pypi sqlite 3.45.3 h5eee18b_0 starlette 0.37.2 pypi_0 pypi sympy 1.12 py310h06a4308_0 tensorrt 8.6.1 pypi_0 pypi tensorrt-bindings 8.6.1 pypi_0 pypi tensorrt-cu12 10.2.0.post1 pypi_0 pypi tensorrt-cu12-bindings 10.2.0.post1 pypi_0 pypi tensorrt-cu12-libs 10.2.0.post1 pypi_0 pypi tensorrt-libs 8.6.1 pypi_0 pypi threadpoolctl 3.5.0 pypi_0 pypi tifffile 2024.7.24 pypi_0 pypi tk 8.6.14 h39e8969_0 tomli 2.0.1 pypi_0 pypi tomlkit 0.12.0 pypi_0 pypi toolz 0.12.1 pypi_0 pypi torch 2.3.0+cu118 pypi_0 pypi torch-tensorrt 2.3.0 pypi_0 pypi torch2trt 0.5.0 pypi_0 pypi torchaudio 2.3.0+cu118 pypi_0 pypi torchvision 0.18.0+cu118 pypi_0 pypi tqdm 4.66.4 pypi_0 pypi triton 2.3.0 pypi_0 pypi typer 0.12.3 pypi_0 pypi typing-extensions 4.12.2 pypi_0 pypi tyro 0.8.5 pypi_0 pypi tzdata 2024.1 pypi_0 pypi urllib3 2.2.2 pypi_0 pypi uvicorn 0.30.3 pypi_0 pypi uvloop 0.19.0 pypi_0 pypi watchfiles 0.22.0 pypi_0 pypi wcwidth 0.2.13 pypi_0 pypi websockets 11.0.3 pypi_0 pypi wheel 0.43.0 py310h06a4308_0 xz 5.4.6 h5eee18b_1 zlib 1.2.13 h4ab18f5_6 conda-forge

linhcentrio commented 2 months ago

(/home/Efficient-Live-Portrait/venv) [root@aiclip-dfl FasterLivePortrait]# pip show onnxruntime-gpu Name: onnxruntime-gpu Version: 1.18.1 Summary: ONNX Runtime is a runtime accelerator for Machine Learning models Home-page: https://onnxruntime.ai Author: Microsoft Corporation Author-email: onnxruntime@microsoft.com License: MIT License Location: /home/Efficient-Live-Portrait/venv/lib/python3.10/site-packages Requires: coloredlogs, flatbuffers, numpy, packaging, protobuf, sympy Required-by:

warmshao commented 2 months ago

Your Python environment is incorrect, please use the windows integration package or Docker image that I provided.