search

aigc-apps / sd-webui-EasyPhoto

📷 EasyPhoto | Your Smart AI Photo Generator.
Apache License 2.0
4.98k stars 398 forks source link

ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 512 and the array at index 1 has size 1 #265

Closed hkpD closed 1 year ago

hkpD commented 1 year ago

Hello,

This extension was working fine before. I generated 5 models successfully. But now it is not working. I receive the error below every time I try to run training.

Unloaded weights 0.0s.
Loading weights [6ce0161689] from /home/user/Desktop/stable-diffusion-webui/models/Stable-diffusion/v1-5-pruned-emaonly.safetensors
2023-11-17 16:40:48,524 - modelscope - INFO - PyTorch version 2.1.1+rocm5.6 Found.
2023-11-17 16:40:48,526 - modelscope - INFO - TensorFlow version 2.15.0 Found.
2023-11-17 16:40:48,526 - modelscope - INFO - Loading ast index from /home/user/.cache/modelscope/ast_indexer
2023-11-17 16:40:48,551 - modelscope - INFO - Loading done! Current index file version is 1.9.3, with md5 716b813974a759519215f575ef91cf38 and a total number of 943 components indexed
2023-11-17 16:40:50,818 - modelscope - INFO - Use user-specified model revision: v1.0.3
2023-11-17 16:40:51,451 - modelscope - WARNING - ('PIPELINES', 'face_recognition', 'face_recognition') not found in ast index file
2023-11-17 16:40:51,451 - modelscope - INFO - initiate model from /home/user/.cache/modelscope/hub/bubbliiiing/cv_retinafce_recognition
2023-11-17 16:40:51,451 - modelscope - INFO - initiate model from location /home/user/.cache/modelscope/hub/bubbliiiing/cv_retinafce_recognition.
2023-11-17 16:40:51,452 - modelscope - INFO - initialize model from /home/user/.cache/modelscope/hub/bubbliiiing/cv_retinafce_recognition
2023-11-17 16:40:51,453 - modelscope - WARNING - ('MODELS', 'face_recognition', 'face_recognition') not found in ast index file
2023-11-17 16:40:54,098 - modelscope - INFO - Model revision not specified, use revision: v2.0.2
2023-11-17 16:40:55,789 - modelscope - INFO - initiate model from /home/user/.cache/modelscope/hub/damo/cv_resnet50_face-detection_retinaface
2023-11-17 16:40:55,789 - modelscope - INFO - initiate model from location /home/user/.cache/modelscope/hub/damo/cv_resnet50_face-detection_retinaface.
2023-11-17 16:40:55,791 - modelscope - WARNING - No preprocessor field found in cfg.
2023-11-17 16:40:55,791 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2023-11-17 16:40:55,791 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': '/home/user/.cache/modelscope/hub/damo/cv_resnet50_face-detection_retinaface'}. trying to build by task and model information.
2023-11-17 16:40:55,791 - modelscope - WARNING - Find task: face-detection, model type: None. Insufficient information to build preprocessor, skip building preprocessor
2023-11-17 16:40:55,791 - modelscope - INFO - loading model from /home/user/.cache/modelscope/hub/damo/cv_resnet50_face-detection_retinaface/pytorch_model.pt
/home/user/Desktop/stable-diffusion-webui/venv/lib64/python3.10/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
/home/user/Desktop/stable-diffusion-webui/venv/lib64/python3.10/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=None`.
  warnings.warn(msg)
2023-11-17 16:40:56,260 - modelscope - INFO - load model done
2023-11-17 16:40:56,864 - modelscope - INFO - load facefusion models done
2023-11-17 16:40:56,864 - modelscope - INFO - init done
2023-11-17 16:40:56,865 - modelscope - WARNING - No preprocessor field found in cfg.
2023-11-17 16:40:56,865 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2023-11-17 16:40:56,865 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': '/home/user/.cache/modelscope/hub/bubbliiiing/cv_retinafce_recognition'}. trying to build by task and model information.
2023-11-17 16:40:56,865 - modelscope - WARNING - No preprocessor key ('face_recognition', 'face_recognition') found in PREPROCESSOR_MAP, skip building preprocessor.
2023-11-17 16:40:56,865 - modelscope - INFO - image face recognition model init done
2023-11-17 16:40:58,717 - modelscope - INFO - Use user-specified model revision: v2.0.2
2023-11-17 16:41:00,897 - modelscope - INFO - initiate model from /home/user/.cache/modelscope/hub/damo/cv_resnet50_face-detection_retinaface
2023-11-17 16:41:00,897 - modelscope - INFO - initiate model from location /home/user/.cache/modelscope/hub/damo/cv_resnet50_face-detection_retinaface.
2023-11-17 16:41:00,898 - modelscope - WARNING - No preprocessor field found in cfg.
2023-11-17 16:41:00,898 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2023-11-17 16:41:00,898 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': '/home/user/.cache/modelscope/hub/damo/cv_resnet50_face-detection_retinaface'}. trying to build by task and model information.
2023-11-17 16:41:00,898 - modelscope - WARNING - Find task: face-detection, model type: None. Insufficient information to build preprocessor, skip building preprocessor
2023-11-17 16:41:00,898 - modelscope - INFO - loading model from /home/user/.cache/modelscope/hub/damo/cv_resnet50_face-detection_retinaface/pytorch_model.pt
2023-11-17 16:41:01,139 - modelscope - INFO - load model done
2023-11-17 16:41:03,122 - modelscope - INFO - Use user-specified model revision: v1.0.0
2023-11-17 16:41:03,740 - modelscope - INFO - initiate model from /home/user/.cache/modelscope/hub/damo/cv_u2net_salient-detection
2023-11-17 16:41:03,740 - modelscope - INFO - initiate model from location /home/user/.cache/modelscope/hub/damo/cv_u2net_salient-detection.
2023-11-17 16:41:03,740 - modelscope - INFO - initialize model from /home/user/.cache/modelscope/hub/damo/cv_u2net_salient-detection
2023-11-17 16:41:03,964 - modelscope - WARNING - No preprocessor field found in cfg.
2023-11-17 16:41:03,964 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2023-11-17 16:41:03,964 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': '/home/user/.cache/modelscope/hub/damo/cv_u2net_salient-detection'}. trying to build by task and model information.
2023-11-17 16:41:03,964 - modelscope - WARNING - No preprocessor key ('detection', 'semantic-segmentation') found in PREPROCESSOR_MAP, skip building preprocessor.
2023-11-17 16:41:05,936 - modelscope - INFO - Use user-specified model revision: v1.0.2
2023-11-17 16:41:06,419 - modelscope - WARNING - ('PIPELINES', 'skin-retouching-torch', 'skin-retouching-torch') not found in ast index file
2023-11-17 16:41:06,419 - modelscope - INFO - initiate model from /home/user/.cache/modelscope/hub/damo/cv_unet_skin_retouching_torch
2023-11-17 16:41:06,419 - modelscope - INFO - initiate model from location /home/user/.cache/modelscope/hub/damo/cv_unet_skin_retouching_torch.
2023-11-17 16:41:06,421 - modelscope - WARNING - No preprocessor field found in cfg.
2023-11-17 16:41:06,421 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2023-11-17 16:41:06,421 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': '/home/user/.cache/modelscope/hub/damo/cv_unet_skin_retouching_torch'}. trying to build by task and model information.
2023-11-17 16:41:06,421 - modelscope - WARNING - Find task: skin-retouching-torch, model type: None. Insufficient information to build preprocessor, skip building preprocessor
2023-11-17 16:41:08,501 - modelscope - INFO - Model revision not specified, use revision: v2.0.2
2023-11-17 16:41:10,484 - modelscope - INFO - initiate model from /home/user/.cache/modelscope/hub/damo/cv_resnet50_face-detection_retinaface
2023-11-17 16:41:10,484 - modelscope - INFO - initiate model from location /home/user/.cache/modelscope/hub/damo/cv_resnet50_face-detection_retinaface.
2023-11-17 16:41:10,485 - modelscope - WARNING - No preprocessor field found in cfg.
2023-11-17 16:41:10,485 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2023-11-17 16:41:10,485 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': '/home/user/.cache/modelscope/hub/damo/cv_resnet50_face-detection_retinaface'}. trying to build by task and model information.
2023-11-17 16:41:10,486 - modelscope - WARNING - Find task: face-detection, model type: None. Insufficient information to build preprocessor, skip building preprocessor
2023-11-17 16:41:10,486 - modelscope - INFO - loading model from /home/user/.cache/modelscope/hub/damo/cv_resnet50_face-detection_retinaface/pytorch_model.pt
2023-11-17 16:41:10,735 - modelscope - INFO - load model done
EP Error /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1193 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_tensorrt.so with error: libcudnn.so.8: cannot open shared object file: No such file or directory
 when using ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.
2023-11-17 16:41:11.283809812 [E:onnxruntime:Default, provider_bridge_ort.cc:1480 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1193 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory

2023-11-17 16:41:11.283824662 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:747 CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Please reference https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements to ensure all dependencies are met.
2023-11-17 16:41:11.517757402 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2649'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517773802 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2657'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517777162 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2644'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517780722 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2594'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517784472 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2596'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517788522 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2653'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517791752 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2624'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517795392 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2652'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517800072 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2645'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517802892 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2643'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517805892 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2648'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517808392 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2647'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517811532 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2641'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517814652 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2633'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517816962 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2632'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517820412 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2614'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517823362 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2613'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517826242 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2658'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517829302 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2606'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:11.517832222 [W:onnxruntime:, graph.cc:3553 CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2598'. It is not used by any node and should be removed from the model.
2023-11-17 16:41:13,628 - modelscope - INFO - Use user-specified model revision: v1.0.0
2023-11-17 16:41:14,128 - modelscope - INFO - initiate model from /home/user/.cache/modelscope/hub/damo/cv_gpen_image-portrait-enhancement
2023-11-17 16:41:14,128 - modelscope - INFO - initiate model from location /home/user/.cache/modelscope/hub/damo/cv_gpen_image-portrait-enhancement.
2023-11-17 16:41:14,129 - modelscope - INFO - initialize model from /home/user/.cache/modelscope/hub/damo/cv_gpen_image-portrait-enhancement
Loading ResNet ArcFace
2023-11-17 16:41:15,294 - modelscope - INFO - load face enhancer model done
2023-11-17 16:41:15,520 - modelscope - INFO - load face detector model done
2023-11-17 16:41:15,793 - modelscope - INFO - load sr model done
2023-11-17 16:41:16,457 - modelscope - INFO - load fqa model done
  0%|                                                                                                                                                                 | 0/20 [00:00<?, ?it/s]2023-11-17 16:41:16,674 - modelscope - WARNING - task skin-retouching-torch input definition is missing
2023-11-17 16:41:16,872 - modelscope - WARNING - task skin-retouching-torch output keys are missing
2023-11-17 16:41:16,877 - modelscope - WARNING - task face_recognition input definition is missing
2023-11-17 16:41:16,997 - modelscope - INFO - model inference done
2023-11-17 16:41:16,997 - modelscope - WARNING - task face_recognition output keys are missing
  5%|███████▋                                                                                                                                                 | 1/20 [00:00<00:09,  1.95it/s]ERROR:root:Photo detect and count score error, error info: index 0 is out of bounds for axis 0 with size 0
ERROR:root:Photo skin_retouching error, error info: 'output_img'
2023-11-17 16:41:17,107 - modelscope - INFO - model inference done
 15%|██████████████████████▉                                                                                                                                  | 3/20 [00:00<00:02,  5.76it/s]ERROR:root:Photo detect and count score error, error info: index 0 is out of bounds for axis 0 with size 0
2023-11-17 16:41:17,343 - modelscope - INFO - model inference done
 25%|██████████████████████████████████████▎                                                                                                                  | 5/20 [00:00<00:02,  6.90it/s]2023-11-17 16:41:17,569 - modelscope - INFO - model inference done
ERROR:root:Photo detect and count score error, error info: index 0 is out of bounds for axis 0 with size 0
 35%|█████████████████████████████████████████████████████▌                                                                                                   | 7/20 [00:01<00:01,  7.38it/s]ERROR:root:Photo detect and count score error, error info: index 0 is out of bounds for axis 0 with size 0
2023-11-17 16:41:17,794 - modelscope - INFO - model inference done
 45%|████████████████████████████████████████████████████████████████████▊                                                                                    | 9/20 [00:01<00:01,  8.11it/s]2023-11-17 16:41:17,978 - modelscope - INFO - model inference done
 50%|████████████████████████████████████████████████████████████████████████████                                                                            | 10/20 [00:01<00:01,  7.34it/s]ERROR:root:Photo detect and count score error, error info: index 0 is out of bounds for axis 0 with size 0
ERROR:root:Photo detect and count score error, error info: index 0 is out of bounds for axis 0 with size 0
ERROR:root:Photo detect and count score error, error info: index 0 is out of bounds for axis 0 with size 0
ERROR:root:Photo detect and count score error, error info: index 0 is out of bounds for axis 0 with size 0
 70%|██████████████████████████████████████████████████████████████████████████████████████████████████████████▍                                             | 14/20 [00:01<00:00, 10.70it/s]ERROR:root:Photo skin_retouching error, error info: 'output_img'
2023-11-17 16:41:18,619 - modelscope - INFO - model inference done
2023-11-17 16:41:19,117 - modelscope - INFO - model inference done
 80%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌                              | 16/20 [00:02<00:00,  5.14it/s]2023-11-17 16:41:19,660 - modelscope - INFO - model inference done
 85%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏                      | 17/20 [00:03<00:00,  3.95it/s]2023-11-17 16:41:20,196 - modelscope - INFO - model inference done
 90%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊               | 18/20 [00:03<00:00,  3.25it/s]ERROR:root:Photo skin_retouching error, error info: 'output_img'
2023-11-17 16:41:20,545 - modelscope - INFO - model inference done
 95%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍       | 19/20 [00:04<00:00,  3.16it/s]ERROR:root:Photo skin_retouching error, error info: 'output_img'
2023-11-17 16:41:20,859 - modelscope - INFO - model inference done
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:04<00:00,  4.57it/s]
Traceback (most recent call last):
  File "/home/user/Desktop/stable-diffusion-webui/extensions/sd-webui-EasyPhoto/scripts/preprocess.py", line 181, in <module>
    face_id_scores      = compare_jpg_with_face_id(face_id_scores)
  File "/home/user/Desktop/stable-diffusion-webui/extensions/sd-webui-EasyPhoto/scripts/preprocess.py", line 74, in compare_jpg_with_face_id
    embedding_array = np.vstack(embedding_list)
  File "<__array_function__ internals>", line 180, in vstack
  File "/home/user/Desktop/stable-diffusion-webui/venv/lib64/python3.10/site-packages/numpy/core/shape_base.py", line 282, in vstack
    return _nx.concatenate(arrs, 0)
  File "<__array_function__ internals>", line 180, in concatenate
ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 512 and the array at index 1 has size 1
Error executing the command: Command '['/home/user/Desktop/stable-diffusion-webui/venv/bin/python3.10', '/home/user/Desktop/stable-diffusion-webui/extensions/sd-webui-EasyPhoto/scripts/preprocess.py', '--images_save_path=/home/user/Desktop/stable-diffusion-webui/outputs/easyphoto-user-id-infos/remy/processed_images/train', '--json_save_path=/home/user/Desktop/stable-diffusion-webui/outputs/easyphoto-user-id-infos/remy/processed_images/metadata.jsonl', '--validation_prompt=easyphoto_face, easyphoto, 1person', '--inputs_dir=/home/user/Desktop/stable-diffusion-webui/outputs/easyphoto-user-id-infos/remy/original_backup', '--ref_image_path=/home/user/Desktop/stable-diffusion-webui/outputs/easyphoto-user-id-infos/remy/ref_image.jpg', '--skin_retouching_bool']' returned non-zero exit status 1.
Using already loaded model v1-5-pruned-emaonly.safetensors [6ce0161689]: done in 0.0s
hkpD commented 1 year ago

I was using pictures of my puppy's face and the errors went away after I input some pictures of human faces.

wuziheng commented 1 year ago

Yes, this extensions main branch only works for human face, we use modelscope face detection, when detection human face failed, it will may encourage with some bugs, espcially all train images has no human face

wuziheng commented 1 year ago

if you are able to fix this code File "/home/user/Desktop/stable-diffusion-webui/extensions/sd-webui-EasyPhoto/scripts/preprocess.py", line 74, in compare_jpg_with_face_id embedding_array = np.vstack(embedding_list)

check the shape of all embedding in embedding_list, and remove type = None may help

hkpD commented 1 year ago

Closing this issue. The problem was that I was using a dog's face and it wouldn't recognize it.