I can't get facechain to finish training without throwing errors. This is second attempt, here used only 1 image, in first attempt I used 3 images. Running this inside SD Web UI, installed via settings/extensions.
** Setting base model to SD1.5 **
--------uuid: qw
----------work_dir: E:\software\stable-diffusion-webui\extensions\facechain\worker_data\qw\ly261666/cv_portrait_model\nikophoenix
2024-03-16 00:43:44,860 - modelscope - INFO - Use user-specified model revision: v1.0.0
bin E:\software\stable-diffusion-webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda118.dll
2024-03-16 00:43:50,330 - modelscope - INFO - PyTorch version 2.0.1+cu118 Found.
2024-03-16 00:43:50,333 - modelscope - INFO - Loading ast index from C:\Users\maxnb\.cache\modelscope\ast_indexer
2024-03-16 00:43:50,498 - modelscope - INFO - Loading done! Current index file version is 1.13.1, with md5 52b5a71ccce81ee9579827e7da92225f and a total number of 972 components indexed
2024-03-16 00:43:53,109 - modelscope - INFO - Use user-specified model revision: v4.0
2024-03-16 00:43:56,338 - modelscope - INFO - Use user-specified model revision: v1.0.1
2024-03-16 00:43:56,928 - modelscope - WARNING - ('PIPELINES', 'skin-retouching-torch', 'skin-retouching-torch') not found in ast index file
2024-03-16 00:43:56,928 - modelscope - INFO - initiate model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_unet_skin_retouching_torch
2024-03-16 00:43:56,929 - modelscope - INFO - initiate model from location C:\Users\maxnb\.cache\modelscope\hub\damo\cv_unet_skin_retouching_torch.
2024-03-16 00:43:56,934 - modelscope - WARNING - No preprocessor field found in cfg.
2024-03-16 00:43:56,934 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2024-03-16 00:43:56,934 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': 'C:\\Users\\maxnb\\.cache\\modelscope\\hub\\damo\\cv_unet_skin_retouching_torch'}. trying to build by task and model information.
2024-03-16 00:43:56,934 - modelscope - WARNING - Find task: skin-retouching-torch, model type: None. Insufficient information to build preprocessor, skip building preprocessor
2024-03-16 00:43:59,597 - modelscope - WARNING - Model revision not specified, use revision: v2.0.2
2024-03-16 00:44:01,518 - modelscope - INFO - initiate model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_resnet50_face-detection_retinaface
2024-03-16 00:44:01,518 - modelscope - INFO - initiate model from location C:\Users\maxnb\.cache\modelscope\hub\damo\cv_resnet50_face-detection_retinaface.
2024-03-16 00:44:01,523 - modelscope - WARNING - No preprocessor field found in cfg.
2024-03-16 00:44:01,523 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2024-03-16 00:44:01,523 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': 'C:\\Users\\maxnb\\.cache\\modelscope\\hub\\damo\\cv_resnet50_face-detection_retinaface'}. trying to build by task and model information.
2024-03-16 00:44:01,524 - modelscope - WARNING - Find task: face-detection, model type: None. Insufficient information to build preprocessor, skip building preprocessor
2024-03-16 00:44:01,525 - modelscope - INFO - loading model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_resnet50_face-detection_retinaface\pytorch_model.pt
2024-03-16 00:44:01,883 - modelscope - INFO - load model done
2024-03-16 00:44:02.6659705 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_4__1660'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:02.6714730 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'Shape__1662:0'. It is not used by any node and should be
e removed from the model.
2024-03-16 00:44:02.6757284 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'Sub__1664:0'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.0404917 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2649'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.0445014 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2644'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.0494403 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2647'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.0555460 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2658'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.0594158 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2648'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.0634211 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2657'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.0671832 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2653'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.0717476 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2652'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.0755791 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2645'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.0796396 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2643'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.0836930 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2641'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.0883988 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2633'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.0922432 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2632'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.0962716 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2624'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.1001148 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2614'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.1047195 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2613'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.1085622 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2606'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.1125392 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2598'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.1164018 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2596'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:03.1208837 [W:onnxruntime:, graph.cc:3593 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2594'. It is not used by any node and should be removed from the model.
2024-03-16 00:44:05,561 - modelscope - INFO - Use user-specified model revision: v1.1
2024-03-16 00:44:06,094 - modelscope - INFO - initiate model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd
2024-03-16 00:44:06,095 - modelscope - INFO - initiate model from location C:\Users\maxnb\.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd.
2024-03-16 00:44:06,097 - modelscope - INFO - initialize model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd
2024-03-16 00:44:07,624 - mmcv - INFO - initialize PAFPN with init_cfg {'type': 'Xavier', 'layer': 'Conv2d', 'distribution': 'uniform'}
2024-03-16 00:44:07,626 - mmcv - INFO -
lateral_convs.0.conv.weight - torch.Size([16, 64, 1, 1]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:07,626 - mmcv - INFO -
lateral_convs.0.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:07,626 - mmcv - INFO -
lateral_convs.1.conv.weight - torch.Size([16, 120, 1, 1]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:07,626 - mmcv - INFO -
lateral_convs.1.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:07,626 - mmcv - INFO -
lateral_convs.2.conv.weight - torch.Size([16, 160, 1, 1]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:07,626 - mmcv - INFO -
lateral_convs.2.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:07,626 - mmcv - INFO -
fpn_convs.0.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:07,626 - mmcv - INFO -
fpn_convs.0.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:07,627 - mmcv - INFO -
fpn_convs.1.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:07,627 - mmcv - INFO -
fpn_convs.1.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:07,627 - mmcv - INFO -
fpn_convs.2.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:07,627 - mmcv - INFO -
fpn_convs.2.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:07,627 - mmcv - INFO -
downsample_convs.0.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:07,627 - mmcv - INFO -
downsample_convs.0.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:07,627 - mmcv - INFO -
downsample_convs.1.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:07,627 - mmcv - INFO -
downsample_convs.1.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:07,628 - mmcv - INFO -
pafpn_convs.0.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:07,628 - mmcv - INFO -
pafpn_convs.0.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:07,628 - mmcv - INFO -
pafpn_convs.1.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:07,628 - mmcv - INFO -
pafpn_convs.1.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:07,628 - modelscope - INFO - loading model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd\pytorch_model.pt
load checkpoint from local path: C:\Users\maxnb\.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd\pytorch_model.pt
2024-03-16 00:44:07,803 - modelscope - INFO - load model done
2024-03-16 00:44:09,883 - modelscope - INFO - Use user-specified model revision: v1.0.1
2024-03-16 00:44:10,438 - modelscope - INFO - initiate model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_resnet101_image-multiple-human-parsing
2024-03-16 00:44:10,439 - modelscope - INFO - initiate model from location C:\Users\maxnb\.cache\modelscope\hub\damo\cv_resnet101_image-multiple-human-parsing.
2024-03-16 00:44:10,441 - modelscope - INFO - initialize model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_resnet101_image-multiple-human-parsing
2024-03-16 00:44:10,846 - modelscope - INFO - loading model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_resnet101_image-multiple-human-parsing\pytorch_model.pt
2024-03-16 00:44:11,144 - modelscope - INFO - criterion.empty_weight doesn't exist in current model, skip loading.
2024-03-16 00:44:11,170 - modelscope - INFO - load model done
2024-03-16 00:44:11,193 - modelscope - WARNING - No preprocessor field found in cfg.
2024-03-16 00:44:11,194 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2024-03-16 00:44:11,194 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': 'C:\\Users\\maxnb\\.cache\\modelscope\\hub\\damo\\cv_resnet101_image-multiple-human-parsing'}. trying to build by task and model information.
2024-03-16 00:44:11,194 - modelscope - WARNING - No preprocessor key ('m2fp', 'image-segmentation') found in PREPROCESSOR_MAP, skip building preprocessor.
2024-03-16 00:44:13,121 - modelscope - INFO - Use user-specified model revision: v2.0.2
2024-03-16 00:44:14,748 - modelscope - INFO - initiate model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_resnet34_face-attribute-recognition_fairface
2024-03-16 00:44:14,749 - modelscope - INFO - initiate model from location C:\Users\maxnb\.cache\modelscope\hub\damo\cv_resnet34_face-attribute-recognition_fairface.
2024-03-16 00:44:14,754 - modelscope - WARNING - No preprocessor field found in cfg.
2024-03-16 00:44:14,754 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2024-03-16 00:44:14,754 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': 'C:\\Users\\maxnb\\.cache\\modelscope\\hub\\damo\\cv_resnet34_face-attribute-recognition_fairface'}. trying to build by task and model information.
2024-03-16 00:44:14,755 - modelscope - WARNING - Find task: face-attribute-recognition, model type: None. Insufficient information to build preprocessor, skip building preprocessor
2024-03-16 00:44:17,430 - modelscope - WARNING - Model revision not specified, use revision: v1.1
2024-03-16 00:44:17,875 - modelscope - INFO - initiate model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd
2024-03-16 00:44:17,876 - modelscope - INFO - initiate model from location C:\Users\maxnb\.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd.
2024-03-16 00:44:17,878 - modelscope - INFO - initialize model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd
2024-03-16 00:44:17,910 - mmcv - INFO - initialize PAFPN with init_cfg {'type': 'Xavier', 'layer': 'Conv2d', 'distribution': 'uniform'}
2024-03-16 00:44:17,911 - mmcv - INFO -
lateral_convs.0.conv.weight - torch.Size([16, 64, 1, 1]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:17,912 - mmcv - INFO -
lateral_convs.0.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:17,912 - mmcv - INFO -
lateral_convs.1.conv.weight - torch.Size([16, 120, 1, 1]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:17,912 - mmcv - INFO -
lateral_convs.1.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:17,912 - mmcv - INFO -
lateral_convs.2.conv.weight - torch.Size([16, 160, 1, 1]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:17,912 - mmcv - INFO -
lateral_convs.2.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:17,912 - mmcv - INFO -
fpn_convs.0.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:17,912 - mmcv - INFO -
fpn_convs.0.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:17,912 - mmcv - INFO -
fpn_convs.1.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:17,912 - mmcv - INFO -
fpn_convs.1.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:17,912 - mmcv - INFO -
fpn_convs.2.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:17,912 - mmcv - INFO -
fpn_convs.2.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:17,913 - mmcv - INFO -
downsample_convs.0.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:17,913 - mmcv - INFO -
downsample_convs.0.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:17,913 - mmcv - INFO -
downsample_convs.1.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:17,913 - mmcv - INFO -
downsample_convs.1.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:17,913 - mmcv - INFO -
pafpn_convs.0.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:17,913 - mmcv - INFO -
pafpn_convs.0.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:17,913 - mmcv - INFO -
pafpn_convs.1.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:17,913 - mmcv - INFO -
pafpn_convs.1.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:17,914 - modelscope - INFO - loading model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd\pytorch_model.pt
load checkpoint from local path: C:\Users\maxnb\.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd\pytorch_model.pt
2024-03-16 00:44:17,967 - modelscope - INFO - load model done
2024-03-16 00:44:17,978 - modelscope - INFO - loading model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_resnet34_face-attribute-recognition_fairface\pytorch_model.pt
2024-03-16 00:44:18,323 - modelscope - INFO - load model done
2024-03-16 00:44:20,875 - modelscope - INFO - Use user-specified model revision: v2.5
2024-03-16 00:44:21,337 - modelscope - INFO - initiate model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_manual_facial-landmark-confidence_flcm
2024-03-16 00:44:21,337 - modelscope - INFO - initiate model from location C:\Users\maxnb\.cache\modelscope\hub\damo\cv_manual_facial-landmark-confidence_flcm.
2024-03-16 00:44:21,343 - modelscope - WARNING - No preprocessor field found in cfg.
2024-03-16 00:44:21,343 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2024-03-16 00:44:21,343 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': 'C:\\Users\\maxnb\\.cache\\modelscope\\hub\\damo\\cv_manual_facial-landmark-confidence_flcm'}. trying to build by task and model information.
2024-03-16 00:44:21,343 - modelscope - WARNING - Find task: face-2d-keypoints, model type: None. Insufficient information to build preprocessor, skip building preprocessor
2024-03-16 00:44:23,409 - modelscope - WARNING - Model revision not specified, use revision: v1.1
2024-03-16 00:44:23,926 - modelscope - INFO - initiate model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd
2024-03-16 00:44:23,927 - modelscope - INFO - initiate model from location C:\Users\maxnb\.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd.
2024-03-16 00:44:23,929 - modelscope - INFO - initialize model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd
2024-03-16 00:44:23,961 - mmcv - INFO - initialize PAFPN with init_cfg {'type': 'Xavier', 'layer': 'Conv2d', 'distribution': 'uniform'}
2024-03-16 00:44:23,963 - mmcv - INFO -
lateral_convs.0.conv.weight - torch.Size([16, 64, 1, 1]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:23,963 - mmcv - INFO -
lateral_convs.0.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:23,963 - mmcv - INFO -
lateral_convs.1.conv.weight - torch.Size([16, 120, 1, 1]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:23,963 - mmcv - INFO -
lateral_convs.1.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:23,963 - mmcv - INFO -
lateral_convs.2.conv.weight - torch.Size([16, 160, 1, 1]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:23,963 - mmcv - INFO -
lateral_convs.2.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:23,963 - mmcv - INFO -
fpn_convs.0.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:23,963 - mmcv - INFO -
fpn_convs.0.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:23,964 - mmcv - INFO -
fpn_convs.1.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:23,964 - mmcv - INFO -
fpn_convs.1.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:23,964 - mmcv - INFO -
fpn_convs.2.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:23,964 - mmcv - INFO -
fpn_convs.2.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:23,964 - mmcv - INFO -
downsample_convs.0.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:23,964 - mmcv - INFO -
downsample_convs.0.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:23,964 - mmcv - INFO -
downsample_convs.1.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:23,964 - mmcv - INFO -
downsample_convs.1.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:23,964 - mmcv - INFO -
pafpn_convs.0.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:23,964 - mmcv - INFO -
pafpn_convs.0.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:23,965 - mmcv - INFO -
pafpn_convs.1.conv.weight - torch.Size([16, 16, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0
2024-03-16 00:44:23,965 - mmcv - INFO -
pafpn_convs.1.conv.bias - torch.Size([16]):
The value is the same before and after calling `init_weights` of PAFPN
2024-03-16 00:44:23,965 - modelscope - INFO - loading model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd\pytorch_model.pt
load checkpoint from local path: C:\Users\maxnb\.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd\pytorch_model.pt
2024-03-16 00:44:24,008 - modelscope - INFO - load model done
2024-03-16 00:44:24,018 - modelscope - INFO - loading model from C:\Users\maxnb\.cache\modelscope\hub\damo\cv_manual_facial-landmark-confidence_flcm\pytorch_model.pt
2024-03-16 00:44:24,034 - modelscope - INFO - load model done
2024-03-16 00:44:25,623 - modelscope - WARNING - task skin-retouching-torch input definition is missing
2024-03-16 00:44:27,497 - modelscope - WARNING - task skin-retouching-torch output keys are missing
000.jpg 0.951323539018631
black_eyes, facial_hair, looking_at_viewer, realistic, simple_background, solo, transparent_background
[['black_eyes', 'facial_hair', 'looking_at_viewer', 'realistic', 'simple_background', 'solo', 'transparent_background']]
0.png a handsome man
instance_data_dir E:\software\stable-diffusion-webui\extensions\facechain\worker_data\qw\training_data\ly261666/cv_portrait_model\nikophoenix
Traceback (most recent call last):
File "E:\software\stable-diffusion-webui\extensions\facechain\facechain\train_text_to_image_lora.py", line 31, in <module>
import datasets
ModuleNotFoundError: No module named 'datasets'
Error executing the command: Command '['python', 'E:\\software\\stable-diffusion-webui\\extensions\\facechain/facechain/train_text_to_image_lora.py', '--pretrained_model_name_or_path=ly261666/cv_portrait_model', '--revision=v2.0', '--sub_path=film/film', '--output_dataset_name=E:\\software\\stable-diffusion-webui\\extensions\\facechain\\worker_data\\qw\\training_data\\ly261666/cv_portrait_model\\nikophoenix', '--caption_column=text', '--resolution=512', '--random_flip', '--train_batch_size=1', '--num_train_epochs=200', '--checkpointing_steps=5000', '--learning_rate=1.5e-04', '--lr_scheduler=cosine', '--lr_warmup_steps=0', '--seed=42', '--output_dir=E:\\software\\stable-diffusion-webui\\extensions\\facechain\\worker_data\\qw\\ly261666/cv_portrait_model\\nikophoenix', '--lora_r=4', '--lora_alpha=32', '--lora_text_encoder_r=32', '--lora_text_encoder_alpha=32', '--resume_from_checkpoint=fromfacecommon']' returned non-zero exit status 1.
Traceback (most recent call last):
File "E:\software\stable-diffusion-webui\extensions\facechain\app.py", line 147, in train_lora_fn
subprocess.run(command, check=True)
File "C:\Users\maxnb\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['python', 'E:\\software\\stable-diffusion-webui\\extensions\\facechain/facechain/train_text_to_image_lora.py', '--pretrained_model_name_or_path=ly261666/cv_portrait_model', '--revision=v2.0', '--sub_path=film/film', '--output_dataset_name=E:\\software\\stable-diffusion-webui\\extensions\\facechain\\worker_data\\qw\\training_data\\ly261666/cv_portrait_model\\nikophoenix', '--caption_column=text', '--resolution=512', '--random_flip', '--train_batch_size=1', '--num_train_epochs=200', '--checkpointing_steps=5000', '--learning_rate=1.5e-04', '--lr_scheduler=cosine', '--lr_warmup_steps=0', '--seed=42', '--output_dir=E:\\software\\stable-diffusion-webui\\extensions\\facechain\\worker_data\\qw\\ly261666/cv_portrait_model\\nikophoenix', '--lora_r=4', '--lora_alpha=32', '--lora_text_encoder_r=32', '--lora_text_encoder_alpha=32', '--resume_from_checkpoint=fromfacecommon']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "E:\software\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
output = await app.get_blocks().process_api(
File "E:\software\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
result = await self.call_function(
File "E:\software\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
prediction = await anyio.to_thread.run_sync(
File "E:\software\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "E:\software\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "E:\software\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
result = context.run(func, *args)
File "E:\software\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
response = f(*args, **kwargs)
File "E:\software\stable-diffusion-webui\extensions\facechain\app.py", line 804, in run
train_lora_fn(base_model_path=base_model_path,
File "E:\software\stable-diffusion-webui\extensions\facechain\app.py", line 150, in train_lora_fn
raise gr.Error("训练失败 (Training failed)")
gradio.exceptions.Error: '训练失败 (Training failed)'
nikocraft
commented
6 months ago
I can't get facechain to finish training without throwing errors. This is second attempt, here used only 1 image, in first attempt I used 3 images. Running this inside SD Web UI, installed via settings/extensions.