Thank you for your contribution, but I'm having training problems. After loading the environment, run python scepter/tools/run_train py - CFG scepter/methods/scedit/t2i/sd15_512_sce_t2i yaml command, with the following error

[twinkleyang1]

(scepter) twinkle@login-SYS-7049GP-TRT:/data/twinkle/app/scepter$ python scepter/tools/run_train.py --cfg scepter/methods/scedit/t2i/sd15_512_sce_t2i.yaml 2024-09-23 14:57:34,570 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-23 14:57:34,576 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-23 14:57:34,670 - modelscope - INFO - Updating the files for the changes of local files, first time updating will take longer time! Please wait till updating done! 2024-09-23 14:57:34,715 - modelscope - INFO - AST-Scanning the path "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope" with the following sub folders ['models', 'metrics', 'pipelines', 'preprocessors', 'trainers', 'msdatasets', 'exporters'] /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/scepter/modules/inference/control_inference.py:19: UserWarning: Import swift failed, please check it. warnings.warn('Import swift failed, please check it.') 2024-09-23 14:57:34,746 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-23 14:57:34,750 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-23 14:57:34,835 - modelscope - INFO - Updating the files for the changes of local files, first time updating will take longer time! Please wait till updating done! 2024-09-23 14:57:34,880 - modelscope - INFO - AST-Scanning the path "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope" with the following sub folders ['models', 'metrics', 'pipelines', 'preprocessors', 'trainers', 'msdatasets', 'exporters'] /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/scepter/modules/inference/tuner_inference.py:19: UserWarning: Import swift error, please deal with this problem: Failed to import swift.tuners.base because of the following error (look up to see its traceback): During ast indexing the file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/models/audio/aec/layers/activations.py, a related error excepted in the file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/ast_utils.py at line: 151: "attr = getattr(node, field)" with error msg: "AttributeError: 'ClassDef' object has no attribute 'type_params'", please double check the origin file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/models/audio/aec/layers/activations.py to see whether the file is correctly edited. warnings.warn(f'Import swift error, please deal with this problem: {e}') /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') [Info]: Loading config from scepter/methods/scedit/t2i/sd15_512_sce_t2i.yaml [Info]: System take scepter/methods/scedit/t2i/sd15_512_sce_t2i.yaml as yaml, because we find yaml in this file [Info]: SEED is not set and will use default SEED as 2023; If want to change this value, please set it in your config. [Info]: Parse cfg file as { "ENV": { "BACKEND": "nccl", "SEED": 2023 }, "SOLVER": { "NAME": "LatentDiffusionSolver", "RESUME_FROM": null, "LOAD_MODEL_ONLY": true, "USE_FSDP": false, "SHARDING_STRATEGY": null, "USE_AMP": true, "DTYPE": "float16", "CHANNELS_LAST": true, "MAX_STEPS": 2000, "MAX_EPOCHS": -1, "NUM_FOLDS": 1, "ACCU_STEP": 1, "EVAL_INTERVAL": 100, "RESCALE_LR": false, "WORK_DIR": "./cache/save_data/sd15_512_sce_t2i", "LOG_FILE": "std_log.txt", "FILE_SYSTEM": { "NAME": "ModelscopeFs", "TEMP_DIR": "./cache/cache_data" }, "FREEZE": { "FREEZE_PART": [ "first_stage_model", "cond_stage_model", "model" ], "TRAIN_PART": [ "lsc_identity" ] }, "MODEL": { "NAME": "LatentDiffusionSCETuning", "PARAMETERIZATION": "eps", "TIMESTEPS": 1000, "MIN_SNR_GAMMA": null, "ZERO_TERMINAL_SNR": false, "PRETRAINED_MODEL": "ms://AI-ModelScope/stable-diffusion-v1-5@v1-5-pruned-emaonly.safetensors", "IGNORE_KEYS": [], "SCALE_FACTOR": 0.18215, "SIZE_FACTOR": 8, "DEFAULT_N_PROMPT": null, "SCHEDULE_ARGS": { "NAME": "scaled_linear", "BETA_MIN": 0.00085, "BETA_MAX": 0.012 }, "USE_EMA": false, "DIFFUSION_MODEL": { "NAME": "DiffusionUNet", "IN_CHANNELS": 4, "OUT_CHANNELS": 4, "MODEL_CHANNELS": 320, "NUM_HEADS": 8, "NUM_RES_BLOCKS": 2, "ATTENTION_RESOLUTIONS": [ 4, 2, 1 ], "CHANNEL_MULT": [ 1, 2, 4, 4 ], "CONV_RESAMPLE": true, "DIMS": 2, "USE_CHECKPOINT": false, "USE_SCALE_SHIFT_NORM": false, "RESBLOCK_UPDOWN": false, "USE_SPATIAL_TRANSFORMER": true, "TRANSFORMER_DEPTH": 1, "CONTEXT_DIM": 768, "DISABLE_MIDDLE_SELF_ATTN": false, "USE_LINEAR_IN_TRANSFORMER": false, "PRETRAINED_MODEL": null, "IGNORE_KEYS": [] }, "FIRST_STAGE_MODEL": { "NAME": "AutoencoderKL", "EMBED_DIM": 4, "PRETRAINED_MODEL": null, "IGNORE_KEYS": [], "BATCH_SIZE": 4, "ENCODER": { "NAME": "Encoder", "CH": 128, "OUT_CH": 3, "NUM_RES_BLOCKS": 2, "IN_CHANNELS": 3, "ATTN_RESOLUTIONS": [], "CH_MULT": [ 1, 2, 4, 4 ], "Z_CHANNELS": 4, "DOUBLE_Z": true, "DROPOUT": 0.0, "RESAMP_WITH_CONV": true }, "DECODER": { "NAME": "Decoder", "CH": 128, "OUT_CH": 3, "NUM_RES_BLOCKS": 2, "IN_CHANNELS": 3, "ATTN_RESOLUTIONS": [], "CH_MULT": [ 1, 2, 4, 4 ], "Z_CHANNELS": 4, "DROPOUT": 0.0, "RESAMP_WITH_CONV": true, "GIVE_PRE_END": false, "TANH_OUT": false } }, "TOKENIZER": { "NAME": "ClipTokenizer", "PRETRAINED_PATH": "ms://AI-ModelScope/clip-vit-large-patch14", "LENGTH": 77, "CLEAN": true }, "COND_STAGE_MODEL": { "NAME": "FrozenCLIPEmbedder", "FREEZE": true, "LAYER": "last", "PRETRAINED_MODEL": "ms://AI-ModelScope/clip-vit-large-patch14" }, "LOSS": { "NAME": "ReconstructLoss", "LOSS_TYPE": "l2" }, "TUNER_MODEL": { "SC_TUNER_CFG": { "NAME": "SCTuner", "TUNER_NAME": "SCEAdapter", "DOWN_RATIO": 1.0 } } }, "SAMPLE_ARGS": { "SAMPLER": "ddim", "SAMPLE_STEPS": 50, "SEED": 2023, "GUIDE_SCALE": 7.5, "GUIDE_RESCALE": 0.5, "DISCRETIZATION": "trailing", "IMAGE_SIZE": [ 512, 512 ], "RUN_TRAIN_N": false }, "OPTIMIZER": { "NAME": "AdamW", "LEARNING_RATE": 0.0001, "BETAS": [ 0.9, 0.999 ], "EPS": "1e-8", "WEIGHT_DECAY": "1e-2", "AMSGRAD": false }, "TRAIN_DATA": { "NAME": "ImageTextPairMSDataset", "MODE": "train", "MS_DATASET_NAME": "style_custom_dataset", "MS_DATASET_NAMESPACE": "damo", "MS_DATASET_SUBNAME": "3D", "PROMPT_PREFIX": "", "MS_DATASET_SPLIT": "train_short", "MS_REMAP_KEYS": { "Image:FILE": "Target:FILE" }, "REPLACE_STYLE": false, "PIN_MEMORY": true, "BATCH_SIZE": 1, "NUM_WORKERS": 4, "SAMPLER": { "NAME": "LoopSampler" }, "TRANSFORMS": [ { "NAME": "LoadImageFromFile", "RGB_ORDER": "RGB", "BACKEND": "pillow" }, { "NAME": "Resize", "SIZE": 512, "INTERPOLATION": "bilinear", "INPUT_KEY": [ "img" ], "OUTPUT_KEY": [ "img" ], "BACKEND": "pillow" }, { "NAME": "CenterCrop", "SIZE": 512, "INPUT_KEY": [ "img" ], "OUTPUT_KEY": [ "img" ], "BACKEND": "pillow" }, { "NAME": "ImageToTensor", "INPUT_KEY": [ "img" ], "OUTPUT_KEY": [ "img" ], "BACKEND": "pillow" }, { "NAME": "Normalize", "MEAN": [ 0.5, 0.5, 0.5 ], "STD": [ 0.5, 0.5, 0.5 ], "INPUT_KEY": [ "img" ], "OUTPUT_KEY": [ "image" ], "BACKEND": "torchvision" }, { "NAME": "Select", "KEYS": [ "image", "prompt" ], "META_KEYS": [ "data_key" ] } ] }, "EVAL_DATA": { "NAME": "ImageTextPairMSDataset", "MODE": "eval", "MS_DATASET_NAME": "style_custom_dataset", "MS_DATASET_NAMESPACE": "damo", "MS_DATASET_SUBNAME": "3D", "PROMPT_PREFIX": "", "MS_REMAP_KEYS": { "Image": "Target:FILE" }, "MS_DATASET_SPLIT": "test_short", "OUTPUT_SIZE": [ 512, 512 ], "REPLACE_STYLE": false, "PIN_MEMORY": true, "BATCH_SIZE": 4, "NUM_WORKERS": 4, "FILE_SYSTEM": { "NAME": "ModelscopeFs", "TEMP_DIR": "./cache/cache_data" }, "TRANSFORMS": [ { "NAME": "Select", "KEYS": [ "prompt" ], "META_KEYS": [ "image_size" ] } ] }, "TRAIN_HOOKS": [ { "NAME": "BackwardHook", "PRIORITY": 0 }, { "NAME": "LogHook", "LOG_INTERVAL": 50 }, { "NAME": "CheckpointHook", "INTERVAL": 1000 }, { "NAME": "ProbeDataHook", "PROB_INTERVAL": 100 } ], "EVAL_HOOKS": [ { "NAME": "ProbeDataHook", "PROB_INTERVAL": 100 } ] } } 2024-09-23 14:57:46,582 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-23 14:57:46,587 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-23 14:57:46,678 - modelscope - INFO - Updating the files for the changes of local files, first time updating will take longer time! Please wait till updating done! 2024-09-23 14:57:46,722 - modelscope - INFO - AST-Scanning the path "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope" with the following sub folders ['models', 'metrics', 'pipelines', 'preprocessors', 'trainers', 'msdatasets', 'exporters'] /data/twinkle/app/scepter/scepter/modules/inference/control_inference.py:19: UserWarning: Import swift failed, please check it. warnings.warn('Import swift failed, please check it.') 2024-09-23 14:57:46,760 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-23 14:57:46,766 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-23 14:57:46,869 - modelscope - INFO - Updating the files for the changes of local files, first time updating will take longer time! Please wait till updating done! 2024-09-23 14:57:46,919 - modelscope - INFO - AST-Scanning the path "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope" with the following sub folders ['models', 'metrics', 'pipelines', 'preprocessors', 'trainers', 'msdatasets', 'exporters'] /data/twinkle/app/scepter/scepter/modules/inference/tuner_inference.py:19: UserWarning: Import swift error, please deal with this problem: Failed to import swift.tuners.base because of the following error (look up to see its traceback): During ast indexing the file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/models/audio/aec/layers/activations.py, a related error excepted in the file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/ast_utils.py at line: 151: "attr = getattr(node, field)" with error msg: "AttributeError: 'ClassDef' object has no attribute 'type_params'", please double check the origin file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/models/audio/aec/layers/activations.py to see whether the file is correctly edited. warnings.warn(f'Import swift error, please deal with this problem: {e}') 2024-09-23 14:57:46,974 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-23 14:57:46,982 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-23 14:57:47,087 - modelscope - INFO - Updating the files for the changes of local files, first time updating will take longer time! Please wait till updating done! 2024-09-23 14:57:47,144 - modelscope - INFO - AST-Scanning the path "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope" with the following sub folders ['models', 'metrics', 'pipelines', 'preprocessors', 'trainers', 'msdatasets', 'exporters'] /data/twinkle/app/scepter/scepter/modules/inference/control_inference.py:19: UserWarning: Import swift failed, please check it. warnings.warn('Import swift failed, please check it.') 2024-09-23 14:57:47,189 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-23 14:57:47,194 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-23 14:57:47,297 - modelscope - INFO - Updating the files for the changes of local files, first time updating will take longer time! Please wait till updating done! 2024-09-23 14:57:47,298 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-23 14:57:47,305 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-23 14:57:47,347 - modelscope - INFO - AST-Scanning the path "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope" with the following sub folders ['models', 'metrics', 'pipelines', 'preprocessors', 'trainers', 'msdatasets', 'exporters'] /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/inference/tuner_inference.py:19: UserWarning: Import swift error, please deal with this problem: Failed to import swift.tuners.base because of the following error (look up to see its traceback): During ast indexing the file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/models/audio/aec/layers/activations.py, a related error excepted in the file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/ast_utils.py at line: 151: "attr = getattr(node, field)" with error msg: "AttributeError: 'ClassDef' object has no attribute 'type_params'", please double check the origin file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/models/audio/aec/layers/activations.py to see whether the file is correctly edited. warnings.warn(f'Import swift error, please deal with this problem: {e}') 2024-09-23 14:57:47,403 - modelscope - INFO - Updating the files for the changes of local files, first time updating will take longer time! Please wait till updating done! 2024-09-23 14:57:47,449 - modelscope - INFO - AST-Scanning the path "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope" with the following sub folders ['models', 'metrics', 'pipelines', 'preprocessors', 'trainers', 'msdatasets', 'exporters'] /data/twinkle/app/scepter/scepter/modules/inference/control_inference.py:19: UserWarning: Import swift failed, please check it. warnings.warn('Import swift failed, please check it.') 2024-09-23 14:57:47,484 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-23 14:57:47,488 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-23 14:57:47,586 - modelscope - INFO - Updating the files for the changes of local files, first time updating will take longer time! Please wait till updating done! 2024-09-23 14:57:47,634 - modelscope - INFO - AST-Scanning the path "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope" with the following sub folders ['models', 'metrics', 'pipelines', 'preprocessors', 'trainers', 'msdatasets', 'exporters'] /data/twinkle/app/scepter/scepter/modules/inference/tuner_inference.py:19: UserWarning: Import swift error, please deal with this problem: Failed to import swift.tuners.base because of the following error (look up to see its traceback): During ast indexing the file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/models/audio/aec/layers/activations.py, a related error excepted in the file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/ast_utils.py at line: 151: "attr = getattr(node, field)" with error msg: "AttributeError: 'ClassDef' object has no attribute 'type_params'", please double check the origin file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/models/audio/aec/layers/activations.py to see whether the file is correctly edited. warnings.warn(f'Import swift error, please deal with this problem: {e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') 2024-09-23 14:57:48,815 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-23 14:57:48,820 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-23 14:57:48,919 - modelscope - INFO - Updating the files for the changes of local files, first time updating will take longer time! Please wait till updating done! 2024-09-23 14:57:48,965 - modelscope - INFO - AST-Scanning the path "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope" with the following sub folders ['models', 'metrics', 'pipelines', 'preprocessors', 'trainers', 'msdatasets', 'exporters'] /data/twinkle/app/scepter/scepter/modules/inference/control_inference.py:19: UserWarning: Import swift failed, please check it. warnings.warn('Import swift failed, please check it.') 2024-09-23 14:57:49,012 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-23 14:57:49,018 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-23 14:57:49,119 - modelscope - INFO - Updating the files for the changes of local files, first time updating will take longer time! Please wait till updating done! 2024-09-23 14:57:49,168 - modelscope - INFO - AST-Scanning the path "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope" with the following sub folders ['models', 'metrics', 'pipelines', 'preprocessors', 'trainers', 'msdatasets', 'exporters'] /data/twinkle/app/scepter/scepter/modules/inference/tuner_inference.py:19: UserWarning: Import swift error, please deal with this problem: Failed to import swift.tuners.base because of the following error (look up to see its traceback): During ast indexing the file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/models/audio/aec/layers/activations.py, a related error excepted in the file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/ast_utils.py at line: 151: "attr = getattr(node, field)" with error msg: "AttributeError: 'ClassDef' object has no attribute 'type_params'", please double check the origin file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/models/audio/aec/layers/activations.py to see whether the file is correctly edited. warnings.warn(f'Import swift error, please deal with this problem: {e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') [Info]: Now running in the distributed environment with world size 4! [Info]: PMI rank 0! [Info]: Nums of gpu 4! [Info]: Current rank 2 current devices num 4 current machine rank 0 and all world size 4 [Info]: Now running in the distributed environment with world size 4! [Info]: PMI rank 0! [Info]: Nums of gpu 4! [Info]: Current rank 3 current devices num 4 current machine rank 0 and all world size 4 [Info]: Now running in the distributed environment with world size 4! [Info]: PMI rank 0! [Info]: Nums of gpu 4! [Info]: Current rank 0 current devices num 4 current machine rank 0 and all world size 4 scepter [INFO] 2024-09-23 14:57:49,453 [File: base_solver.py Function: init at line 308] Parse work dir ./cache/save_data/sd15_512_sce_t2i's prefix is /data/twinkle/app/scepter scepter [INFO] 2024-09-23 14:57:49,454 [File: diffusion_solver.py Function: init at line 113] Use default backend. [Info]: Now running in the distributed environment with world size 4! [Info]: PMI rank 0! [Info]: Nums of gpu 4! [Info]: Current rank 1 current devices num 4 current machine rank 0 and all world size 4 scepter [INFO] 2024-09-23 14:57:49,463 [File: base_solver.py Function: init at line 308] Parse work dir ./cache/save_data/sd15_512_sce_t2i's prefix is /data/twinkle/app/scepter scepter [INFO] 2024-09-23 14:57:49,463 [File: diffusion_solver.py Function: init at line 113] Use default backend. scepter [INFO] 2024-09-23 14:57:49,467 [File: base_solver.py Function: init at line 308] Parse work dir ./cache/save_data/sd15_512_sce_t2i's prefix is /data/twinkle/app/scepter scepter [INFO] 2024-09-23 14:57:49,467 [File: diffusion_solver.py Function: init at line 113] Use default backend. scepter [INFO] 2024-09-23 14:57:49,469 [File: base_solver.py Function: init at line 308] Parse work dir ./cache/save_data/sd15_512_sce_t2i's prefix is /data/twinkle/app/scepter scepter [INFO] 2024-09-23 14:57:49,470 [File: diffusion_solver.py Function: init at line 113] Use default backend. scepter [INFO] 2024-09-23 14:57:49,471 [File: logger.py Function: init_logger at line 86] Running task with log file: /data/twinkle/app/scepter/./cache/save_data/sd15_512_sce_t2i/std_log.txt 2024-09-23 14:57:49,474 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-23 14:57:49,481 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-23 14:57:49,487 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-23 14:57:49,490 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-23 14:57:49,491 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-23 14:57:49,492 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-23 14:57:49,496 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-23 14:57:49,497 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-23 14:57:49,584 - modelscope - INFO - Updating the files for the changes of local files, first time updating will take longer time! Please wait till updating done! 2024-09-23 14:57:49,588 - modelscope - INFO - Updating the files for the changes of local files, first time updating will take longer time! Please wait till updating done! 2024-09-23 14:57:49,601 - modelscope - INFO - Updating the files for the changes of local files, first time updating will take longer time! Please wait till updating done! 2024-09-23 14:57:49,630 - modelscope - INFO - AST-Scanning the path "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope" with the following sub folders ['models', 'metrics', 'pipelines', 'preprocessors', 'trainers', 'msdatasets', 'exporters'] 2024-09-23 14:57:49,631 - modelscope - INFO - AST-Scanning the path "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope" with the following sub folders ['models', 'metrics', 'pipelines', 'preprocessors', 'trainers', 'msdatasets', 'exporters'] 2024-09-23 14:57:49,646 - modelscope - INFO - AST-Scanning the path "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope" with the following sub folders ['models', 'metrics', 'pipelines', 'preprocessors', 'trainers', 'msdatasets', 'exporters'] 2024-09-23 14:57:49,708 - modelscope - INFO - Updating the files for the changes of local files, first time updating will take longer time! Please wait till updating done! 2024-09-23 14:57:49,750 - modelscope - INFO - AST-Scanning the path "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope" with the following sub folders ['models', 'metrics', 'pipelines', 'preprocessors', 'trainers', 'msdatasets', 'exporters'] Traceback (most recent call last): File "scepter/tools/run_train.py", line 62, in run() File "scepter/tools/run_train.py", line 58, in run we.init_env(cfg, logger=None, fn=run_task) File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/scepter/modules/utils/distribute.py", line 431, in init_env mp.spawn(mp_worker, File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 239, in spawn return start_processes(fn, args, nprocs, join, daemon, start_method='spawn') File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 197, in start_processes while not context.join(): File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 160, in join raise ProcessRaisedException(msg, error_index, failed_process.pid) torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 3 terminated with the following error: Traceback (most recent call last): File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/ast_utils.py", line 466, in _get_single_file_scan_result output = self.astScaner.generate_ast(file) File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/ast_utils.py", line 365, in generate_ast output = self.scan_import(node, show_offsets=False) File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/ast_utils.py", line 164, in scan_import local_out = _scan_import(el, type(el).name) File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/ast_utils.py", line 133, in _scan_import return self.scan_import( File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/ast_utils.py", line 151, in scan_import attr = getattr(node, field) AttributeError: 'ClassDef' object has no attribute 'type_params'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/data/twinkle/app/scepter/scepter/modules/data/dataset/registry.py", line 353, in build_dataset_config dataset = req_type_entry(cfg, logger=logger, *args, **kwargs) File "/data/twinkle/app/scepter/scepter/modules/data/dataset/ms_dataset.py", line 87, in init from modelscope import MsDataset File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/init.py", line 4, in from modelscope.utils.import_utils import LazyImportModule File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/init.py", line 1, in from .hub import create_model_if_not_exist, read_config File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/hub.py", line 12, in from modelscope.utils.config import Config File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/config.py", line 22, in from modelscope.utils.import_utils import import_modules_from_file File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/import_utils.py", line 380, in class LazyImportModule(ModuleType): File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/import_utils.py", line 383, in LazyImportModule AST_INDEX = load_index() File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/ast_utils.py", line 722, in load_index _update_index(index, files_mtime) File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/ast_utils.py", line 638, in _update_index updated_index = file_scanner.get_files_scan_results(updated_files) File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/ast_utils.py", line 532, in get_files_scan_results decorator_list, import_list = self._get_single_file_scan_result( File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/ast_utils.py", line 469, in _get_single_file_scan_result raise Exception( Exception: During ast indexing the file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/models/audio/aec/layers/activations.py, a related error excepted in the file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/ast_utils.py at line: 151: "attr = getattr(node, field)" with error msg: "AttributeError: 'ClassDef' object has no attribute 'type_params'", please double check the origin file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/models/audio/aec/layers/activations.py to see whether the file is correctly edited.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap fn(i, *args) File "/data/twinkle/app/scepter/scepter/modules/utils/distribute.py", line 314, in mp_worker fn(cfg) File "/data/twinkle/app/scepter/scepter/tools/run_train.py", line 25, in run_task solver.set_up() File "/data/twinkle/app/scepter/scepter/modules/solver/diffusion_solver.py", line 125, in set_up self.construct_data() File "/data/twinkle/app/scepter/scepter/modules/solver/diffusion_solver.py", line 146, in construct_data train_data = DATASETS.build(self.cfg.TRAIN_DATA, File "/data/twinkle/app/scepter/scepter/modules/utils/registry.py", line 135, in build return self.build_func(cfg, File "/data/twinkle/app/scepter/scepter/modules/data/dataset/registry.py", line 357, in build_dataset_config raise Exception(f'Failed to init class {req_type_entry}, with {e}') Exception: Failed to init class <class 'scepter.modules.data.dataset.ms_dataset.ImageTextPairMSDataset'>, with During ast indexing the file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/models/audio/aec/layers/activations.py, a related error excepted in the file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/utils/ast_utils.py at line: 151: "attr = getattr(node, field)" with error msg: "AttributeError: 'ClassDef' object has no attribute 'type_params'", please double check the origin file /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/modelscope/models/audio/aec/layers/activations.py to see whether the file is correctly edited.

[yaosheng216]

@twinkleyang1 Hi, on the my development environment, not met this problem, please ensure your modelscope and ms-swift package of same version with me, below is my dependent package information.

[twinkleyang1]

Thank you for your answer！I removed the environment and after reinstalling it ran successfully: Python scepter/tools/run_train. Py -- -- CFG scepter/methods/scedit/t2i/sd15_512_sce_t2i yaml, but when I want to run a python Scepter/tools/run_train. Py -- -- CFG scepter/methods/scedit/CTR/sd15_512_sce_ctr_hed yaml appeared the following error:(scepter) twinkle@login-SYS-7049GP-TRT:/data/twinkle/app/scepter$ python scepter/tools/run_train.py --cfg scepter/methods/scedit/ctr/sd15_512_sce_ctr_hed.yaml 2024-09-26 12:01:51,390 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:01:51,391 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:01:51,428 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed /data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') [Info]: Loading config from scepter/methods/scedit/ctr/sd15_512_sce_ctr_hed.yaml [Info]: System take scepter/methods/scedit/ctr/sd15_512_sce_ctr_hed.yaml as yaml, because we find yaml in this file [Info]: SEED is not set and will use default SEED as 2023; If want to change this value, please set it in your config. [Info]: Parse cfg file as { "ENV": { "BACKEND": "nccl", "SEED": 2023 }, "SOLVER": { "NAME": "LatentDiffusionSolver", "RESUME_FROM": null, "LOAD_MODEL_ONLY": true, "USE_FSDP": false, "SHARDING_STRATEGY": null, "USE_AMP": true, "DTYPE": "float16", "CHANNELS_LAST": true, "MAX_STEPS": 200, "MAX_EPOCHS": -1, "NUM_FOLDS": 1, "ACCU_STEP": 1, "EVAL_INTERVAL": 100, "RESCALE_LR": false, "WORK_DIR": "./cache/save_data/sd15_512_sce_ctr_hed", "LOG_FILE": "std_log.txt", "FILE_SYSTEM": { "NAME": "ModelscopeFs", "TEMP_DIR": "./cache/cache_data" }, "FREEZE": { "FREEZE_PART": [ "first_stage_model", "cond_stage_model", "model" ], "TRAIN_PART": [ "control_blocks" ] }, "MODEL": { "NAME": "LatentDiffusionSCEControl", "PARAMETERIZATION": "eps", "TIMESTEPS": 1000, "MIN_SNR_GAMMA": null, "ZERO_TERMINAL_SNR": false, "PRETRAINED_MODEL": "ms://AI-ModelScope/stable-diffusion-v1-5@v1-5-pruned-emaonly.safetensors", "IGNORE_KEYS": [], "SCALE_FACTOR": 0.18215, "SIZE_FACTOR": 8, "DEFAULT_N_PROMPT": null, "SCHEDULE_ARGS": { "NAME": "scaled_linear", "BETA_MIN": 0.00085, "BETA_MAX": 0.012 }, "USE_EMA": false, "DIFFUSION_MODEL": { "NAME": "DiffusionUNet", "IN_CHANNELS": 4, "OUT_CHANNELS": 4, "MODEL_CHANNELS": 320, "NUM_HEADS": 8, "NUM_RES_BLOCKS": 2, "ATTENTION_RESOLUTIONS": [ 4, 2, 1 ], "CHANNEL_MULT": [ 1, 2, 4, 4 ], "CONV_RESAMPLE": true, "DIMS": 2, "USE_CHECKPOINT": false, "USE_SCALE_SHIFT_NORM": false, "RESBLOCK_UPDOWN": false, "USE_SPATIAL_TRANSFORMER": true, "TRANSFORMER_DEPTH": 1, "CONTEXT_DIM": 768, "DISABLE_MIDDLE_SELF_ATTN": false, "USE_LINEAR_IN_TRANSFORMER": false, "PRETRAINED_MODEL": null, "IGNORE_KEYS": [] }, "FIRST_STAGE_MODEL": { "NAME": "AutoencoderKL", "EMBED_DIM": 4, "PRETRAINED_MODEL": null, "IGNORE_KEYS": [], "BATCH_SIZE": 4, "ENCODER": { "NAME": "Encoder", "CH": 128, "OUT_CH": 3, "NUM_RES_BLOCKS": 2, "IN_CHANNELS": 3, "ATTN_RESOLUTIONS": [], "CH_MULT": [ 1, 2, 4, 4 ], "Z_CHANNELS": 4, "DOUBLE_Z": true, "DROPOUT": 0.0, "RESAMP_WITH_CONV": true }, "DECODER": { "NAME": "Decoder", "CH": 128, "OUT_CH": 3, "NUM_RES_BLOCKS": 2, "IN_CHANNELS": 3, "ATTN_RESOLUTIONS": [], "CH_MULT": [ 1, 2, 4, 4 ], "Z_CHANNELS": 4, "DROPOUT": 0.0, "RESAMP_WITH_CONV": true, "GIVE_PRE_END": false, "TANH_OUT": false } }, "TOKENIZER": { "NAME": "ClipTokenizer", "PRETRAINED_PATH": "ms://AI-ModelScope/clip-vit-large-patch14", "LENGTH": 77, "CLEAN": true }, "COND_STAGE_MODEL": { "NAME": "FrozenCLIPEmbedder", "FREEZE": true, "LAYER": "last", "PRETRAINED_MODEL": "ms://AI-ModelScope/clip-vit-large-patch14" }, "LOSS": { "NAME": "ReconstructLoss", "LOSS_TYPE": "l2" }, "CONTROL_MODEL": { "NAME": "CSCTuners", "PRE_HINT_IN_CHANNELS": 3, "PRE_HINT_OUT_CHANNELS": 256, "DENSE_HINT_KERNAL": 3, "SCALE": 1.0, "SC_TUNER_CFG": { "NAME": "SCTuner", "TUNER_NAME": "SCEAdapter", "DOWN_RATIO": 1.0 } }, "CONTROL_ANNO": { "NAME": "HedAnnotator", "PRETRAINED_MODEL": "ms://iic/scepter_scedit@annotator/ckpts/ControlNetHED.pth" } }, "SAMPLE_ARGS": { "SAMPLER": "ddim", "SAMPLE_STEPS": 50, "SEED": 2023, "GUIDE_SCALE": 7.5, "GUIDE_RESCALE": 0.5, "DISCRETIZATION": "trailing", "IMAGE_SIZE": [ 512, 512 ], "RUN_TRAIN_N": false }, "OPTIMIZER": { "NAME": "AdamW", "LEARNING_RATE": 0.0001, "BETAS": [ 0.9, 0.999 ], "EPS": "1e-8", "WEIGHT_DECAY": "1e-2", "AMSGRAD": false }, "TRAIN_DATA": { "NAME": "ImageTextPairMSDataset", "MODE": "train", "MS_DATASET_NAME": "style_custom_dataset", "MS_DATASET_NAMESPACE": "damo", "MS_DATASET_SUBNAME": "3D", "PROMPT_PREFIX": "", "MS_DATASET_SPLIT": "train_short", "MS_REMAP_KEYS": { "Image:FILE": "Target:FILE" }, "REPLACE_STYLE": false, "PIN_MEMORY": true, "BATCH_SIZE": 1, "NUM_WORKERS": 4, "SAMPLER": { "NAME": "LoopSampler" }, "TRANSFORMS": [ { "NAME": "LoadImageFromFile", "RGB_ORDER": "RGB", "BACKEND": "pillow" }, { "NAME": "Resize", "SIZE": 768, "INTERPOLATION": "bilinear", "INPUT_KEY": [ "img" ], "OUTPUT_KEY": [ "img" ], "BACKEND": "pillow" }, { "NAME": "CenterCrop", "SIZE": 768, "INPUT_KEY": [ "img" ], "OUTPUT_KEY": [ "img" ], "BACKEND": "pillow" }, { "NAME": "ToNumpy", "INPUT_KEY": [ "img" ], "OUTPUT_KEY": [ "image_preprocess" ] }, { "NAME": "ImageToTensor", "INPUT_KEY": [ "img" ], "OUTPUT_KEY": [ "img" ], "BACKEND": "pillow" }, { "NAME": "Normalize", "MEAN": [ 0.5, 0.5, 0.5 ], "STD": [ 0.5, 0.5, 0.5 ], "INPUT_KEY": [ "img" ], "OUTPUT_KEY": [ "img" ], "BACKEND": "torchvision" }, { "NAME": "Rename", "INPUT_KEY": [ "img", "image_preprocess" ], "OUTPUT_KEY": [ "image", "image_preprocess" ] }, { "NAME": "Select", "KEYS": [ "image", "prompt", "image_preprocess" ], "META_KEYS": [ "data_key" ] } ] }, "EVAL_DATA": { "NAME": "ImageTextPairMSDataset", "MODE": "eval", "MS_DATASET_NAME": "style_custom_dataset", "MS_DATASET_NAMESPACE": "damo", "MS_DATASET_SUBNAME": "3D", "PROMPT_PREFIX": "", "MS_DATASET_SPLIT": "train_short", "MS_REMAP_KEYS": { "Image:FILE": "Target:FILE" }, "REPLACE_STYLE": false, "PIN_MEMORY": true, "BATCH_SIZE": 10, "NUM_WORKERS": 4, "TRANSFORMS": [ { "NAME": "LoadImageFromFile", "RGB_ORDER": "RGB", "BACKEND": "pillow" }, { "NAME": "Resize", "SIZE": 768, "INTERPOLATION": "bilinear", "INPUT_KEY": [ "img" ], "OUTPUT_KEY": [ "img" ], "BACKEND": "pillow" }, { "NAME": "CenterCrop", "SIZE": 768, "INPUT_KEY": [ "img" ], "OUTPUT_KEY": [ "img" ], "BACKEND": "pillow" }, { "NAME": "ToNumpy", "INPUT_KEY": [ "img" ], "OUTPUT_KEY": [ "image_preprocess" ] }, { "NAME": "ImageToTensor", "INPUT_KEY": [ "img" ], "OUTPUT_KEY": [ "img" ], "BACKEND": "pillow" }, { "NAME": "Normalize", "MEAN": [ 0.5, 0.5, 0.5 ], "STD": [ 0.5, 0.5, 0.5 ], "INPUT_KEY": [ "img" ], "OUTPUT_KEY": [ "img" ], "BACKEND": "torchvision" }, { "NAME": "Rename", "INPUT_KEY": [ "img", "image_preprocess" ], "OUTPUT_KEY": [ "image", "image_preprocess" ] }, { "NAME": "Select", "KEYS": [ "image", "prompt", "image_preprocess" ], "META_KEYS": [ "data_key" ] } ] }, "TRAIN_HOOKS": [ { "NAME": "BackwardHook", "PRIORITY": 0 }, { "NAME": "LogHook", "LOG_INTERVAL": 50 }, { "NAME": "CheckpointHook", "INTERVAL": 100 }, { "NAME": "ProbeDataHook", "PROB_INTERVAL": 100 } ], "EVAL_HOOKS": [ { "NAME": "ProbeDataHook", "PROB_INTERVAL": 100 } ] } } 2024-09-26 12:01:57,840 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:01:57,842 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:01:57,861 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:01:57,862 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:01:57,877 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:01:57,897 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:01:57,929 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:01:57,930 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:01:57,966 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:01:58,001 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:01:58,002 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:01:58,039 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') [Info]: Now running in the distributed environment with world size 4! [Info]: PMI rank 0! [Info]: Nums of gpu 4! [Info]: Current rank 0 current devices num 4 current machine rank 0 and all world size 4 [Info]: Now running in the distributed environment with world size 4! [Info]: PMI rank 0! [Info]: Nums of gpu 4! [Info]: Current rank 1 current devices num 4 current machine rank 0 and all world size 4 scepter [INFO] 2024-09-26 12:01:59,583 [File: base_solver.py Function: init at line 308] Parse work dir ./cache/save_data/sd15_512_sce_ctr_hed's prefix is /data/twinkle/app/scepter scepter [INFO] 2024-09-26 12:01:59,584 [File: diffusion_solver.py Function: init at line 113] Use default backend. [Info]: Now running in the distributed environment with world size 4! [Info]: PMI rank 0! [Info]: Nums of gpu 4! [Info]: Current rank 2 current devices num 4 current machine rank 0 and all world size 4 scepter [INFO] 2024-09-26 12:01:59,585 [File: base_solver.py Function: init at line 308] Parse work dir ./cache/save_data/sd15_512_sce_ctr_hed's prefix is /data/twinkle/app/scepter scepter [INFO] 2024-09-26 12:01:59,585 [File: diffusion_solver.py Function: init at line 113] Use default backend. scepter [INFO] 2024-09-26 12:01:59,585 [File: logger.py Function: init_logger at line 86] Running task with log file: /data/twinkle/app/scepter/./cache/save_data/sd15_512_sce_ctr_hed/std_log.txt [Info]: Now running in the distributed environment with world size 4! [Info]: PMI rank 0! [Info]: Nums of gpu 4! [Info]: Current rank 3 current devices num 4 current machine rank 0 and all world size 4 scepter [INFO] 2024-09-26 12:01:59,591 [File: base_solver.py Function: init at line 308] Parse work dir ./cache/save_data/sd15_512_sce_ctr_hed's prefix is /data/twinkle/app/scepter scepter [INFO] 2024-09-26 12:01:59,592 [File: diffusion_solver.py Function: init at line 113] Use default backend. scepter [INFO] 2024-09-26 12:01:59,592 [File: base_solver.py Function: init at line 308] Parse work dir ./cache/save_data/sd15_512_sce_ctr_hed's prefix is /data/twinkle/app/scepter scepter [INFO] 2024-09-26 12:01:59,592 [File: diffusion_solver.py Function: init at line 113] Use default backend. 2024-09-26 12:02:00,049 - modelscope - INFO - dataset_type: 2 2024-09-26 12:02:00,075 - modelscope - INFO - dataset_type: 2 2024-09-26 12:02:00,092 - modelscope - INFO - dataset_type: 2 2024-09-26 12:02:00,111 - modelscope - INFO - dataset_type: 2 2024-09-26 12:02:00,896 - modelscope - WARNING - Reusing dataset dataset_builder (/home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files) 2024-09-26 12:02:00,896 - modelscope - INFO - Generating dataset dataset_builder (/home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files) 2024-09-26 12:02:00,896 - modelscope - INFO - Reusing cached meta-data file: /home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files/20773d60e56183e65c9ac537a8adcd83 2024-09-26 12:02:02,050 - modelscope - WARNING - Reusing dataset dataset_builder (/home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files) 2024-09-26 12:02:02,050 - modelscope - INFO - Generating dataset dataset_builder (/home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files) 2024-09-26 12:02:02,050 - modelscope - INFO - Reusing cached meta-data file: /home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files/20773d60e56183e65c9ac537a8adcd83 2024-09-26 12:02:02,289 - modelscope - WARNING - Reusing dataset dataset_builder (/home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files) 2024-09-26 12:02:02,290 - modelscope - INFO - Generating dataset dataset_builder (/home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files) 2024-09-26 12:02:02,290 - modelscope - INFO - Reusing cached meta-data file: /home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files/20773d60e56183e65c9ac537a8adcd83 scepter [INFO] 2024-09-26 12:02:02,327 [File: registry.py Function: init at line 185] Built dataloader with len 9223372036854775807 scepter [INFO] 2024-09-26 12:02:02,598 [File: registry.py Function: init at line 185] Built dataloader with len 9223372036854775807 2024-09-26 12:02:02,616 - modelscope - WARNING - Reusing dataset dataset_builder (/home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files) 2024-09-26 12:02:02,616 - modelscope - INFO - Generating dataset dataset_builder (/home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files) 2024-09-26 12:02:02,616 - modelscope - INFO - Reusing cached meta-data file: /home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files/20773d60e56183e65c9ac537a8adcd83 2024-09-26 12:02:02,708 - modelscope - INFO - dataset_type: 2 scepter [INFO] 2024-09-26 12:02:02,899 [File: registry.py Function: init at line 185] Built dataloader with len 9223372036854775807 2024-09-26 12:02:02,969 - modelscope - INFO - dataset_type: 2 scepter [INFO] 2024-09-26 12:02:03,242 [File: registry.py Function: init at line 185] Built dataloader with len 9223372036854775807 2024-09-26 12:02:03,292 - modelscope - INFO - dataset_type: 2 2024-09-26 12:02:03,441 - modelscope - WARNING - Reusing dataset dataset_builder (/home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files) 2024-09-26 12:02:03,441 - modelscope - INFO - Generating dataset dataset_builder (/home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files) 2024-09-26 12:02:03,441 - modelscope - INFO - Reusing cached meta-data file: /home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files/20773d60e56183e65c9ac537a8adcd83 2024-09-26 12:02:03,611 - modelscope - INFO - dataset_type: 2 2024-09-26 12:02:03,775 - modelscope - WARNING - Reusing dataset dataset_builder (/home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files) 2024-09-26 12:02:03,775 - modelscope - INFO - Generating dataset dataset_builder (/home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files) 2024-09-26 12:02:03,775 - modelscope - INFO - Reusing cached meta-data file: /home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files/20773d60e56183e65c9ac537a8adcd83 scepter [INFO] 2024-09-26 12:02:04,086 [File: registry.py Function: init at line 185] Built dataloader with len 1 2024-09-26 12:02:04,128 - modelscope - WARNING - Reusing dataset dataset_builder (/home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files) 2024-09-26 12:02:04,128 - modelscope - INFO - Generating dataset dataset_builder (/home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files) 2024-09-26 12:02:04,128 - modelscope - INFO - Reusing cached meta-data file: /home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files/20773d60e56183e65c9ac537a8adcd83 scepter [INFO] 2024-09-26 12:02:04,372 [File: registry.py Function: init at line 185] Built dataloader with len 1 2024-09-26 12:02:04,454 - modelscope - WARNING - Reusing dataset dataset_builder (/home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files) 2024-09-26 12:02:04,455 - modelscope - INFO - Generating dataset dataset_builder (/home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files) 2024-09-26 12:02:04,455 - modelscope - INFO - Reusing cached meta-data file: /home/twinkle/.cache/modelscope/hub/datasets/damo/style_custom_dataset/master/data_files/20773d60e56183e65c9ac537a8adcd83 scepter [INFO] 2024-09-26 12:02:04,676 [File: registry.py Function: init at line 185] Built dataloader with len 1 scepter [INFO] 2024-09-26 12:02:04,989 [File: registry.py Function: init at line 185] Built dataloader with len 1 scepter [INFO] 2024-09-26 12:02:10,988 [File: ldm.py Function: construct_network at line 147] all parameters:859.52M scepter [INFO] 2024-09-26 12:02:11,495 [File: ldm.py Function: construct_network at line 147] all parameters:859.52M scepter [INFO] 2024-09-26 12:02:11,685 [File: ldm.py Function: construct_network at line 147] all parameters:859.52M scepter [INFO] 2024-09-26 12:02:11,797 [File: ae_module.py Function: construct_model at line 76] AE Module XFORMERS_IS_AVAILBLE : True scepter [INFO] 2024-09-26 12:02:12,096 [File: ldm.py Function: construct_network at line 147] all parameters:859.52M scepter [INFO] 2024-09-26 12:02:12,258 [File: ae_module.py Function: construct_model at line 76] AE Module XFORMERS_IS_AVAILBLE : True scepter [INFO] 2024-09-26 12:02:12,399 [File: ae_module.py Function: construct_model at line 76] AE Module XFORMERS_IS_AVAILBLE : True scepter [INFO] 2024-09-26 12:02:12,899 [File: ae_module.py Function: construct_model at line 76] AE Module XFORMERS_IS_AVAILBLE : True 2024-09-26 12:02:15,732 - modelscope - WARNING - Model revision not specified, use revision: v1.0.9 2024-09-26 12:02:16,374 - modelscope - WARNING - Model revision not specified, use revision: v1.0.9 2024-09-26 12:02:16,592 - modelscope - WARNING - Model revision not specified, use revision: v1.0.9 2024-09-26 12:02:17,177 - modelscope - WARNING - Model revision not specified, use revision: v1.0.9 scepter [INFO] 2024-09-26 12:02:19,573 [File: diffusion_solver.py Function: print_model_params_status at line 745] Load trainable params 113827928 / 1180063235 = 9.65%, train part: {'control_blocks.0': 99111760, 'netNetwork.norm': 3, 'netNetwork.block1': 38785, 'netNetwork.block2': 221569, 'netNetwork.block3': 1475585, 'netNetwork.block4': 5900289, 'netNetwork.block5': 7079937}. scepter [INFO] 2024-09-26 12:02:19,573 [File: diffusion_solver.py Function: print_model_params_status at line 749] Load forzen params 1066235307 / 1180063235 = 90.35%, forzen part: {'time_embed': 2050560, 'input_blocks': 249624320, 'middle_block': 97038080, 'output_blocks': 510795840, 'out': 12164, 'encoder': 34163592, 'decoder': 49490179, 'conv1': 72, 'conv2': 20, 'transformer': 123060480}. scepter [INFO] 2024-09-26 12:02:22,433 [File: ldm.py Function: init_from_ckpt at line 218] Restored from ./cache/cache_data/95c8c1dcca842facbdcb9606c314f7ae/AI-ModelScope/stable-diffusion-v1-5/v1-5-pruned-emaonly.safetensors with 123 missing and 15 unexpected keys scepter [INFO] 2024-09-26 12:02:22,438 [File: ldm.py Function: init_from_ckpt at line 222] Missing Keys: ['model.control_blocks.0.pre_hint_blocks.0.weight', 'model.control_blocks.0.pre_hint_blocks.0.bias', 'model.control_blocks.0.pre_hint_blocks.2.weight', 'model.control_blocks.0.pre_hint_blocks.2.bias', 'model.control_blocks.0.pre_hint_blocks.4.weight', 'model.control_blocks.0.pre_hint_blocks.4.bias', 'model.control_blocks.0.pre_hint_blocks.6.weight', 'model.control_blocks.0.pre_hint_blocks.6.bias', 'model.control_blocks.0.pre_hint_blocks.8.weight', 'model.control_blocks.0.pre_hint_blocks.8.bias', 'model.control_blocks.0.pre_hint_blocks.10.weight', 'model.control_blocks.0.pre_hint_blocks.10.bias', 'model.control_blocks.0.pre_hint_blocks.12.weight', 'model.control_blocks.0.pre_hint_blocks.12.bias', 'model.control_blocks.0.dense_hint_blocks.0.1.weight', 'model.control_blocks.0.dense_hint_blocks.0.1.bias', 'model.control_blocks.0.dense_hint_blocks.1.1.weight', 'model.control_blocks.0.dense_hint_blocks.1.1.bias', 'model.control_blocks.0.dense_hint_blocks.2.1.weight', 'model.control_blocks.0.dense_hint_blocks.2.1.bias', 'model.control_blocks.0.dense_hint_blocks.3.1.weight', 'model.control_blocks.0.dense_hint_blocks.3.1.bias', 'model.control_blocks.0.dense_hint_blocks.4.1.weight', 'model.control_blocks.0.dense_hint_blocks.4.1.bias', 'model.control_blocks.0.dense_hint_blocks.5.1.weight', 'model.control_blocks.0.dense_hint_blocks.5.1.bias', 'model.control_blocks.0.dense_hint_blocks.6.1.weight', 'model.control_blocks.0.dense_hint_blocks.6.1.bias', 'model.control_blocks.0.dense_hint_blocks.7.1.weight', 'model.control_blocks.0.dense_hint_blocks.7.1.bias', 'model.control_blocks.0.dense_hint_blocks.8.1.weight', 'model.control_blocks.0.dense_hint_blocks.8.1.bias', 'model.control_blocks.0.dense_hint_blocks.9.1.weight', 'model.control_blocks.0.dense_hint_blocks.9.1.bias', 'model.control_blocks.0.dense_hint_blocks.10.1.weight', 'model.control_blocks.0.dense_hint_blocks.10.1.bias', 'model.control_blocks.0.dense_hint_blocks.11.1.weight', 'model.control_blocks.0.dense_hint_blocks.11.1.bias', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln2.bias', 'control_processor.netNetwork.norm', 'control_processor.netNetwork.block1.convs.0.weight', 'control_processor.netNetwork.block1.convs.0.bias', 'control_processor.netNetwork.block1.convs.1.weight', 'control_processor.netNetwork.block1.convs.1.bias', 'control_processor.netNetwork.block1.projection.weight', 'control_processor.netNetwork.block1.projection.bias', 'control_processor.netNetwork.block2.convs.0.weight', 'control_processor.netNetwork.block2.convs.0.bias', 'control_processor.netNetwork.block2.convs.1.weight', 'control_processor.netNetwork.block2.convs.1.bias', 'control_processor.netNetwork.block2.projection.weight', 'control_processor.netNetwork.block2.projection.bias', 'control_processor.netNetwork.block3.convs.0.weight', 'control_processor.netNetwork.block3.convs.0.bias', 'control_processor.netNetwork.block3.convs.1.weight', 'control_processor.netNetwork.block3.convs.1.bias', 'control_processor.netNetwork.block3.convs.2.weight', 'control_processor.netNetwork.block3.convs.2.bias', 'control_processor.netNetwork.block3.projection.weight', 'control_processor.netNetwork.block3.projection.bias', 'control_processor.netNetwork.block4.convs.0.weight', 'control_processor.netNetwork.block4.convs.0.bias', 'control_processor.netNetwork.block4.convs.1.weight', 'control_processor.netNetwork.block4.convs.1.bias', 'control_processor.netNetwork.block4.convs.2.weight', 'control_processor.netNetwork.block4.convs.2.bias', 'control_processor.netNetwork.block4.projection.weight', 'control_processor.netNetwork.block4.projection.bias', 'control_processor.netNetwork.block5.convs.0.weight', 'control_processor.netNetwork.block5.convs.0.bias', 'control_processor.netNetwork.block5.convs.1.weight', 'control_processor.netNetwork.block5.convs.1.bias', 'control_processor.netNetwork.block5.convs.2.weight', 'control_processor.netNetwork.block5.convs.2.bias', 'control_processor.netNetwork.block5.projection.weight', 'control_processor.netNetwork.block5.projection.bias'] scepter [INFO] 2024-09-26 12:02:22,438 [File: ldm.py Function: init_from_ckpt at line 224]
Unexpected Keys: ['alphas_cumprod', 'alphas_cumprod_prev', 'betas', 'log_one_minus_alphas_cumprod', 'model_ema.decay', 'model_ema.num_updates', 'posterior_log_variance_clipped', 'posterior_mean_coef1', 'posterior_mean_coef2', 'posterior_variance', 'sqrt_alphas_cumprod', 'sqrt_one_minus_alphas_cumprod', 'sqrt_recip_alphas_cumprod', 'sqrt_recipm1_alphas_cumprod', 'cond_stage_model.transformer.text_model.embeddings.position_ids'] scepter [INFO] 2024-09-26 12:02:23,318 [File: diffusion_solver.py Function: print_model_params_status at line 745] Load trainable params 113827928 / 1180063235 = 9.65%, train part: {'control_blocks.0': 99111760, 'netNetwork.norm': 3, 'netNetwork.block1': 38785, 'netNetwork.block2': 221569, 'netNetwork.block3': 1475585, 'netNetwork.block4': 5900289, 'netNetwork.block5': 7079937}. scepter [INFO] 2024-09-26 12:02:23,322 [File: diffusion_solver.py Function: print_model_params_status at line 749] Load forzen params 1066235307 / 1180063235 = 90.35%, forzen part: {'time_embed': 2050560, 'input_blocks': 249624320, 'middle_block': 97038080, 'output_blocks': 510795840, 'out': 12164, 'encoder': 34163592, 'decoder': 49490179, 'conv1': 72, 'conv2': 20, 'transformer': 123060480}. scepter [INFO] 2024-09-26 12:02:23,729 [File: diffusion_solver.py Function: print_model_params_status at line 745] Load trainable params 113827928 / 1180063235 = 9.65%, train part: {'control_blocks.0': 99111760, 'netNetwork.norm': 3, 'netNetwork.block1': 38785, 'netNetwork.block2': 221569, 'netNetwork.block3': 1475585, 'netNetwork.block4': 5900289, 'netNetwork.block5': 7079937}. scepter [INFO] 2024-09-26 12:02:23,730 [File: diffusion_solver.py Function: print_model_params_status at line 749] Load forzen params 1066235307 / 1180063235 = 90.35%, forzen part: {'time_embed': 2050560, 'input_blocks': 249624320, 'middle_block': 97038080, 'output_blocks': 510795840, 'out': 12164, 'encoder': 34163592, 'decoder': 49490179, 'conv1': 72, 'conv2': 20, 'transformer': 123060480}. scepter [INFO] 2024-09-26 12:02:23,903 [File: diffusion_solver.py Function: print_model_params_status at line 745] Load trainable params 113827928 / 1180063235 = 9.65%, train part: {'control_blocks.0': 99111760, 'netNetwork.norm': 3, 'netNetwork.block1': 38785, 'netNetwork.block2': 221569, 'netNetwork.block3': 1475585, 'netNetwork.block4': 5900289, 'netNetwork.block5': 7079937}. scepter [INFO] 2024-09-26 12:02:23,903 [File: diffusion_solver.py Function: print_model_params_status at line 749] Load forzen params 1066235307 / 1180063235 = 90.35%, forzen part: {'time_embed': 2050560, 'input_blocks': 249624320, 'middle_block': 97038080, 'output_blocks': 510795840, 'out': 12164, 'encoder': 34163592, 'decoder': 49490179, 'conv1': 72, 'conv2': 20, 'transformer': 123060480}. 0it [00:00, ?it/s]2024-09-26 12:02:36,709 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:36,711 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:36,723 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:36,724 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:36,764 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:36,768 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:36,808 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:36,809 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:36,844 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:36,846 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:36,849 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:36,875 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:36,877 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:36,886 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:36,891 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:36,893 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:36,918 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:36,921 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:36,922 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:36,934 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:36,963 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:36,963 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:36,964 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:36,976 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:36,979 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:37,005 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:37,005 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:37,007 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:37,046 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:37,050 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:37,051 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:37,053 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:37,093 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:37,137 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:37,139 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:37,154 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:37,156 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:37,180 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:37,196 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:37,308 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:37,310 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:37,351 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:37,454 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:37,456 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:37,502 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') 2024-09-26 12:02:37,558 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:37,561 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') 2024-09-26 12:02:37,605 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:37,606 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:37,625 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:37,645 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:37,646 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') 2024-09-26 12:02:37,651 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') 2024-09-26 12:02:37,679 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:37,681 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:37,694 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:37,706 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:37,708 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:37,730 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') 2024-09-26 12:02:37,750 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') 2024-09-26 12:02:37,812 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:37,814 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:37,870 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') 2024-09-26 12:02:37,884 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:37,886 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') 2024-09-26 12:02:37,930 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') 2024-09-26 12:02:38,134 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:38,136 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') 2024-09-26 12:02:38,169 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:38,171 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:38,204 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:38,219 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:38,236 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:38,237 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:38,250 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:38,251 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:38,257 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:38,259 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:38,263 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:38,265 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:38,269 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:38,271 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:38,276 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:38,289 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:38,300 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:38,303 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:38,307 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:38,309 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:38,310 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:38,373 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') 2024-09-26 12:02:38,685 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:38,687 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') 2024-09-26 12:02:38,707 - modelscope - INFO - PyTorch version 2.0.1 Found. 2024-09-26 12:02:38,709 - modelscope - INFO - Loading ast index from /home/twinkle/.cache/modelscope/ast_indexer 2024-09-26 12:02:38,726 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed 2024-09-26 12:02:38,752 - modelscope - INFO - Loading done! Current index file version is 1.14.0, with md5 d2b0059d36c5e4e075a8a672220b6aa2 and a total number of 976 components indexed /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') /data/twinkle/app/scepter/scepter/modules/solver/base_solver.py:185: UserWarning: No module named 'pytorch_lightning' warnings.warn(f'{e}') 0it [00:11, ?it/s] 0it [00:12, ?it/s] 0it [00:12, ?it/s] 0it [00:12, ?it/s] Traceback (most recent call last): File "scepter/tools/run_train.py", line 62, in run() File "scepter/tools/run_train.py", line 58, in run we.init_env(cfg, logger=None, fn=run_task) File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/scepter/modules/utils/distribute.py", line 431, in init_env mp.spawn(mp_worker, File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 239, in spawn return start_processes(fn, args, nprocs, join, daemon, start_method='spawn') File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 197, in start_processes while not context.join(): File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 160, in join raise ProcessRaisedException(msg, error_index, failed_process.pid) torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 0 terminated with the following error: Traceback (most recent call last): File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap fn(i, args) File "/data/twinkle/app/scepter/scepter/modules/utils/distribute.py", line 314, in mp_worker fn(cfg) File "/data/twinkle/app/scepter/scepter/tools/run_train.py", line 26, in run_task solver.solve() File "/data/twinkle/app/scepter/scepter/modules/solver/diffusion_solver.py", line 328, in solve self.run_train() File "/data/twinkle/app/scepter/scepter/modules/solver/diffusion_solver.py", line 341, in run_train self.run_eval() File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(args, kwargs) File "/data/twinkle/app/scepter/scepter/modules/solver/diffusion_solver.py", line 389, in run_eval results = self.run_step_eval(transfer_data_to_cuda(batch_data), File "/data/twinkle/app/scepter/scepter/modules/solver/base_solver.py", line 559, in run_step_eval results = self.model(batch_data) File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, kwargs) File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 1156, in forward output = self._run_ddp_forward(*inputs, *kwargs) File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 1110, in _run_ddp_forward return module_to_run(inputs[0], kwargs[0]) # type: ignore[index] File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(args, kwargs) File "/data/twinkle/app/scepter/scepter/modules/model/network/ldm/ldm.py", line 288, in forward return self.forward_test(kwargs) File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(args, kwargs) File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/amp/autocast_mode.py", line 14, in decorate_autocast return func(*args, *kwargs) File "/data/twinkle/app/scepter/scepter/modules/model/network/ldm/ldm_sce.py", line 128, in forward_test kwargs['hint'] = self.get_control_input(image_preprocess) File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(args, kwargs) File "/data/twinkle/app/scepter/scepter/modules/model/network/ldm/ldm_sce.py", line 108, in get_control_input hint = TT.ToTensor()(hint) File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torchvision/transforms/transforms.py", line 137, in call return F.to_tensor(pic) File "/data/twinkle/anaconda3/envs/scepter/lib/python3.8/site-packages/torchvision/transforms/functional.py", line 143, in to_tensor raise ValueError(f"pic should be 2/3 dimensional. Got {pic.ndim} dimensions.") ValueError: pic should be 2/3 dimensional. Got 4 dimensions.

/data/twinkle/anaconda3/envs/scepter/lib/python3.8/multiprocessing/resource_tracker.py:203: UserWarning: resource_tracker: There appear to be 160 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d ' when the output log is as follows:scepter [INFO] 2024-09-25 20:25:30,454 [File: logger.py Function: init_logger at line 86] Running task with log file: /data/twinkle/app/scepter/./cache/save_data/sd15_512_sce_ctr_hed/std_log.txt scepter [INFO] 2024-09-25 20:25:36,793 [File: registry.py Function: init at line 185] Built dataloader with len 9223372036854775807 scepter [INFO] 2024-09-25 20:25:39,084 [File: registry.py Function: init at line 185] Built dataloader with len 1 scepter [INFO] 2024-09-25 20:25:45,789 [File: ldm.py Function: construct_network at line 147] all parameters:859.52M scepter [INFO] 2024-09-25 20:25:46,549 [File: ae_module.py Function: construct_model at line 76] AE Module XFORMERS_IS_AVAILBLE : True scepter [INFO] 2024-09-25 20:25:58,095 [File: ldm.py Function: init_from_ckpt at line 218] Restored from ./cache/cache_data/95c8c1dcca842facbdcb9606c314f7ae/AI-ModelScope/stable-diffusion-v1-5/v1-5-pruned-emaonly.safetensors with 123 missing and 15 unexpected keys scepter [INFO] 2024-09-25 20:25:58,095 [File: ldm.py Function: init_from_ckpt at line 222] Missing Keys: ['model.control_blocks.0.pre_hint_blocks.0.weight', 'model.control_blocks.0.pre_hint_blocks.0.bias', 'model.control_blocks.0.pre_hint_blocks.2.weight', 'model.control_blocks.0.pre_hint_blocks.2.bias', 'model.control_blocks.0.pre_hint_blocks.4.weight', 'model.control_blocks.0.pre_hint_blocks.4.bias', 'model.control_blocks.0.pre_hint_blocks.6.weight', 'model.control_blocks.0.pre_hint_blocks.6.bias', 'model.control_blocks.0.pre_hint_blocks.8.weight', 'model.control_blocks.0.pre_hint_blocks.8.bias', 'model.control_blocks.0.pre_hint_blocks.10.weight', 'model.control_blocks.0.pre_hint_blocks.10.bias', 'model.control_blocks.0.pre_hint_blocks.12.weight', 'model.control_blocks.0.pre_hint_blocks.12.bias', 'model.control_blocks.0.dense_hint_blocks.0.1.weight', 'model.control_blocks.0.dense_hint_blocks.0.1.bias', 'model.control_blocks.0.dense_hint_blocks.1.1.weight', 'model.control_blocks.0.dense_hint_blocks.1.1.bias', 'model.control_blocks.0.dense_hint_blocks.2.1.weight', 'model.control_blocks.0.dense_hint_blocks.2.1.bias', 'model.control_blocks.0.dense_hint_blocks.3.1.weight', 'model.control_blocks.0.dense_hint_blocks.3.1.bias', 'model.control_blocks.0.dense_hint_blocks.4.1.weight', 'model.control_blocks.0.dense_hint_blocks.4.1.bias', 'model.control_blocks.0.dense_hint_blocks.5.1.weight', 'model.control_blocks.0.dense_hint_blocks.5.1.bias', 'model.control_blocks.0.dense_hint_blocks.6.1.weight', 'model.control_blocks.0.dense_hint_blocks.6.1.bias', 'model.control_blocks.0.dense_hint_blocks.7.1.weight', 'model.control_blocks.0.dense_hint_blocks.7.1.bias', 'model.control_blocks.0.dense_hint_blocks.8.1.weight', 'model.control_blocks.0.dense_hint_blocks.8.1.bias', 'model.control_blocks.0.dense_hint_blocks.9.1.weight', 'model.control_blocks.0.dense_hint_blocks.9.1.bias', 'model.control_blocks.0.dense_hint_blocks.10.1.weight', 'model.control_blocks.0.dense_hint_blocks.10.1.bias', 'model.control_blocks.0.dense_hint_blocks.11.1.weight', 'model.control_blocks.0.dense_hint_blocks.11.1.bias', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln2.bias', 'control_processor.netNetwork.norm', 'control_processor.netNetwork.block1.convs.0.weight', 'control_processor.netNetwork.block1.convs.0.bias', 'control_processor.netNetwork.block1.convs.1.weight', 'control_processor.netNetwork.block1.convs.1.bias', 'control_processor.netNetwork.block1.projection.weight', 'control_processor.netNetwork.block1.projection.bias', 'control_processor.netNetwork.block2.convs.0.weight', 'control_processor.netNetwork.block2.convs.0.bias', 'control_processor.netNetwork.block2.convs.1.weight', 'control_processor.netNetwork.block2.convs.1.bias', 'control_processor.netNetwork.block2.projection.weight', 'control_processor.netNetwork.block2.projection.bias', 'control_processor.netNetwork.block3.convs.0.weight', 'control_processor.netNetwork.block3.convs.0.bias', 'control_processor.netNetwork.block3.convs.1.weight', 'control_processor.netNetwork.block3.convs.1.bias', 'control_processor.netNetwork.block3.convs.2.weight', 'control_processor.netNetwork.block3.convs.2.bias', 'control_processor.netNetwork.block3.projection.weight', 'control_processor.netNetwork.block3.projection.bias', 'control_processor.netNetwork.block4.convs.0.weight', 'control_processor.netNetwork.block4.convs.0.bias', 'control_processor.netNetwork.block4.convs.1.weight', 'control_processor.netNetwork.block4.convs.1.bias', 'control_processor.netNetwork.block4.convs.2.weight', 'control_processor.netNetwork.block4.convs.2.bias', 'control_processor.netNetwork.block4.projection.weight', 'control_processor.netNetwork.block4.projection.bias', 'control_processor.netNetwork.block5.convs.0.weight', 'control_processor.netNetwork.block5.convs.0.bias', 'control_processor.netNetwork.block5.convs.1.weight', 'control_processor.netNetwork.block5.convs.1.bias', 'control_processor.netNetwork.block5.convs.2.weight', 'control_processor.netNetwork.block5.convs.2.bias', 'control_processor.netNetwork.block5.projection.weight', 'control_processor.netNetwork.block5.projection.bias'] scepter [INFO] 2024-09-25 20:25:58,096 [File: ldm.py Function: init_from_ckpt at line 224]
Unexpected Keys: ['alphas_cumprod', 'alphas_cumprod_prev', 'betas', 'log_one_minus_alphas_cumprod', 'model_ema.decay', 'model_ema.num_updates', 'posterior_log_variance_clipped', 'posterior_mean_coef1', 'posterior_mean_coef2', 'posterior_variance', 'sqrt_alphas_cumprod', 'sqrt_one_minus_alphas_cumprod', 'sqrt_recip_alphas_cumprod', 'sqrt_recipm1_alphas_cumprod', 'cond_stage_model.transformer.text_model.embeddings.position_ids'] scepter [INFO] 2024-09-25 20:26:00,271 [File: diffusion_solver.py Function: print_model_params_status at line 745] Load trainable params 113827928 / 1180063235 = 9.65%, train part: {'control_blocks.0': 99111760, 'netNetwork.norm': 3, 'netNetwork.block1': 38785, 'netNetwork.block2': 221569, 'netNetwork.block3': 1475585, 'netNetwork.block4': 5900289, 'netNetwork.block5': 7079937}. scepter [INFO] 2024-09-25 20:26:00,271 [File: diffusion_solver.py Function: print_model_params_status at line 749] Load forzen params 1066235307 / 1180063235 = 90.35%, forzen part: {'time_embed': 2050560, 'input_blocks': 249624320, 'middle_block': 97038080, 'output_blocks': 510795840, 'out': 12164, 'encoder': 34163592, 'decoder': 49490179, 'conv1': 72, 'conv2': 20, 'transformer': 123060480}. scepter [INFO] 2024-09-25 21:43:33,088 [File: logger.py Function: init_logger at line 86] Running task with log file: /data/twinkle/app/scepter/./cache/save_data/sd15_512_sce_ctr_hed/std_log.txt scepter [INFO] 2024-09-25 21:43:35,304 [File: registry.py Function: init at line 185] Built dataloader with len 9223372036854775807 scepter [INFO] 2024-09-25 21:43:37,392 [File: registry.py Function: init at line 185] Built dataloader with len 1 scepter [INFO] 2024-09-25 21:43:44,410 [File: ldm.py Function: construct_network at line 147] all parameters:859.52M scepter [INFO] 2024-09-25 21:43:45,426 [File: ae_module.py Function: construct_model at line 76] AE Module XFORMERS_IS_AVAILBLE : True scepter [INFO] 2024-09-25 21:43:57,616 [File: ldm.py Function: init_from_ckpt at line 218] Restored from ./cache/cache_data/95c8c1dcca842facbdcb9606c314f7ae/AI-ModelScope/stable-diffusion-v1-5/v1-5-pruned-emaonly.safetensors with 123 missing and 15 unexpected keys scepter [INFO] 2024-09-25 21:43:57,617 [File: ldm.py Function: init_from_ckpt at line 222] Missing Keys: ['model.control_blocks.0.pre_hint_blocks.0.weight', 'model.control_blocks.0.pre_hint_blocks.0.bias', 'model.control_blocks.0.pre_hint_blocks.2.weight', 'model.control_blocks.0.pre_hint_blocks.2.bias', 'model.control_blocks.0.pre_hint_blocks.4.weight', 'model.control_blocks.0.pre_hint_blocks.4.bias', 'model.control_blocks.0.pre_hint_blocks.6.weight', 'model.control_blocks.0.pre_hint_blocks.6.bias', 'model.control_blocks.0.pre_hint_blocks.8.weight', 'model.control_blocks.0.pre_hint_blocks.8.bias', 'model.control_blocks.0.pre_hint_blocks.10.weight', 'model.control_blocks.0.pre_hint_blocks.10.bias', 'model.control_blocks.0.pre_hint_blocks.12.weight', 'model.control_blocks.0.pre_hint_blocks.12.bias', 'model.control_blocks.0.dense_hint_blocks.0.1.weight', 'model.control_blocks.0.dense_hint_blocks.0.1.bias', 'model.control_blocks.0.dense_hint_blocks.1.1.weight', 'model.control_blocks.0.dense_hint_blocks.1.1.bias', 'model.control_blocks.0.dense_hint_blocks.2.1.weight', 'model.control_blocks.0.dense_hint_blocks.2.1.bias', 'model.control_blocks.0.dense_hint_blocks.3.1.weight', 'model.control_blocks.0.dense_hint_blocks.3.1.bias', 'model.control_blocks.0.dense_hint_blocks.4.1.weight', 'model.control_blocks.0.dense_hint_blocks.4.1.bias', 'model.control_blocks.0.dense_hint_blocks.5.1.weight', 'model.control_blocks.0.dense_hint_blocks.5.1.bias', 'model.control_blocks.0.dense_hint_blocks.6.1.weight', 'model.control_blocks.0.dense_hint_blocks.6.1.bias', 'model.control_blocks.0.dense_hint_blocks.7.1.weight', 'model.control_blocks.0.dense_hint_blocks.7.1.bias', 'model.control_blocks.0.dense_hint_blocks.8.1.weight', 'model.control_blocks.0.dense_hint_blocks.8.1.bias', 'model.control_blocks.0.dense_hint_blocks.9.1.weight', 'model.control_blocks.0.dense_hint_blocks.9.1.bias', 'model.control_blocks.0.dense_hint_blocks.10.1.weight', 'model.control_blocks.0.dense_hint_blocks.10.1.bias', 'model.control_blocks.0.dense_hint_blocks.11.1.weight', 'model.control_blocks.0.dense_hint_blocks.11.1.bias', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln2.bias', 'control_processor.netNetwork.norm', 'control_processor.netNetwork.block1.convs.0.weight', 'control_processor.netNetwork.block1.convs.0.bias', 'control_processor.netNetwork.block1.convs.1.weight', 'control_processor.netNetwork.block1.convs.1.bias', 'control_processor.netNetwork.block1.projection.weight', 'control_processor.netNetwork.block1.projection.bias', 'control_processor.netNetwork.block2.convs.0.weight', 'control_processor.netNetwork.block2.convs.0.bias', 'control_processor.netNetwork.block2.convs.1.weight', 'control_processor.netNetwork.block2.convs.1.bias', 'control_processor.netNetwork.block2.projection.weight', 'control_processor.netNetwork.block2.projection.bias', 'control_processor.netNetwork.block3.convs.0.weight', 'control_processor.netNetwork.block3.convs.0.bias', 'control_processor.netNetwork.block3.convs.1.weight', 'control_processor.netNetwork.block3.convs.1.bias', 'control_processor.netNetwork.block3.convs.2.weight', 'control_processor.netNetwork.block3.convs.2.bias', 'control_processor.netNetwork.block3.projection.weight', 'control_processor.netNetwork.block3.projection.bias', 'control_processor.netNetwork.block4.convs.0.weight', 'control_processor.netNetwork.block4.convs.0.bias', 'control_processor.netNetwork.block4.convs.1.weight', 'control_processor.netNetwork.block4.convs.1.bias', 'control_processor.netNetwork.block4.convs.2.weight', 'control_processor.netNetwork.block4.convs.2.bias', 'control_processor.netNetwork.block4.projection.weight', 'control_processor.netNetwork.block4.projection.bias', 'control_processor.netNetwork.block5.convs.0.weight', 'control_processor.netNetwork.block5.convs.0.bias', 'control_processor.netNetwork.block5.convs.1.weight', 'control_processor.netNetwork.block5.convs.1.bias', 'control_processor.netNetwork.block5.convs.2.weight', 'control_processor.netNetwork.block5.convs.2.bias', 'control_processor.netNetwork.block5.projection.weight', 'control_processor.netNetwork.block5.projection.bias'] scepter [INFO] 2024-09-25 21:43:57,617 [File: ldm.py Function: init_from_ckpt at line 224]
Unexpected Keys: ['alphas_cumprod', 'alphas_cumprod_prev', 'betas', 'log_one_minus_alphas_cumprod', 'model_ema.decay', 'model_ema.num_updates', 'posterior_log_variance_clipped', 'posterior_mean_coef1', 'posterior_mean_coef2', 'posterior_variance', 'sqrt_alphas_cumprod', 'sqrt_one_minus_alphas_cumprod', 'sqrt_recip_alphas_cumprod', 'sqrt_recipm1_alphas_cumprod', 'cond_stage_model.transformer.text_model.embeddings.position_ids'] scepter [INFO] 2024-09-25 21:44:01,524 [File: diffusion_solver.py Function: print_model_params_status at line 745] Load trainable params 113827928 / 1180063235 = 9.65%, train part: {'control_blocks.0': 99111760, 'netNetwork.norm': 3, 'netNetwork.block1': 38785, 'netNetwork.block2': 221569, 'netNetwork.block3': 1475585, 'netNetwork.block4': 5900289, 'netNetwork.block5': 7079937}. scepter [INFO] 2024-09-25 21:44:01,530 [File: diffusion_solver.py Function: print_model_params_status at line 749] Load forzen params 1066235307 / 1180063235 = 90.35%, forzen part: {'time_embed': 2050560, 'input_blocks': 249624320, 'middle_block': 97038080, 'output_blocks': 510795840, 'out': 12164, 'encoder': 34163592, 'decoder': 49490179, 'conv1': 72, 'conv2': 20, 'transformer': 123060480}. scepter [INFO] 2024-09-26 11:55:44,170 [File: logger.py Function: init_logger at line 86] Running task with log file: /data/twinkle/app/scepter/./cache/save_data/sd15_512_sce_ctr_hed/std_log.txt scepter [INFO] 2024-09-26 11:55:46,858 [File: registry.py Function: init at line 185] Built dataloader with len 9223372036854775807 scepter [INFO] 2024-09-26 11:55:48,803 [File: registry.py Function: init at line 185] Built dataloader with len 1 scepter [INFO] 2024-09-26 11:55:55,857 [File: ldm.py Function: construct_network at line 147] all parameters:859.52M scepter [INFO] 2024-09-26 11:55:56,719 [File: ae_module.py Function: construct_model at line 76] AE Module XFORMERS_IS_AVAILBLE : True scepter [INFO] 2024-09-26 11:56:06,668 [File: ldm.py Function: init_from_ckpt at line 218] Restored from ./cache/cache_data/95c8c1dcca842facbdcb9606c314f7ae/AI-ModelScope/stable-diffusion-v1-5/v1-5-pruned-emaonly.safetensors with 123 missing and 15 unexpected keys scepter [INFO] 2024-09-26 11:56:06,670 [File: ldm.py Function: init_from_ckpt at line 222] Missing Keys: ['model.control_blocks.0.pre_hint_blocks.0.weight', 'model.control_blocks.0.pre_hint_blocks.0.bias', 'model.control_blocks.0.pre_hint_blocks.2.weight', 'model.control_blocks.0.pre_hint_blocks.2.bias', 'model.control_blocks.0.pre_hint_blocks.4.weight', 'model.control_blocks.0.pre_hint_blocks.4.bias', 'model.control_blocks.0.pre_hint_blocks.6.weight', 'model.control_blocks.0.pre_hint_blocks.6.bias', 'model.control_blocks.0.pre_hint_blocks.8.weight', 'model.control_blocks.0.pre_hint_blocks.8.bias', 'model.control_blocks.0.pre_hint_blocks.10.weight', 'model.control_blocks.0.pre_hint_blocks.10.bias', 'model.control_blocks.0.pre_hint_blocks.12.weight', 'model.control_blocks.0.pre_hint_blocks.12.bias', 'model.control_blocks.0.dense_hint_blocks.0.1.weight', 'model.control_blocks.0.dense_hint_blocks.0.1.bias', 'model.control_blocks.0.dense_hint_blocks.1.1.weight', 'model.control_blocks.0.dense_hint_blocks.1.1.bias', 'model.control_blocks.0.dense_hint_blocks.2.1.weight', 'model.control_blocks.0.dense_hint_blocks.2.1.bias', 'model.control_blocks.0.dense_hint_blocks.3.1.weight', 'model.control_blocks.0.dense_hint_blocks.3.1.bias', 'model.control_blocks.0.dense_hint_blocks.4.1.weight', 'model.control_blocks.0.dense_hint_blocks.4.1.bias', 'model.control_blocks.0.dense_hint_blocks.5.1.weight', 'model.control_blocks.0.dense_hint_blocks.5.1.bias', 'model.control_blocks.0.dense_hint_blocks.6.1.weight', 'model.control_blocks.0.dense_hint_blocks.6.1.bias', 'model.control_blocks.0.dense_hint_blocks.7.1.weight', 'model.control_blocks.0.dense_hint_blocks.7.1.bias', 'model.control_blocks.0.dense_hint_blocks.8.1.weight', 'model.control_blocks.0.dense_hint_blocks.8.1.bias', 'model.control_blocks.0.dense_hint_blocks.9.1.weight', 'model.control_blocks.0.dense_hint_blocks.9.1.bias', 'model.control_blocks.0.dense_hint_blocks.10.1.weight', 'model.control_blocks.0.dense_hint_blocks.10.1.bias', 'model.control_blocks.0.dense_hint_blocks.11.1.weight', 'model.control_blocks.0.dense_hint_blocks.11.1.bias', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln2.bias', 'control_processor.netNetwork.norm', 'control_processor.netNetwork.block1.convs.0.weight', 'control_processor.netNetwork.block1.convs.0.bias', 'control_processor.netNetwork.block1.convs.1.weight', 'control_processor.netNetwork.block1.convs.1.bias', 'control_processor.netNetwork.block1.projection.weight', 'control_processor.netNetwork.block1.projection.bias', 'control_processor.netNetwork.block2.convs.0.weight', 'control_processor.netNetwork.block2.convs.0.bias', 'control_processor.netNetwork.block2.convs.1.weight', 'control_processor.netNetwork.block2.convs.1.bias', 'control_processor.netNetwork.block2.projection.weight', 'control_processor.netNetwork.block2.projection.bias', 'control_processor.netNetwork.block3.convs.0.weight', 'control_processor.netNetwork.block3.convs.0.bias', 'control_processor.netNetwork.block3.convs.1.weight', 'control_processor.netNetwork.block3.convs.1.bias', 'control_processor.netNetwork.block3.convs.2.weight', 'control_processor.netNetwork.block3.convs.2.bias', 'control_processor.netNetwork.block3.projection.weight', 'control_processor.netNetwork.block3.projection.bias', 'control_processor.netNetwork.block4.convs.0.weight', 'control_processor.netNetwork.block4.convs.0.bias', 'control_processor.netNetwork.block4.convs.1.weight', 'control_processor.netNetwork.block4.convs.1.bias', 'control_processor.netNetwork.block4.convs.2.weight', 'control_processor.netNetwork.block4.convs.2.bias', 'control_processor.netNetwork.block4.projection.weight', 'control_processor.netNetwork.block4.projection.bias', 'control_processor.netNetwork.block5.convs.0.weight', 'control_processor.netNetwork.block5.convs.0.bias', 'control_processor.netNetwork.block5.convs.1.weight', 'control_processor.netNetwork.block5.convs.1.bias', 'control_processor.netNetwork.block5.convs.2.weight', 'control_processor.netNetwork.block5.convs.2.bias', 'control_processor.netNetwork.block5.projection.weight', 'control_processor.netNetwork.block5.projection.bias'] scepter [INFO] 2024-09-26 11:56:06,673 [File: ldm.py Function: init_from_ckpt at line 224]
Unexpected Keys: ['alphas_cumprod', 'alphas_cumprod_prev', 'betas', 'log_one_minus_alphas_cumprod', 'model_ema.decay', 'model_ema.num_updates', 'posterior_log_variance_clipped', 'posterior_mean_coef1', 'posterior_mean_coef2', 'posterior_variance', 'sqrt_alphas_cumprod', 'sqrt_one_minus_alphas_cumprod', 'sqrt_recip_alphas_cumprod', 'sqrt_recipm1_alphas_cumprod', 'cond_stage_model.transformer.text_model.embeddings.position_ids'] scepter [INFO] 2024-09-26 11:56:08,655 [File: diffusion_solver.py Function: print_model_params_status at line 745] Load trainable params 113827928 / 1180063235 = 9.65%, train part: {'control_blocks.0': 99111760, 'netNetwork.norm': 3, 'netNetwork.block1': 38785, 'netNetwork.block2': 221569, 'netNetwork.block3': 1475585, 'netNetwork.block4': 5900289, 'netNetwork.block5': 7079937}. scepter [INFO] 2024-09-26 11:56:08,661 [File: diffusion_solver.py Function: print_model_params_status at line 749] Load forzen params 1066235307 / 1180063235 = 90.35%, forzen part: {'time_embed': 2050560, 'input_blocks': 249624320, 'middle_block': 97038080, 'output_blocks': 510795840, 'out': 12164, 'encoder': 34163592, 'decoder': 49490179, 'conv1': 72, 'conv2': 20, 'transformer': 123060480}. scepter [INFO] 2024-09-26 12:01:59,585 [File: logger.py Function: init_logger at line 86] Running task with log file: /data/twinkle/app/scepter/./cache/save_data/sd15_512_sce_ctr_hed/std_log.txt scepter [INFO] 2024-09-26 12:02:02,899 [File: registry.py Function: init at line 185] Built dataloader with len 9223372036854775807 scepter [INFO] 2024-09-26 12:02:04,676 [File: registry.py Function: init at line 185] Built dataloader with len 1 scepter [INFO] 2024-09-26 12:02:11,685 [File: ldm.py Function: construct_network at line 147] all parameters:859.52M scepter [INFO] 2024-09-26 12:02:12,399 [File: ae_module.py Function: construct_model at line 76] AE Module XFORMERS_IS_AVAILBLE : True scepter [INFO] 2024-09-26 12:02:22,433 [File: ldm.py Function: init_from_ckpt at line 218] Restored from ./cache/cache_data/95c8c1dcca842facbdcb9606c314f7ae/AI-ModelScope/stable-diffusion-v1-5/v1-5-pruned-emaonly.safetensors with 123 missing and 15 unexpected keys scepter [INFO] 2024-09-26 12:02:22,438 [File: ldm.py Function: init_from_ckpt at line 222] Missing Keys: ['model.control_blocks.0.pre_hint_blocks.0.weight', 'model.control_blocks.0.pre_hint_blocks.0.bias', 'model.control_blocks.0.pre_hint_blocks.2.weight', 'model.control_blocks.0.pre_hint_blocks.2.bias', 'model.control_blocks.0.pre_hint_blocks.4.weight', 'model.control_blocks.0.pre_hint_blocks.4.bias', 'model.control_blocks.0.pre_hint_blocks.6.weight', 'model.control_blocks.0.pre_hint_blocks.6.bias', 'model.control_blocks.0.pre_hint_blocks.8.weight', 'model.control_blocks.0.pre_hint_blocks.8.bias', 'model.control_blocks.0.pre_hint_blocks.10.weight', 'model.control_blocks.0.pre_hint_blocks.10.bias', 'model.control_blocks.0.pre_hint_blocks.12.weight', 'model.control_blocks.0.pre_hint_blocks.12.bias', 'model.control_blocks.0.dense_hint_blocks.0.1.weight', 'model.control_blocks.0.dense_hint_blocks.0.1.bias', 'model.control_blocks.0.dense_hint_blocks.1.1.weight', 'model.control_blocks.0.dense_hint_blocks.1.1.bias', 'model.control_blocks.0.dense_hint_blocks.2.1.weight', 'model.control_blocks.0.dense_hint_blocks.2.1.bias', 'model.control_blocks.0.dense_hint_blocks.3.1.weight', 'model.control_blocks.0.dense_hint_blocks.3.1.bias', 'model.control_blocks.0.dense_hint_blocks.4.1.weight', 'model.control_blocks.0.dense_hint_blocks.4.1.bias', 'model.control_blocks.0.dense_hint_blocks.5.1.weight', 'model.control_blocks.0.dense_hint_blocks.5.1.bias', 'model.control_blocks.0.dense_hint_blocks.6.1.weight', 'model.control_blocks.0.dense_hint_blocks.6.1.bias', 'model.control_blocks.0.dense_hint_blocks.7.1.weight', 'model.control_blocks.0.dense_hint_blocks.7.1.bias', 'model.control_blocks.0.dense_hint_blocks.8.1.weight', 'model.control_blocks.0.dense_hint_blocks.8.1.bias', 'model.control_blocks.0.dense_hint_blocks.9.1.weight', 'model.control_blocks.0.dense_hint_blocks.9.1.bias', 'model.control_blocks.0.dense_hint_blocks.10.1.weight', 'model.control_blocks.0.dense_hint_blocks.10.1.bias', 'model.control_blocks.0.dense_hint_blocks.11.1.weight', 'model.control_blocks.0.dense_hint_blocks.11.1.bias', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.0.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.1.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.2.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.3.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.4.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.5.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.6.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.7.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.8.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.9.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.10.tuner_op.ln2.bias', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln1.weight', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln1.bias', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln2.weight', 'model.control_blocks.0.lsc_tuner_blocks.11.tuner_op.ln2.bias', 'control_processor.netNetwork.norm', 'control_processor.netNetwork.block1.convs.0.weight', 'control_processor.netNetwork.block1.convs.0.bias', 'control_processor.netNetwork.block1.convs.1.weight', 'control_processor.netNetwork.block1.convs.1.bias', 'control_processor.netNetwork.block1.projection.weight', 'control_processor.netNetwork.block1.projection.bias', 'control_processor.netNetwork.block2.convs.0.weight', 'control_processor.netNetwork.block2.convs.0.bias', 'control_processor.netNetwork.block2.convs.1.weight', 'control_processor.netNetwork.block2.convs.1.bias', 'control_processor.netNetwork.block2.projection.weight', 'control_processor.netNetwork.block2.projection.bias', 'control_processor.netNetwork.block3.convs.0.weight', 'control_processor.netNetwork.block3.convs.0.bias', 'control_processor.netNetwork.block3.convs.1.weight', 'control_processor.netNetwork.block3.convs.1.bias', 'control_processor.netNetwork.block3.convs.2.weight', 'control_processor.netNetwork.block3.convs.2.bias', 'control_processor.netNetwork.block3.projection.weight', 'control_processor.netNetwork.block3.projection.bias', 'control_processor.netNetwork.block4.convs.0.weight', 'control_processor.netNetwork.block4.convs.0.bias', 'control_processor.netNetwork.block4.convs.1.weight', 'control_processor.netNetwork.block4.convs.1.bias', 'control_processor.netNetwork.block4.convs.2.weight', 'control_processor.netNetwork.block4.convs.2.bias', 'control_processor.netNetwork.block4.projection.weight', 'control_processor.netNetwork.block4.projection.bias', 'control_processor.netNetwork.block5.convs.0.weight', 'control_processor.netNetwork.block5.convs.0.bias', 'control_processor.netNetwork.block5.convs.1.weight', 'control_processor.netNetwork.block5.convs.1.bias', 'control_processor.netNetwork.block5.convs.2.weight', 'control_processor.netNetwork.block5.convs.2.bias', 'control_processor.netNetwork.block5.projection.weight', 'control_processor.netNetwork.block5.projection.bias'] scepter [INFO] 2024-09-26 12:02:22,438 [File: ldm.py Function: init_from_ckpt at line 224]
Unexpected Keys: ['alphas_cumprod', 'alphas_cumprod_prev', 'betas', 'log_one_minus_alphas_cumprod', 'model_ema.decay', 'model_ema.num_updates', 'posterior_log_variance_clipped', 'posterior_mean_coef1', 'posterior_mean_coef2', 'posterior_variance', 'sqrt_alphas_cumprod', 'sqrt_one_minus_alphas_cumprod', 'sqrt_recip_alphas_cumprod', 'sqrt_recipm1_alphas_cumprod', 'cond_stage_model.transformer.text_model.embeddings.position_ids'] scepter [INFO] 2024-09-26 12:02:23,729 [File: diffusion_solver.py Function: print_model_params_status at line 745] Load trainable params 113827928 / 1180063235 = 9.65%, train part: {'control_blocks.0': 99111760, 'netNetwork.norm': 3, 'netNetwork.block1': 38785, 'netNetwork.block2': 221569, 'netNetwork.block3': 1475585, 'netNetwork.block4': 5900289, 'netNetwork.block5': 7079937}. scepter [INFO] 2024-09-26 12:02:23,730 [File: diffusion_solver.py Function: print_model_params_status at line 749] Load forzen params 1066235307 / 1180063235 = 90.35%, forzen part: {'time_embed': 2050560, 'input_blocks': 249624320, 'middle_block': 97038080, 'output_blocks': 510795840, 'out': 12164, 'encoder': 34163592, 'decoder': 49490179, 'conv1': 72, 'conv2': 20, 'transformer': 123060480}.

[yaosheng216]

@twinkleyang1 Thanks for your feedback, we are repaired this problem, you can pull new code from main branch to execution.