lingtengqiu
commented
8 months ago Hi, ND 是四个通道的VAE， albedo 是3个通道的VAE
bash scripts/train_abledo/objaverse/objaverse_finetune_mvsd-4.sh 这个代码你看英文他是跑的albedo，所以自然要载入albedo的权重
Closed 1192267986 closed 8 months ago
lingtengqiu
commented
8 months ago Hi, ND 是四个通道的VAE， albedo 是3个通道的VAE
bash scripts/train_abledo/objaverse/objaverse_finetune_mvsd-4.sh 这个代码你看英文他是跑的albedo，所以自然要载入albedo的权重
1192267986
commented
8 months ago 但是我看配置文件里是ckpt_path: '/path/to/sd-v2-1.ckpt'，这个是不是应该用v2-1_512-ema-pruned.ckpt这个进行训练呢
感谢您出色的工作！ 在我下载了nd-laion.ckpt使用bash scripts/train_abledo/objaverse/objaverse_finetune_mvsd-4.sh --gpus 0,1,2,3,4,5,6,7, model.params.ckpt_path=${Normal-Depth-Diffusion} 训练的时候出现了问题： ================= cat_d Manual init: model.diffusion_model.input_blocks.0.0.weight Manual init: model_ema.diffusion_modelinput_blocks00weight Traceback (most recent call last): File "/data/vjuicefs_hz_cv_enhance_v1/11164886/normal-depth-diffusion/main.py", line 548, in
model = instantiate_from_config(config.model)
File "/data/vjuicefs_hz_cv_enhance_v1/11164886/normal-depth-diffusion/ldm/util.py", line 116, in instantiate_from_config
return get_obj_from_str(config['target'])(**config.get('params', dict()))
File "/data/vjuicefs_hz_cv_enhance_v1/11164886/normal-depth-diffusion/ldm/models/diffusion/mvvideo_ddpm.py", line 893, in init
self.init_from_ckpt(ckpt_path, ignore_keys)
File "/data/vjuicefs_hz_cv_enhance_v1/11164886/normal-depth-diffusion/ldm/models/diffusion/mvvideo_ddpm.py", line 348, in init_from_ckpt
missing, unexpected = self.load_state_dict(
File "/opt/anaconda3/envs/nd/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1604, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for LatentDiffusion:
但当我使用albedo_mv_ema.ckpt作为model.params.ckpt_path的时候就可以训练，请问这是怎么回事呢？