fenglinglwb / MAT

MAT: Mask-Aware Transformer for Large Hole Image Inpainting
Other
753 stars 84 forks source link

About the frequency of snapshots #76

Open kdavidlp123 opened 1 year ago

kdavidlp123 commented 1 year ago

Hi, thank you for your great work and the provided pretrained model. I have some problem when I try to modify the code in the train_loop.py file. I have total 52001 images and I set 1k image as 1 tick, also I take a snap shot every 1 tick. Below is where I modified : total_kimg = 52, # Total length of the training, measured in thousands of real images. kimg_per_tick = 1, # Progress snapshot interval. image_snapshot_ticks = 1, # How often to save image snapshots? None = disable. network_snapshot_ticks = 1, # How often to save network snapshots? None = disable.

And the training script I used is: python train.py --outdir=output_path --gpus=1 --batch=2 --metrics=fid36k5_full --data=C:/Users/blue/MAT/customdatasets/images.zip --dataloader=datasets.dataset_512.ImageFolderMaskDataset --mirror=True --cond=False --cfg=places512 --aug=noaug --generator=networks.mat.Generator --discriminator=networks.mat.Discriminator --loss=losses.loss.TwoStageLoss --pr=0.1 --pl=False --truncation=0.5 --style_mix=0.5 --ema=10 --lr=0.001 --resume=ffhq512

And here are the files in the output_folder: 2023-04-08 200240

It seems that it only saved for the 1st tick. May I ask for some help?

fenglinglwb commented 1 year ago

The network-snapshot-0000000.pkl is the initial model. It seems that the program doesn't save another snapshot any more. Has an error been reported?

136LYJ commented 1 month ago

image image