search

NVlabs / stylegan3

Official PyTorch implementation of StyleGAN3
Other
6.28k stars 1.1k forks source link

Data loader warning #234

Open trincaw opened 1 year ago

trincaw commented 1 year ago

I am attempting to train StyleGAN3 on Google Cloud using the following setup:

2vCPU
13GB of RAM
Nvidia T4
PyTorch 1.13
CUDA 11.3
Python 3.10

My dataset consists of approximately 14,000 images of dresses.

The training process starts with the following command: python train.py --data=/home/..../dataset.zip --outdir=/home/..../training-runs --cfg=stylegan3-t --gpus=1 --batch=32 --gamma=2 --batch-gpu=16 --snap=10 --mirror=1 --workers=2 And throw this warning: waring message displayed

And when the evaluation process start it freeze and remain stucked forever. I tryed with --workers=1 but metrics still seems freeze so I have to disable them.

Thank you for your assistance.

PDillis commented 1 year ago

Turn off the metrics, that's what's taking a long time (the first time you run them that is, but I don't think you're interested in the metrics when the network hasn't been trained). If you're curious, you can always run them later by running calc_metrics.py.

To turn them off, add to your command --metrics=none and it should start training, don't worry about the number of workers.

jasuriy commented 2 months ago

@trincaw did you fix the issue? were you able to train the model with your own dataset?