search

PDillis / stylegan3-fun

Modifications of the official PyTorch implementation of StyleGAN3. Let's easily generate images and videos with StyleGAN2/2-ADA/3!
Other
230 stars 36 forks source link

some files in the zip get corrupted when trying to zip it #19

Closed nicolai256 closed 2 years ago

nicolai256 commented 2 years ago

Describe the bug some files in the zip get corrupted when trying to zip it how can i use it without zip files?

tick 0     kimg 11252.0  time 1m 39s       sec/tick 22.9    sec/kimg 2864.63 maintenance 75.8   cpumem 4.66   gpumem 17.19  reserved 19.70  augment 11.202
tick 1     kimg 11256.0  time 17m 46s      sec/tick 953.9   sec/kimg 238.48  maintenance 13.3   cpumem 4.71   gpumem 14.83  reserved 18.82  augment 11.186
tick 2     kimg 11260.0  time 33m 31s      sec/tick 931.1   sec/kimg 232.77  maintenance 13.6   cpumem 4.71   gpumem 14.88  reserved 18.82  augment 11.170
tick 3     kimg 11264.0  time 49m 06s      sec/tick 922.5   sec/kimg 230.63  maintenance 13.2   cpumem 4.72   gpumem 14.85  reserved 18.82  augment 11.155
tick 4     kimg 11268.0  time 1h 04m 52s   sec/tick 932.2   sec/kimg 233.06  maintenance 13.8   cpumem 4.72   gpumem 15.01  reserved 18.82  augment 11.140
tick 5     kimg 11272.0  time 1h 20m 43s   sec/tick 936.9   sec/kimg 234.21  maintenance 13.8   cpumem 4.72   gpumem 14.96  reserved 18.82  augment 11.132
tick 6     kimg 11276.0  time 1h 36m 30s   sec/tick 933.1   sec/kimg 233.28  maintenance 13.6   cpumem 4.72   gpumem 14.76  reserved 18.82  augment 11.117
Traceback (most recent call last):
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\StyleGAN3\train.py", line 330, in <module>
    main()  # pylint: disable=no-value-for-parameter
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\venv\voc_base\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\venv\voc_base\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\venv\voc_base\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\venv\voc_base\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\StyleGAN3\train.py", line 323, in main
    launch_training(c=c, desc=desc, outdir=opts.outdir, dry_run=opts.dry_run)
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\StyleGAN3\train.py", line 92, in launch_training
    subprocess_fn(rank=0, c=c, temp_dir=temp_dir)
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\StyleGAN3\train.py", line 50, in subprocess_fn
    training_loop.training_loop(rank=rank, **c)
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\StyleGAN3\training\training_loop.py", line 260, in training_loop
    phase_real_img, phase_real_c = next(training_set_iterator)
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\venv\voc_base\lib\site-packages\torch\utils\data\dataloader.py", line 521, in __next__
    data = self._next_data()
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\venv\voc_base\lib\site-packages\torch\utils\data\dataloader.py", line 1203, in _next_data
    return self._process_data(data)
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\venv\voc_base\lib\site-packages\torch\utils\data\dataloader.py", line 1229, in _process_data
    data.reraise()
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\venv\voc_base\lib\site-packages\torch\_utils.py", line 425, in reraise
    raise self.exc_type(msg)
zipfile.BadZipFile: Caught BadZipFile in DataLoader worker process 2.
Original Traceback (most recent call last):
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\venv\voc_base\lib\site-packages\torch\utils\data\_utils\worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\venv\voc_base\lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\venv\voc_base\lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\StyleGAN3\training\dataset.py", line 97, in __getitem__
    image = self._load_raw_image(self._raw_idx[idx])
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\StyleGAN3\training\dataset.py", line 227, in _load_raw_image
    image = np.array(PIL.Image.open(f))
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\venv\voc_base\lib\site-packages\PIL\Image.py", line 719, in __array__
    new["data"] = self.tobytes()
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\venv\voc_base\lib\site-packages\PIL\Image.py", line 762, in tobytes
    self.load()
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\venv\voc_base\lib\site-packages\PIL\ImageFile.py", line 239, in load
    s = read(self.decodermaxblock)
  File "C:\Users\Gebruiker\AppData\Roaming\Visions of Chaos\Examples\MachineLearning\venv\voc_base\lib\site-packages\PIL\PngImagePlugin.py", line 921, in load_read
    self.fp.
read(4)  # CRC
  File "C:\Python\lib\zipfile.py", line 922, in read
    data = self._read1(n)
  File "C:\Python\lib\zipfile.py", line 1012, in _read1
    self._update_crc(data)
  File "C:\Python\lib\zipfile.py", line 940, in _update_crc
    raise BadZipFile("Bad CRC-32 for file %r" % self.name)
zipfile.BadZipFile: Bad CRC-32 for file '00006/img00006513.png'
PDillis commented 2 years ago

You can't use without a .zip file, but have you checked the '00006/img00006513.png' image? See if it has more or less channels, the format, or the size. I haven't encountered this problem before, so look into redoing the .zip file and checking your dataset.

Looking around, I found this, so let me know if it helps or if you have found a solution to this problem.