Open cynthia-rempel opened 3 years ago
It would also be really helpful is we could log the filename of the truncated image
@cynthia-rempel would you like to make a PR with this fix?
Agreed, would be very useful!
I tried switching to this dataset: https://huggingface.co/datasets/cats_vs_dogs
@cynthia-rempel 's solution above addresses the truncated image issue, but I'm hitting some kind of snag during training, which causes values to turn to 'nan
', after which (as you can imagine) training progress is broken.
Has anyone experienced similar / does anyone have any ideas on how to start debugging this issue?
For what it's worth, I created an image checker as well (probably logging too much for this example project, but useful for the above debugging):
from pathlib import Path
from PIL import Image, ImageFile
# Load truncated images
ImageFile.LOAD_TRUNCATED_IMAGES = True
# Check images are not corrupt and conform to rules
img_rules = {
"suffix": ['jpg', 'jpeg', 'png'],
"formats": ['JPEG', 'PNG'],
"size": 1000,
"modes": ['RGB']
}
img_counts = {
"total": 0,
"rejected": 0,
}
def is_image(path):
try:
img_counts["total"] += 1
file = Path(path)
# Check image is correct suffix
suffix = f"{file.suffix.replace('.','')}"
if suffix not in img_rules['suffix']:
print(f"{path} >> ❌ Suffix")
img_counts["rejected"] += 1
return False
# Open image and check each rule
img = Image.open(path)
rej = None
if img.format not in img_rules['formats']:
rej = f"Format: {img.format}"
if img.size[0] > img_rules['size'] or img.size[1] > img_rules['size']:
rej = f"Size: {img.size}"
if img.mode not in img_rules['modes']:
rej = f"Mode: {img.mode}"
if rej:
print(f"{path} >> ❌ {rej}")
img_counts["rejected"] += 1
return False
return True
except:
return False
print(
f"{img_counts['total']} Images | ❌ Rejected: {img_counts['rejected']} | ✅ Accepted: {img_counts['total'] - img_counts['rejected']}")
You can then enable the above checker function for every loaded image, by adding the is_valid_file
param to datasets.ImageFolder:
# Create the dataset
dataset = dset.ImageFolder(root=dataroot,
transform=transforms.Compose([
transforms.Resize(image_size),
transforms.CenterCrop(image_size),
transforms.ToTensor(),
transforms.Normalize(
(0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
],),
is_valid_file=is_image
)
When I was using the dcgan example, it had:
So I added:
to examples/tree/master/dcgan/main.py and that fixed it Thanks!