Problem with .png file truncated issue during model training or opening the file

[jxwleong]

When running the model training or opening certain .png data files. The following error is reported.

Converting mcr_lter1_out10m_pole5-6_qu1_20100404.jpg to png...  Traceback (most recent call last):
  File ".\convert_jpg_to_png.py", line 22, in <module>
    img.save(output_file)
  File "C:\Python38\lib\site-packages\PIL\Image.py", line 2317, in save
    self._ensure_mutable()
  File "C:\Python38\lib\site-packages\PIL\Image.py", line 599, in _ensure_mutable
    self._copy()
  File "C:\Python38\lib\site-packages\PIL\Image.py", line 592, in _copy
    self.load()
  File "C:\Python38\lib\site-packages\PIL\ImageFile.py", line 254, in load
    raise OSError(
OSError: image file is truncated (51 bytes not processed)

[jxwleong]

Set this setting ImageFile.LOAD_TRUNCATED_IMAGES = True seems to solve the issue on convert_jpg_to_png.py. WIll try the png image files only on the model training and see whether it solve the problem.

from PIL import Image, ImageFile

# Tell PIL to accept truncated images
ImageFile.LOAD_TRUNCATED_IMAGES = True

[jxwleong]

Removed the icc_profile in the image during jpg to png conversion else it will show libpng warning: iCCP: known incorrect sRGB profile during model training.

Updated convert_jpt_to_png.py

from PIL import Image, ImageFile
import os
import sys

ROOT_DIR = os.path.normpath(os.path.join(os.path.abspath(__file__), "..", "..", ".."))
sys.path.insert(0, ROOT_DIR)
from config.path import IMAGE_DIR

input_folder = IMAGE_DIR
output_folder = input_folder  # Saving output to the same folder

# Tell PIL to accept truncated images
ImageFile.LOAD_TRUNCATED_IMAGES = True

# Set to True if you want to overwrite existing files
overwrite_existing_files = True

for jpeg_file in os.listdir(input_folder):
    if jpeg_file.endswith(".jpg"):
        png_file = os.path.splitext(jpeg_file)[0] + ".png"
        output_file = os.path.join(output_folder, png_file)
        if os.path.exists(output_file) and not overwrite_existing_files:
            print(f"{output_file} exists! Skipping...")
            continue
        try:
            print(f"Converting {jpeg_file} to png... ", end=" ")
            img = Image.open(os.path.join(input_folder, jpeg_file))
            # Remove ICC profile as it caused this issue:
            # libpng warning: iCCP: known incorrect sRGB profile
            if 'icc_profile' in img.info:
                del img.info['icc_profile']
            img.save(output_file)
            print("DONE")
        except Exception as e:
            print(f"Error converting {jpeg_file} to png: {str(e)}")

[jxwleong]

Didn't see any of Corrupt JPEG data: bad Huffman code and Premature end of JPEG file issues when training the models. Will close this for now.