about low training speed for Protocol-CAT, especially the slow speed for compRASIE dataset.

[chchshshhh]

I'm experiencing extremely slow model training speed when using the Protocol-CAT method on IMDLBenco. Specifically, running a single epoch on the Segformer backbone takes five times longer compared to training on CASIA2.0, even though the data size only differs by 6k (12k vs 18k). Have you encountered this issue, or could it be a problem with my configuration settings?"

[SunnyHaze]

Hi, thanks for your attention.

I am willing to solve your issue, but the current statement is slightly confusing. I am not sure why CAT-Net is related to Segformer. Please kindly explain the exact issue you are caring about. Thank you.

[chchshshhh]

Sorry, I may not have explained my question clearly. What I meant is that during the process of retraining the backbone using the CAT-Net training method (where 1,840 images are drawn from each of the ten tampering datasets, and a total of 18,400 images are iterated in one epoch), the speed is significantly slower compared to training using only the MVSS-Net method (where the entire 12,600 images from CASIA2.0 are iterated in one epoch.

[chchshshhh]

[SunnyHaze]

I see your point.

For this situation, I recommend you slightly adjust it in the parameter of num_workers. Since the data loading process is slightly complicated for CAT-Protocol.

The exact number of num_worker depends on the performance of your device; I'm not sure if the larger is better or the smaller is better. But I believe adjusting this will help to some extent. Please refer to the performance on your device.

If it is still not working or you have a huge time discrepancy between the two protocols, Please feel free to reach out.

Thanks again for your attention to our work.

[Inkyl]

Here is the speed comparison we observed when running the CAT-Protocol and MVSS-Protocol on four 4090 GPUs using the Segformer backbone. Both protocols were tested using datasets that do not contain real images, consisting of approximately 18,000 and 5,000 images, respectively. Based on the results, it is evident that the processing time is proportional to the size of the dataset.

[chchshshhh]

I seem to have discovered the problem: the slowdown is caused by the compRAISE dataset. When this dataset is not included, the speed is normal, but once it is added, the speed slows down several times.

[SunnyHaze]

Hi, thanks for locating the exact issue.

Sorry for that. I forgot a pre-processing that needs to be done on this dataset. Since the resolution of this dataset is relatively huge (often larger than 4000x4000), it becomes a burden for system IO.

Thus, we resize all images to have a long side of 1024 pixels while maintaining the aspect ratio and then read them into IMDL-Benco normally. This can reduce the I/O burden to about 1/16 of the original.

It may take a long time to resize. Recommend using ThreadPooling or a multi-threading method to do it.

This may solve this issue; we are waiting for your feedback!

[SunnyHaze]

Since this dataset was pre-processed quite some time ago within our group, we forgot to explicitly mention this in advance. Sorry then.

[SunnyHaze]

I have find the code for resizing, you can apply this as well

import os
from PIL import Image
from concurrent.futures import ThreadPoolExecutor

def process_image(filename, directory, output_directory, target_size):
    try:
        with Image.open(os.path.join(directory, filename)) as img:
            width, height = img.size
            print(f'Processing Image: {filename} | Resolution: {width}x{height}')

            # 确定长边为1024的缩放比例
            if max(width, height) > target_size:
                if width > height:
                    new_width = target_size
                    new_height = int((target_size / width) * height)
                else:
                    new_height = target_size
                    new_width = int((target_size / height) * width)

                # 调整图片尺寸
                img_resized = img.resize((new_width, new_height), Image.ANTIALIAS)

                # 保存图片到指定文件夹
                output_path = os.path.join(output_directory, filename)
                img_resized.save(output_path)
                print(f'Resized and saved {filename} to {output_directory} with resolution {new_width}x{new_height}')
            else:
                # 如果图片不需要调整，直接复制到目标文件夹
                img.save(os.path.join(output_directory, filename))
                print(f'Image {filename} already meets the target size and was saved without resizing.')
            return 1  # 返回处理成功的计数
    except Exception as e:
        print(f"Cannot process {filename}: {e}")
        return 0  # 返回处理失败的计数

def get_image_resolutions_and_resize(directory='.', output_directory='resized_images', target_size=1024):
    # 创建输出文件夹，如果不存在则创建
    if not os.path.exists(output_directory):
        os.makedirs(output_directory)

    # 获取所有图片文件
    image_files = [f for f in os.listdir(directory) if f.lower().endswith(('png', 'jpg', 'jpeg', 'bmp', 'gif', 'tiff'))]

    # 使用线程池处理图片
    total_processed = 0
    with ThreadPoolExecutor() as executor:
        futures = [executor.submit(process_image, filename, directory, output_directory, target_size) for filename in image_files]

        # 等待所有线程完成并累加处理的数量
        for future in futures:
            total_processed += future.result()

    # 输出总图片数量
    print(f"\nTotal number of images processed: {total_processed}")

# 执行函数
get_image_resolutions_and_resize(
    directory="./compRAISE",
    output_directory="./compRAISE1024"
)

[chchshshhh]

I have find the code for resizing, you can apply this as well

import os
from PIL import Image
from concurrent.futures import ThreadPoolExecutor

def process_image(filename, directory, output_directory, target_size):
    try:
        with Image.open(os.path.join(directory, filename)) as img:
            width, height = img.size
            print(f'Processing Image: {filename} | Resolution: {width}x{height}')

            # 确定长边为1024的缩放比例
            if max(width, height) > target_size:
                if width > height:
                    new_width = target_size
                    new_height = int((target_size / width) * height)
                else:
                    new_height = target_size
                    new_width = int((target_size / height) * width)

                # 调整图片尺寸
                img_resized = img.resize((new_width, new_height), Image.ANTIALIAS)

                # 保存图片到指定文件夹
                output_path = os.path.join(output_directory, filename)
                img_resized.save(output_path)
                print(f'Resized and saved {filename} to {output_directory} with resolution {new_width}x{new_height}')
            else:
                # 如果图片不需要调整，直接复制到目标文件夹
                img.save(os.path.join(output_directory, filename))
                print(f'Image {filename} already meets the target size and was saved without resizing.')
            return 1  # 返回处理成功的计数
    except Exception as e:
        print(f"Cannot process {filename}: {e}")
        return 0  # 返回处理失败的计数

def get_image_resolutions_and_resize(directory='.', output_directory='resized_images', target_size=1024):
    # 创建输出文件夹，如果不存在则创建
    if not os.path.exists(output_directory):
        os.makedirs(output_directory)

    # 获取所有图片文件
    image_files = [f for f in os.listdir(directory) if f.lower().endswith(('png', 'jpg', 'jpeg', 'bmp', 'gif', 'tiff'))]

    # 使用线程池处理图片
    total_processed = 0
    with ThreadPoolExecutor() as executor:
        futures = [executor.submit(process_image, filename, directory, output_directory, target_size) for filename in image_files]

        # 等待所有线程完成并累加处理的数量
        for future in futures:
            total_processed += future.result()

    # 输出总图片数量
    print(f"\nTotal number of images processed: {total_processed}")

# 执行函数
get_image_resolutions_and_resize(
    directory="./compRAISE",
    output_directory="./compRAISE1024"
)

Thank you very much! I sincerely hope that IMDLBenCo can become a widely recognized and utilized benchmark framework for image manipulation detection within the research community, promoting better development in this field!

[SunnyHaze]

Thank you for your encouragement! This will become our motivation to contribute more fantastic projects to the community.

[SunnyHaze]

We have also added guidance about this pre-processing requirement to the documentation.

https://scu-zjz.github.io/IMDLBenCo-doc/guide/quickstart/0_dataprepare.html#preprocessing-high-resolution-images

Sincerely, thanks again for reminding and pointing out this issue!