This Python project aims to efficiently compare large datasets of images to identify duplicates. The user can specify various actions to take with the duplicates, such as saving them to a text file, moving them to a separate folder, or deleting them altogether. The pipeline involves several stages including user input, discovery, comparison, and result generation.
The user provides input specifying the dataset and desired actions for handling duplicates.
The program scans the dataset to identify all images and their locations.
Metadata of the images, such as size, date, and camera information, are compared. If the metadata is the same, no further comparisons need to be made. This is however the users to decide should they want to go for a deep scan.
If the metadata is not identical, preprocessing steps are applied to standardize the images:
The program generates a report indicating which images are duplicates based on the specified criteria. The user can then choose to save this list to a text file, move the duplicates to a separate folder, or delete them.
Contributions are welcome! Feel free to fork the project and submit pull requests for any improvements or new features.
This project is licensed under the MIT License. See the LICENSE file for details.