search

Imageomics / cautious-robot

Simple images from CSV downloader that runs and records checksums on downloaded image folder.
MIT License
2 stars 0 forks source link

Add BuddyCheck verifier #8

Closed egrace479 closed 4 months ago

egrace479 commented 5 months ago

Adds a verification step through the BuddyCheck class. It checks that checksums in the source CSV match those calculated with sum-buddy after download (compares both filename and checksum columns to not undercount duplicates with different names, since filenames are forced to be unique). Also notes the number of images downloaded and how many were expected (based on number of non-null values in the selected filename column).

This is the last update for the alpha version.

egrace479 commented 5 months ago

I also noticed that sample images weren't downloading when I turned off streaming, and have turned it back on. The new verification features should alert the user to instances of 200 responses where streaming is interrupted.

egrace479 commented 4 months ago

@johnbradley, now it throws an exception and exits in the main files. Thanks for catching that :)