Closed benoit74 closed 2 years ago
The logs have weird lines with number of images done at the end of the scrape. Looks like some images have been counted in the total but are not reported as done. Probably has something to do with duplicates detection.
[MainThread::2022-05-27 13:13:03,331] DEBUG:Images 403882/448874 [MainThread::2022-05-27 13:13:03,331] DEBUG:Images 403883/448874 [MainThread::2022-05-27 13:13:03,331] DEBUG:Images 403884/448874
It had something to do with duplicate detection. Since this counter + logs are not really used anyway, we just have to remove it for now.
The logs have weird lines with number of images done at the end of the scrape. Looks like some images have been counted in the total but are not reported as done. Probably has something to do with duplicates detection.