search

Sibusten / derpibooru-downloader

A downloader for imageboards running Philomena, such as Derpibooru.
MIT License
62 stars 6 forks source link

Files Missing Names #27

Open Twi-Hard opened 5 years ago

Twi-Hard commented 5 years ago

While downloading the last 200,000 images I got 112 errors. 111 were files not found because they were missing a filename and 1 was because it was missing an extension. Is there a way to identify the images and fix the errors?

Missing extension: [23:07:00] - Network Error - 203 - Error transferring https://derpicdn.net/img/view/2019/6/28/2077126__safe_derpy+hooves_cutie+mark_cutie+mark+only_no+pony_solo_svg_-dot-svg+available_vector. - server replied: Not Found

Missing filenames: [23:17:56] - Network Error - 203 - Error transferring https://derpicdn.net/img/view/2019/5/15/.png - server replied: Not Found [23:20:09] - Network Error - 203 - Error transferring https://derpicdn.net/img/view/2019/5/6/.jpeg - server replied: Not Found [23:22:58] - Network Error - 203 - Error transferring https://derpicdn.net/img/view/2019/4/25/.jpeg - server replied: Not Found [23:24:30] - Network Error - 203 - Error transferring https://derpicdn.net/img/view/2019/4/20/.jpeg - server replied: Not Found [23:27:34] - Network Error - 203 - Error transferring https://derpicdn.net/img/view/2019/4/9/.jpeg - server replied: Not Found

I can provide the whole log if you want. :)

Sibusten commented 5 years ago

Hello,

These errors appear to be the result of issues with some of the JSON data files on Derpibooru's end. I may be able to write a workaround that attempts to build the url from the image ID and type (assuming the images are actually there and not missing).

Unfortunately, I can't identify the images that are missing filenames since the IDs are not there. The logging system I wrote is very old and not that good, it should have included contextual information like that. When I fix the logging, I'll let you know and you can run it again and get the IDs.