[Feature] Overwrite ugoira

[Toyem]

Prerequisites

• [x] Did you read FAQ section in readme.md?
• [x] Did you test with the latest releases or commit ?
• [x] Did you search for existing issues in Issues?

Description

New feature proposed by @cglmrfreeman Add a reprocess function to scan all the ugoira manga types and overwrite the old/bad files.

Actual behavior

This new feature work pretty much like the overwrite one present in the config.ini file only this one is targeted at ugoira.

If the option overwriteUgoira is True in the config.ini file, it overwrites old ugoira that are trying to be download and present in the database db.sqlte (basically ugoira already download). However, it doesn't overwrite them is the option checkLastModified or skipUnknownSize is set to True (same behaviour as the overwrite option).

It's a more specific approach of the overwrite option aiming to replace more efficiently ugoira that might be corrupted as noticed in the issue #1109.

Disclaimer

It doesn't scan ALL the ugoira of the database, but when an ugoira already download is trying to be re-download it can overwrite it.

Improvement

To archive the exact behaviour as stated in the description, a way to do it would be to be able to export the whole database as a txt file and re-download it using the option 4. Download from list of the main menu.

Versions

PixivDownloader2 version 20220311-beta3

[Toyem]

I leave the issue open for some time if you have possible improvements to propose So I'm not going to make a pull request right away

[biggestsonicfan]

I wouldn't mind a u option under Others for an en-mass ugoira re-encode. Better versions of ffmpeg come out, and I actually have quite a bit of corrupt webm files, some from artists who probably don't exist on Pixiv anymore.

[Toyem]

If the feature to re-encode en-mass ugoira is made, is it really useful to keep the option describe in the issue (the overwriteUgoira) ? Cause i feel like it's redondante and not really relevant if a re-encode option is available. Should i delete it and only keep the re-encode one ?

[PatrickL546]

Nice! I had to quickly bash some code to re-encode my ugoira when the new settings were added. Saved me like 200gb's. Now other people don't have to go through what I did.

[Toyem]

This might be of help to someone (https://github.com/PatrickL546/PixivUtil2-batch-downloader/blob/master/!Scripts/7%20Convert%20ugoira%20to%20webm.py) If you just downloaded that script, it has to be in the same folder as 'PixivUtil2'. Haven't added a way to choose an install folder because it's included. I have a script to encode .ugoria using pixivutil2. Doesn't have to be webM as it uses the config settings of pixivutil2. If you don't have .ugoira I think you can use pixivutil2 to make it with "createUgoira", True), you have to redownload it if the zip file is not present

The code that goes through files is actually pretty synthetic and quite good. I probably will improve the actual one with it cause rn, it can take a hell lot of time if you have a lot of zip files.

[Toyem]

@PatrickL546 I didn't find it necessary to keep this option since all new ugoira downloaded are encoded by ffmepg without errors and there is now an option (u) to re-encode old ugoira based on the zip (because the zip always contains all the images, it's only the files like gif or webp that can be wrong).

Re-downloading ugoira with the overwriteUgoira option to re-encode them won't change anything (except consuming your bandwidth 😋 ) because as said before, the zip is perfectly clean and not buggy (regarding issue 1109).

[Toyem]

Sorry to reopen the issue, I just found out that there is a problem with the function i made. The gif is not reprocessed if the artwork doesn't exist on pixiv anymore. It's mainly due to the fact that i choose to select "zip" as base of the reprocessing where using an "ugoira" would have been more logical. Working on it ...