mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
10.68k stars 881 forks source link

Can't recover deleted artwork on pixiv #5788

Closed verystupidlysimple closed 1 day ago

verystupidlysimple commented 4 days ago

idk if it's supported but other dl apps can actually restore deleted things as long as you have the URL/id after getting the refresh token working i tried restoring the deleted art but it only showed the placeholder wayback machine doesn't having anything about it too image i can share my "config" file if needed

the link is https://www.pixiv.net/en/artworks/119669848 if you want to try it out

thatfuckingbird commented 4 days ago

What are those other apps that can do this? Sounds interesting.

verystupidlysimple commented 3 days ago

What are those other apps that can do this? Sounds interesting.

there's some chrome extensions that can download art, but those can't preview deleted stuff currently my config looks something like this

{
    "extractor":
    {
        "skip": true,
        "sleep": 0,

        "pixiv":
        {
            "base-directory": "dest-directory",
            "directory": ["{user[id]} {user[account]}"],
            "archive": "dest-directory/archive.db",
            "filename": "{id}_p{num}.{extension}",
            "refresh-token": "xxx",
            "metadata": true,
            "tags": "original",
            "skip": "abort:10",
            "postprocessors": ["metadata-pixiv"]
        }
    },

    "postprocessor":
    {
        "metadata-pixiv":
        {
            "name": "metadata",
            "mode": "json",
            "event": "post",
            "filename": "{id}.json"
        }
    },

    "output":
    {
        "skip": false
    }
}

i think tampermonkey can view it but the image is long deleted from the database

Hrxn commented 3 days ago

downloading something that is not actually there, very interesting concept actually!

verystupidlysimple commented 3 days ago

downloading something that is not actually there, very interesting concept actually!

it was there.. at one point before the artist deleted it

biggestsonicfan commented 2 days ago

Pixiv does not keep deleted works on it's servers. Some services do for a while until they are purged, but Pixiv is not usually one of them. A link to a 1200 pixel downscaled version of the image is here but is forbidden to view.

verystupidlysimple commented 2 days ago

Pixiv does not keep deleted works on it's servers. Some services do for a while until they are purged, but Pixiv is not usually one of them. A link to a 1200 pixel downscaled version of the image is here but is forbidden to view.

well thanks for finding the .img but it SHOULD have been restorable clicked the closed button by accident

mikf commented 1 day ago

but is forbidden to view

Pixiv requires a Referer header to access its media files. This link still returns a 404 though.

# directlink extractor sends a Referer header automatically
$ gallery-dl -o extractor.modules=directlink https://i.pximg.net/c/600x1200_90/img-master/img/2024/06/15/23/05/44/119669848_p0_master1200.jpg
[downloader.http][warning] '404 Not Found' for 'https://i.pximg.net/c/600x1200_90/img-master/img/2024/06/15/23/05/44/119669848_p0_master1200.jpg'
[download][error] Failed to download i.pximg.net_c_600x1200_90_img-master_img_2024_06_15_23_05_44_119669848_p0_master1200.jpg
$ wget --referer https://www.pixiv.net/ https://i.pximg.net/c/600x1200_90/img-master/img/2024/06/15/23/05/44/119669848_p0_master1200.jpg
--2024-06-27 14:41:42--  https://i.pximg.net/c/600x1200_90/img-master/img/2024/06/15/23/05/44/119669848_p0_master1200.jpg
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving i.pximg.net (i.pximg.net)... 210.140.139.136, 210.140.139.129, 210.140.139.130, ...
Connecting to i.pximg.net (i.pximg.net)|210.140.139.136|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-06-27 14:41:44 ERROR 404: Not Found.

edit: the /img-original/ URL returns a 404 as well.

verystupidlysimple commented 1 day ago

but is forbidden to view

Pixiv requires a Referer header to access its media files. This link still returns a 404 though.

# directlink extractor sends a Referer header automatically
$ gallery-dl -o extractor.modules=directlink https://i.pximg.net/c/600x1200_90/img-master/img/2024/06/15/23/05/44/119669848_p0_master1200.jpg
[downloader.http][warning] '404 Not Found' for 'https://i.pximg.net/c/600x1200_90/img-master/img/2024/06/15/23/05/44/119669848_p0_master1200.jpg'
[download][error] Failed to download i.pximg.net_c_600x1200_90_img-master_img_2024_06_15_23_05_44_119669848_p0_master1200.jpg

$ wget --referer https://www.pixiv.net/ https://i.pximg.net/c/600x1200_90/img-master/img/2024/06/15/23/05/44/119669848_p0_master1200.jpg
--2024-06-27 14:41:42--  https://i.pximg.net/c/600x1200_90/img-master/img/2024/06/15/23/05/44/119669848_p0_master1200.jpg
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving i.pximg.net (i.pximg.net)... 210.140.139.136, 210.140.139.129, 210.140.139.130, ...
Connecting to i.pximg.net (i.pximg.net)|210.140.139.136|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-06-27 14:41:44 ERROR 404: Not Found.

so the artwork is lost forever? sucks because it was from a well known artist if i were to find a working artwork how would i revive it?

thatfuckingbird commented 1 day ago

Only pixiv controls what appears on their site. If they remove something, it can't be "revived" unless they do it. If they don't and nobody uploaded it to other sites, then it is lost forever. There are some sites that re-host some pixiv content where you might have a chance of finding it: danbooru, gelbooru and artist galleries on exhentai (these usually have the numeric pixiv artist ID in the title) are the main ones. But this is a manual process and can't really be automated.

verystupidlysimple commented 1 day ago

Only pixiv controls what appears on their site. If they remove something, it can't be "revived" unless they do it. If they don't and nobody uploaded it to other sites, then it is lost forever. There are some sites that re-host some pixiv content where you might have a chance of finding it: danbooru, gelbooru and artist galleries on exhentai (these usually have the numeric pixiv artist ID in the title) are the main ones. But this is a manual process and can't really be automated.

me browsing danbooru and then suddenly finding cp: and gelbooru is only cp safebooru probably won't have it since everything is cleaned out manually

and i can't really browse hentai sites but the artist's link is https://www.pixiv.net/en/users/105322906

+JUST WHEN I POSTED THIS THE ARTIST REPOSTED AN OLD DRAWING sucks the Hatate one is gone though

mikf commented 1 day ago

Danbooru allows searching for specific Pixiv IDs, but nobody uploaded it there it seems: https://danbooru.donmai.us/posts?tags=pixiv_id%3A119669848