mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
12.05k stars 981 forks source link

Trouble extracting from Pixiv #872

Closed OvertPerson closed 4 years ago

OvertPerson commented 4 years ago

When I tried to extract from Pixiv, I get this:

[pixiv][info] Logging in as None [pixiv][error] AuthenticationError: Invalid or missing login credentials

I put in my login details at .conf file but still get this.

biznizz commented 4 years ago

Is your conf properly parsed with the proper commas and quote marks? It looks like this?

"pixiv":
        {
            "username": "USER",
            "password": "PASS",
            "ugoira": true
        },
OvertPerson commented 4 years ago

Yes, but i still get the problem

On Wed, Jul 8, 2020 at 10:14 PM biznizz notifications@github.com wrote:

Is your conf properly parsed with the proper commas and quote marks? It looks like this?

"pixiv": { "username": "USER", "password": "PASS", "ugoira": true },

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mikf/gallery-dl/issues/872#issuecomment-655905501, or unsubscribe https://github.com/notifications/unsubscribe-auth/AODTSBVNZRUNYT4KUS6O7VDR2VG3FANCNFSM4OVDAC3A .

biznizz commented 4 years ago

Are gallery-dl and its dependencies up-to-date?

Run pip list --outdated to see if anything like setuptools is out of date, that might be the issue.

Because I can confirm that Pixiv is still working properly on my end with everything on my end up to date.

EDIT: Also, just have to ask. Is the .conf file in the proper location where it's supposed to be, in %USERPROFILE%?

mikf commented 4 years ago

[pixiv][info] Logging in as None means it didn't find any username (None <-> nothing).

Where exactly is your config file located? Because, as @biznizz noted, it might not be at the right place for gallery-dl to automatically find it (see Configuration)

OvertPerson commented 4 years ago

It is located in %USERPROFILE%/gallery-dl.conf and everything is up to date, but still have issues.

On Thu, Jul 9, 2020 at 2:51 PM Mike Fährmann notifications@github.com wrote:

[pixiv][info] Logging in as None means it didn't find any username (None <-> nothing).

Where exactly is your config file located? Because, as @biznizz https://github.com/biznizz noted, it might not be at the right place for gallery-dl to automatically find it (see Configuration https://github.com/mikf/gallery-dl#configuration)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mikf/gallery-dl/issues/872#issuecomment-656368607, or unsubscribe https://github.com/notifications/unsubscribe-auth/AODTSBWCM6OFNEJSXUFXDWLR2Y3UNANCNFSM4OVDAC3A .

shinji257 commented 4 years ago

Check %APPDATA%\gallery-dl for config.json and try editing that one. That's where it was located for me.

I'll also add that I tried it here and it works for me with credentials set in the file at that location.

OvertPerson commented 4 years ago

I don't have one in %APPDATA% and when I tried to have one there, it still doesn't work

On Fri, Jul 10, 2020 at 9:46 AM Robert Pendell notifications@github.com wrote:

Check %APPDATA%\gallery-dl for config.json and try editing that one. That's where it was located for me.

I'll also add that I tried it here and it works for me with credentials set in the file at that location.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mikf/gallery-dl/issues/872#issuecomment-656774336, or unsubscribe https://github.com/notifications/unsubscribe-auth/AODTSBTIWIU6ZC634U4YKR3R25AWLANCNFSM4OVDAC3A .

biznizz commented 4 years ago

So... you DO have a gallery-dl.conf in %USERPROFILE% and you say your extractor settings are properly put in?

Okay, this is just a few things to eliminate possibilities:

Is the file an actual .conf file? It could be a text file that might SAY it's gallery-dl.conf, but it could actually be gallery-dl.conf.txt.

Are you still getting any errors if you use this input in the CMD window: $ gallery-dl -u <username> -p <password> URL

Are you having any issues with configuration with any of the other supported websites?

OvertPerson commented 4 years ago

It is an actual .conf file and I still get errors with $ gallery-dl -u

-p URL. If a website does not work with gallery-dl "URL" then it doesn't work at all On Sat, Jul 11, 2020 at 1:59 AM biznizz wrote: > So... you DO have a gallery-dl.conf in %USERPROFILE% and you say your > extractor settings are properly put in? > > Okay, this is just a few things to eliminate possibilities: > > Is the file an actual .conf file? It could be a text file that might SAY > it's gallery-dl.conf, but it could actually be gallery-dl.conf.txt. > > Are you still getting any errors if you use this input in the CMD window: $ > gallery-dl -u -p URL > > Are you having any issues with configuration with any of the other > supported websites? > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > , > or unsubscribe > > . >
biznizz commented 4 years ago

Alright, I can confirm that using the -u/--username & -p/--password and the -o/--option separately seems to be broken. Temporarily changing the conf name and using either command and then using valid conf name but removing valid credentials and running either command resulted in The system cannot find the file specified. error.

Restoring my Pixiv credentials and running gallery-dl (PIXIV URL) worked like normal.

Okay, I'm going to ask for a bit of work: Can you make a copy of all the information in your conf file, while MAKING SURE TO CENSOR YOUR CREDENTIALS WITH ASTERISKS, into a text file and upload it here for an experiment?

Also, can you confirm that the credentials you are using are valid, as in you can log out and log back in into Pixiv without issue?

OvertPerson commented 4 years ago

{ "extractor": { "base-directory": "./gallery-dl/", "postprocessors": null, "archive": null, "cookies": null, "cookies-update": false, "proxy": null, "skip": true, "sleep": 0, "path-restrict": "auto", "path-remove": "\u0000-\u001f\u007f", "user-agent": "Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0",

    "artstation":
    {
        "external": false
    },
    "danbooru":
    {
        "username": null,
        "password": null,
        "ugoira": true
    },
    "deviantart":
    {
        "refresh-token": "aa",
        "extra": false,
        "flat": true,
        "folders": false,
        "journals": "html",
        "mature": true,
        "metadata": false,
        "original": true,
        "quality": 100,
        "wait-min": 0
    },
    "exhentai":
    {
        "username": null,
        "password": null,
        "original": true,
        "wait-min": 3.0,
        "wait-max": 6.0
    },
    "flickr":
    {
        "access-token": null,
        "access-token-secret": null,
        "videos": true,
        "size-max": null
    },
    "gelbooru":
    {
        "api": true
    },
    "gfycat":
    {
        "format": "mp4"
    },
    "idolcomplex":
    {
        "username": null,
        "password": null,
        "wait-min": 3.0,
        "wait-max": 6.0
    },
    "imgur":
    {
        "mp4": true
    },
    "instagram":
    {
        "highlights": false,
        "videos": true
    },
    "kissmanga":
    {
        "captcha": "stop"
    },
    "nijie":
    {
        "username": null,
        "password": null
    },
    "oauth":
    {
        "browser": true
    },
    "pixiv":
    {
        "username": "am",
        "password": "a",
        "ugoira": true
    },
    "reactor":
    {
        "wait-min": 3.0,
        "wait-max": 6.0
    },
    "readcomiconline":
    {
        "captcha": "stop"
    },
    "recursive":
    {
        "blacklist": ["directlink", "oauth", "recursive", "test"]
    },
    "reddit":
    {
        "refresh-token": null,
        "comments": 0,
        "morecomments": false,
        "date-min": 0,
        "date-max": 253402210800,
        "date-format": "%Y-%m-%dT%H:%M:%S",
        "id-min": "0",
        "id-max": "zik0zj",
        "recursion": 0,
        "user-agent": "Python:gallery-dl:0.8.4 (by /u/mikf1)"
    },
    "sankaku":
    {
        "username": null,
        "password": null,
        "wait-min": 3.0,
        "wait-max": 6.0
    },
    "seiga":
    {
        "username": null,
        "password": null
    },
    "tumblr":
    {
        "avatar": false,
        "external": false,
        "inline": true,
        "posts": "all",
        "reblogs": true
    },
    "twitter":
    {

"username": "a", "password": "a", "content": true, "retweets": true, "videos": true }, "vsco": { "videos": true }, "wallhaven": { "api-key": null }, "booru": { "tags": false } },

"downloader":
{
    "part": true,
    "part-directory": null,

    "http":
    {
        "adjust-extensions": true,
        "mtime": true,
        "rate": null,
        "retries": 4,
        "timeout": 30.0,
        "verify": true
    },

    "ytdl":
    {
        "format": null,
        "forward-cookies": true,
        "mtime": true,
        "outtmpl": null,
        "rate": null,
        "retries": 4,
        "timeout": 30.0,
        "verify": true
    }
},

"output":
{
    "mode": "auto",
    "progress": true,
    "shorten": true,
    "log": "[{name}][{levelname}] {message}",
    "logfile": null,
    "unsupportedfile": null
},

"netrc": false

}

On Sat, Jul 11, 2020 at 1:12 PM biznizz notifications@github.com wrote:

Alright, I can confirm that using the -u/--username & -p/--password and the -o/--option separately seems to be broken. Temporarily changing the conf name and using either command and then using valid conf name but removing valid credentials and running either command resulted in The system cannot find the file specified. error.

Restoring my Pixiv credentials and running gallery-dl (PIXIV URL) worked like normal.

Okay, I'm going to ask for a bit of work: Can you make a copy of all the information in your conf file, while MAKING SURE TO CENSOR YOUR CREDENTIALS WITH ASTERISKS, into a text file and upload it here for an experiment?

Also, can you confirm that the credentials you are using are valid, as in you can log out and log back in into Pixiv without issue?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mikf/gallery-dl/issues/872#issuecomment-657122550, or unsubscribe https://github.com/notifications/unsubscribe-auth/AODTSBQ7GRRSK2JB2WCIOWLR3DBTTANCNFSM4OVDAC3A .

biznizz commented 4 years ago

@OvertPerson

Okay, I copied your configuration, moved my personal config file to a safe location, and made a new one with all your set up save for using my own personal username and password for Pixiv.

My test run downloaded the gallery into the specified location in base-directory, .gallery-dl, and ran with zero issues. Redacted info in the picture, but compare to your configuration.

Example

The issue seems like it can only be your Pixiv login credentials. As mentioned, please confirm your credentials are valid by logging off/on onto the site itself.

Use your Pixiv ID, not your email, as your username in the conf. And if it is a .conf file and not a .json, then make sure it's in %USERPROFILE%, or C:\Users\USERPROFILE.

Other than that, I don't know what else it could possibly be.

FINAL EDIT: Can you confirm if you have a cache file as described here in configuration? https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst#cache-file

I just noticed that Pixiv credentials are stored there, if you have one, delete it and try making a new one with confirmed valid Pixiv values.

biznizz commented 4 years ago

So, were you able to download from Pixiv, or are you still having issues?

OvertPerson commented 4 years ago

it work

On Wed, Jul 15, 2020 at 9:37 PM biznizz notifications@github.com wrote:

So, were you able to download from Pixiv, or are you still having issues?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mikf/gallery-dl/issues/872#issuecomment-659153260, or unsubscribe https://github.com/notifications/unsubscribe-auth/AODTSBVVNZISAIWDQAGVFIDR3Z7ZHANCNFSM4OVDAC3A .

biznizz commented 4 years ago

Cool, glad everything's working now,

Don't forget to close the issue now that it's resolved.