mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
12.02k stars 978 forks source link

I'm having issues with using the config file #652

Closed KirbyFan102 closed 4 years ago

KirbyFan102 commented 4 years ago

It keeps having these weird errors, where it will say it couldn't parse it because a colon was missing somewhere, and it doesn't even say the correct place where it's missing. It said one was missing in line 43, column 13 when it was actually supposed to be line 42, column 42. right now it says I need to put one in line 54 column 19. In this case, I genuinely don't know where it wants me to put it.

mikf commented 4 years ago

The contents of your config file(s) must be valid JSON, so you can use any (online) JSON validator/parser with better error messages than Python's json module to see where you are missing something -> https://github.com/mikf/gallery-dl/issues/576#issuecomment-573525277

As a general rule of thumb, the actual error is usually one line above the one mentioned in the error message and, at least for me, is because of a missing comma at the end of a key-value pair or because I left a comma after the last key-value pair at the end of a block

{
    "_": "example errors",
    "k1": "missing comma"
    "k2": "ok",
    "k3": "comma after the last value",
}
KirbyFan102 commented 4 years ago

After replacing several :s with colons I encountered another problem, I'm having another problem. Only one parser seems able to help, and it says "expecting 'STRING', got ']'. But I don't know what to replace it with. Being literal and substituting the ] for the word STRING also fails.

KirbyFan102 commented 4 years ago

It's right before the part where it says 'downloader'.

KirbyFan102 commented 4 years ago

The offending section: "booru", { "tags": false , }, (this is line with the problematic '}')

"downloader":
Hrxn commented 4 years ago

If you have only "tags": false inside that JSON object (inside the { }), remove that comma below "tags": false, because it implies that there must follow something else.

KirbyFan102 commented 4 years ago

This is what it looks like now:

"booru", { "tags": false }, , "downloader":

The cmd now says Expecting value: line 178 column 5 (char 4259), which is somehow related to the ',' before downloader. I tried to remove it, but then it said it needed a colon there. Christ.

Hrxn commented 4 years ago

Yeah, there's one unneeded/wrong comma there now.. See? There are two.

},
,
"downloader":
KirbyFan102 commented 4 years ago

But then I removed the lower comma and it asked me for a comma.

Hrxn commented 4 years ago

Okay.. you know what, can you post the full content of your config file, ideally here -> https://gist.github.com/ and then link the gist here (just remember to "xxx"-out all usernames, passwords and tokens and so on), and I will have a look then.

KirbyFan102 commented 4 years ago

https://gist.github.com/KirbyFan102/9d272347541605f92707c5f5cd6f918b

There are probably still more errors to find here, this is just the one I got stuck on.

Mrfiregem commented 4 years ago

Here's your config in valid json, I think the program should accept it now, but I can't test it atm.

KirbyFan102 commented 4 years ago

Ok, thanks. I will test it and get back to you.

Mrfiregem commented 4 years ago

The comma warning was due to a missing curly brace in deviantart.postprocessors

KirbyFan102 commented 4 years ago

Okay, now it's complaining about 'extra data' on line 2, column 14,

KirbyFan102 commented 4 years ago

Which is the first line. There's NOTHING there.

Hrxn commented 4 years ago

Someone was faster 😄 ..

The problem starts actually here:

            "metadata": true,
            "postprocessors": [{
                "name": "metadata",
                "mode": "custom",
                "format": "{description}\n",
            "cookies": "%HOMEPATH%/Google Drive/cookies-deviantart-com.txt",
            "cookies-update": true,
            "client-id": "xxxxxx",
            "client-secret": "xxxxxxxx",
            "filename": "{filename}.{extension}"
        },

And it's not just a curly brace, the array syntax ("postprocessors": [{) is wrong. The array is not closed, it has to be closed (]), depending on what postprocessors you intended to use here..

(On top of a wrong object syntax.. it has to be "extractorname" : { ... }, )

But the link posted by @Mrfiregem is correct (https://hastebin.com/raw/ruqekohaqo), that's valid JSON for me as well, that should definitely work..

KirbyFan102 commented 4 years ago

The file @Mrfiregem gave does correct that first problem. And, if by object syntax you mean putting :s instead of ,s after the extractor names, it corrects that as well.

Hrxn commented 4 years ago

And, if by object syntax you mean putting :s instead of ,s after the extractor names, it corrects that as well.

Yeah, basically. And yes, it's a valid JSON file, as said, so all problems are corrected.

KirbyFan102 commented 4 years ago

They're not, actually. It's still not parsing it correctly. It's complaining, as mentioned previously about extra data on the first line: Could not parse 'C:\Users\User\gallery-dl.conf': Extra data: line 2 column 14 (char 14)

biznizz commented 4 years ago

I usually find that looking at the config with a program that keeps good track of line rows like Notepad++ helps me to find when I've messed up the JSON structure, like when I altered my config for the e621 extractor, but forgot to close it properly.

KirbyFan102 commented 4 years ago

I somehow doubt messing up the structure has anything to do with it at this point, I mean, it's the first line. There's no way I messed up the the first line, it comes before everything that's actually important.

KirbyFan102 commented 4 years ago

What IS this 'extra data'? What's going on?

KirbyFan102 commented 4 years ago

So I put the file in a JSON parser. This is the result: Parse error on line 2:

"extractor": { "base-direc --------------^ Expecting 'EOF', '}', ',', ']', got ':'

KirbyFan102 commented 4 years ago

Okay, I figured it out. The extra data it needed was a curly brace in the line before "extractor". I guess @Mrfiregem made a mistake when he posted that file, since that was an error that didn't happen before. Otherwise, it does seem he fixed the issue. It's entirely possible new problems may happen in the future, since it feels as though every time I add something to that damn file, it breaks, but I'll close this issue for now.

(I actually fixed it 3 hours back, but got caught up in stuff, so... sorry I didn't update you guys on the situation when I should have.)

Mrfiregem commented 4 years ago

In my case, if a program is configured through json, I usually write the file in yaml instead and convert it later. Maybe look into that if you keep running into syntax issues

KirbyFan102 commented 4 years ago

Well, I'm only just beginning to learn how to use gallery-dl and I tend to edit the config file using Windows' notepad program. Since I add configuration options as they become useful, I wind up editing it quite frequently. Maybe I should use a more specialized editor?

biznizz commented 4 years ago

Well, I'm only just beginning to learn how to use gallery-dl and I tend to edit the config file using Windows' notepad program. Since I add configuration options as they become useful, I wind up editing it quite frequently. Maybe I should use a more specialized editor?

As I mentioned, try using Notepad++, it's much more useful than the stock Microsoft one.