Please help me with various errors

[tryoutuser]

Here are my installed things : [gallery-dl][debug] Version 1.19.0 [gallery-dl][debug] Python 3.5.7 - Linux-2.6.32-042stab145.3-x86_64-with-Ubuntu-14.04-trusty [gallery-dl][debug] requests 2.25.1 - urllib3 1.26.7

• First, error with config file

[config][warning] Could not parse '/home/user/.gallery-dl.conf': Expecting property name enclosed in double quotes: line 69 column 5 (char 1759)

I don't get what this error want from me and i have checked thoroughly in my config file, there are no values left without quotes before that line and column.

Here is my config from line 1 to that noice 69 :

{
     "extractor":
     {
         "base-directory": "./gallery-dl/",
         "parent-directory": false,
         "postprocessors": null,
         "archive": null,
         "cookies": null,
         "cookies-update": true,
         "proxy": null,
         "skip": true,
         "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0",
         "retries": 5,
         "timeout": 30.0,
         "verify": true,
         "fallback": true,
         "sleep": 0,
         "sleep-request": 0,
         "sleep-extractor": 0,
         "path-restrict": "auto",
         "path-replace": "_",
         "path-remove": "\\u0000-\\u001f\\u007f",
         "path-strip": "auto",
         "extension-map":
         {
             "jpeg": "jpg",
             "jpe" : "jpg",
             "jfif": "jpg",
             "jif" : "jpg",
             "jfi" : "jpg"
         },
         "deviantart":
         {
             "client-id": "value from deviantart",
             "client-secret": "value from deviantart",
             "extra": false,
             "flat": true,
             "folders": false,
            "flat": true,
             "folders": false,
             "include": "gallery",
             "journals": "html",
             "mature": true,
             "metadata": false,
             "original": true,
             "quality": 100,
             "wait-min": 0
         },
         "pixiv":
         {
             "refresh-token": null,
             "avatar": false,
             "tags": "japanese",
             "ugoira": true
         },
         "sankakucomplex":
         {
             "embeds": false,
             "videos": true
         },
         "sankaku":
         {
             "username": null,
             "password": null
         },
         "kemonoparty":
         {
             "cookies": "~/.cache/gallery-dl/cookies.txt",
             "cookies-update": true
         },
     },

• Second, problem downloading on kemono.party

[gallery-dl][debug] Starting DownloadJob for 'https://kemono.party/fanbox/user/871625/post/2782888'
[kemonoparty][debug] Using KemonopartyPostExtractor for 'https://kemono.party/fanbox/user/871625/post/2782888'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): kemono.party:443
[kemonoparty][debug] HTTPSConnectionPool(host='kemono.party', port=443): Max retries exceeded with url: /api/fanbox/user/871625/post/2782888 (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:728)'),)) (1/5)
[urllib3.connectionpool][debug] Starting new HTTPS connection (2): kemono.party:443
[kemonoparty][debug] HTTPSConnectionPool(host='kemono.party', port=443): Max retries exceeded with url: /api/fanbox/user/871625/post/2782888 (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:728)'),)) (2/5)
^C
KeyboardInterrupt

I don't get this one. I have provided my cookies, updated ca-certificates package, upgraded certifi to latest version.

Name: certifi
Version: 2021.10.8
Summary: Python package for providing Mozilla's CA Bundle.
Location: /usr/local/lib/python3.5/dist-packages

At this point i'm stuck trying to figure out what's wrong with this. Please help me how to solve this!

[mikf]

First, error with config file

The config snippet parses successfully when I replace the last comma , with a }. Maybe there's just a , too many at the end?

Second, problem downloading on kemono.party

Probably caused by how Python 3.5 handles SSL/TLS. If you can't update the Python interpreter, it is possible to disable certificate verification with --no-check-certificate or setting "verify" in your config to false.

[tryoutuser]

The config snippet parses successfully when I replace the last comma , with a }. Maybe there's just a , too many at the end?

I edited the config like you suggested and it still throw me the same error

[config][warning] Could not parse '/home/user/.gallery-dl.conf': Expecting property name enclosed in double quotes: line 69 column 5 (char 1759)

Please check if there is something wrong with my complete config here : https://hastebin.com/nesunotofe.yaml

Because i don't know where did i do wrong with the config. I literally took it from config.example in the repo.

I will really appreciate it!

Probably caused by how Python 3.5 handles SSL/TLS.

Alright i will give a try updating python 3.5 to 3.7 or 3.8 and come back with update.

[mikf]

OK, I somehow managed to miss the obvious superfluous comma after the kemonoparty block. It needs to be

         "kemonoparty":
         {
             "cookies": "~/.cache/gallery-dl/cookies.txt",
             "cookies-update": true
         }
     },

(just } and not }, after kemonoparty)

And regarding my suspicion the SSL: CERTIFICATE_VERIFY_FAILED error is due to Python 3.5: Downloading from kemono works on my machine with Python 3.5:

$ python3.5 -m gallery_dl -v https://kemono.party/fanbox/user/6993449/post/506575
[gallery-dl][debug] Version 1.19.1-dev - Git HEAD: fe6ce549
[gallery-dl][debug] Python 3.5.6 - Linux-5.10.56-1-lts-x86_64-with-arch
[gallery-dl][debug] requests 2.25.1 - urllib3 1.26.7
[gallery-dl][debug] Starting DownloadJob for 'https://kemono.party/fanbox/user/6993449/post/506575'
[kemonoparty][debug] Using KemonopartyPostExtractor for 'https://kemono.party/fanbox/user/6993449/post/506575'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): kemono.party:443
[urllib3.connectionpool][debug] https://kemono.party:443 "GET /api/fanbox/user/6993449/post/506575 HTTP/1.1" 200 725
/tmp/kemonoparty/fanbox/6993449/506575_c96取り置き_01_P058kDFYus7DbqAkGlfWTlOr.jpg

So I also don't know what's wrong with your setup. Maybe it's the 7 years old libssl.so that can't handle "modern" certificates, but, even if, there's nothing you could do about that.

[rautamiekka]

Ubuntu 14.04 LTS is supported for another 3y, but it's still comparatively ancient that it wouldn't be a surprise if libssl (or some other pacakge, for that matter) would be too old, so I'd test this problem in a VM with either QEMU or VirtualBox or VMware Player using a newer Ubuntu.

[tryoutuser]

(just } and not }, after kemonoparty)

Damn you are correct. Now it doesn't complain about the config file anymore. After that i found more extra comma in "downloader" block. Thank you!

So I also don't know what's wrong with your setup. Maybe it's the 7 years old libssl.so that can't handle "modern" certificates, but, even if, there's nothing you could do about that.

Weird. Although i have two libssl installed, 1.0.1f-1ubuntu2.27 from official repo and 1.1.1b-1+ubuntu14.04.1+deb.sury.org+1 from launchpad repo. Both properly linked into lib folders. Not sure if 1.1.1b is 7 years old. I don't understand why it fails checking/verifying certificates.

Updating python to 3.7 had no luck. It still throws same SSLError certificate verify failed. Alright i give up.

I will use --no-check-certificate then. You can close this issue. Thank you.

[tryoutuser]

@mikf sorry to bring this up again but i need help for setting up log file.

"output":
     {
         "mode": "auto",
         "progress": true,
         "shorten": true,
         "skip": true,
         "log": "[{name}][{levelname}] {message}",
         "logfile": ~/.cache/gallery-dl/logfile.txt,
         "unsupportedfile": null
     },

Where do i set the variables? Is it in "log"?

Example

"log": {
        "level" : "debug",
        "format": {
                   "debug"  : "debug: {message}",
                   "info"   : "[{name}] {message}",
                   "warning": "Warning: {message}",
                   "error"  : "ERROR: {message}"
                  },
        "format-date": "%Y-%m-%d %H:%M:%S",
        "encoding"   : "utf-8"
}

Is this correct? Also what level that can include all kind of format types?

[mikf]

output.log controls logging output to console, output.logfile to a file.

Both can be a simple string, which gets interpreted as format string for log and file path for logfile. They can also be what is called a Logging Configuration object (what you posted as Example) which allows for a few more detailed options to be set.

Your example is correct, although encoding will be ignored for log (it only applies to logfile and unsupportedfile)

what level that can include all kind of format types?

what do you mean by that?

[Hrxn]

Also what level that can include all kind of format types?

The most verbose/"highest" log level is debug ("level" : "debug"), if that's what you mean?

[tryoutuser]

Also what level that can include all kind of format types?

The most verbose/"highest" log level is debug ("level" : "debug"), if that's what you mean?

Yes this is what i mean so i can get to know what links or files gallery-dl failed when downloading. Thanks.

[tryoutuser]

output.log controls logging output to console, output.logfile to a file.

Both can be a simple string, which gets interpreted as format string for log and file path for logfile. They can also be what is called a Logging Configuration object (what you posted as Example) which allows for a few more detailed options to be set.

Your example is correct, although encoding will be ignored for log (it only applies to logfile and unsupportedfile)

So should i move the variable level, format, format-date and encoding into "logfile"?

I have read the Logging Configuration and confused, that was why i'm asking for help.

I don't know what and where to put in the config.

I want to run gallery-dl without additional --options command/switch except --no-check-certificate and leave it inside screen detached.

[Hrxn]

Yes this is what i mean so i can get to know what links or files gallery-dl failed when downloading. Thanks.

But if it's only about failed downloads, loglevel "debug" is way to explicit. "warning" should be enough.

[tryoutuser]

But if it's only about failed downloads, loglevel "debug" is way to explicit. "warning" should be enough.

Alright thanks.

So this is how i should format it?

"output":
         {
         "mode": "auto",
         "progress": true,
         "shorten": true,
         "skip": true,
         "log": "[{name}][{levelname}] {message}",
         "logfile": {
                     "level" : "warning",
                     "format": {
                                 "info"   : "[{name}] {message}",
                                 "warning": "Warning: {message}",
                                 "error"  : "ERROR: {message}"
                               },
                      "format-date": "%Y-%m-%d %H:%M:%S",
                      "encoding"   : "utf-8",
                      "path" : ~/.cache/gallery-dl/logfile.txt
                    },
         "unsupportedfile": null
         },

[mikf]

You should put the value for path in double quotes, but otherwise this looks OK.

                      "path" : "~/.cache/gallery-dl/logfile.txt"

You might want to add "mode": "a" to the logfile block if you want to keep older log messages. Otherwise logfile.txt gets cleared every time you run gallery-dl.

[Hrxn]

Yeah, that would work...

Just for example, I'll post my current logging setting here, with my options for both "log" and "logfile":

    "output":
    {
        "fallback": true,
        "mode": "auto",
        "shorten": "eaw",
        "skip": true,
        "progress": true,

        "log": {
            "level": "info",
            "format-date": "%Y-%m-%d__%H:%M:%S",
            "format": {
                "debug"  : "\u001b[0;37mDebug  : {name} -> {message}\u001b[0m",
                "info"   : "\u001b[1;37mInfo   : {name} -> {message}\u001b[0m",
                "warning": "\u001b[1;33mWarning: {name} -> {message}\u001b[0m",
                "error"  : "\u001b[1;31mError  : {name} -> {message}\u001b[0m"
            }
        },

        "logfile": {
            "path": "E:\\Home\\Appcache\\gallery-dl\\gallery-dl-log.txt",
            "mode": "a",
            "format": "[{asctime}][{levelname}] {message}",
            "format-date": "%Y-%m-%d__%H:%M:%S",
            "level": "info"
        },

Edit:

Yeah, you have to use double quotes around your path here, good catch: "path" : "~/.cache/gallery-dl/logfile.txt"

This is a string type in JSON parlance and you have to use double quotes.

[tryoutuser]

Ok thank you for all your help and replies!