mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.87k stars 976 forks source link

[Kemono] help with Discord text content #3801

Closed a84r7a3rga76fg closed 1 year ago

a84r7a3rga76fg commented 1 year ago

I can't save the {content} and embeds[url] from a Discord URL, please help if you know how.

            "discord": 
            {
                "#": "discord-specific settings",
                "directory": ["{subcategory} kemono.party {server}", "{channel_name[:25]} {channel}", "{date!s:.10} {id}"],
                "postprocessors": [{
                    "name": "metadata",
                    "event": "post",
                    "filename": "test.txt",
                    "filter": "embed.get('url') or re.search(r'(?i)(redgifs|atomicloli|gfycat|google|drive|onedrive|1drv|mega|xgf|k00|koofr|gigafile|mediafire|porn3dx|gofile|dropbox)', content)",
                    "mode": "custom",
                    "format": "{content}\n{embed[url]:?/\n/}",
                    "archive-format": "{service}_{user}_{id}_{content}_{embed[url]}",
                    "archive": "./gallery-dl/test/test-kemono-discord-archive.db"
                }]
            }
[kemonoparty][debug] Active postprocessor modules: [MetadataPP, MetadataPP]
[kemonoparty][error] An unexpected error occurred: NameError - name 'embed' is not defined. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[kemonoparty][debug]
Traceback (most recent call last):
  File "gallery_dl\job.pyc", line 97, in run
  File "gallery_dl\job.pyc", line 145, in dispatch
  File "gallery_dl\job.pyc", line 314, in handle_directory
  File "gallery_dl\job.pyc", line 549, in _call_hook
  File "<expr>", line 1, in <module>
NameError: name 'embed' is not defined
mikf commented 1 year ago

embed on Discord is called embeds and it is a list (of dicts?) instead of a single dict. Try {embeds:?/\n/}.

a84r7a3rga76fg commented 1 year ago

Getting a new error

[kemonoparty][debug] Active postprocessor modules: [MetadataPP, MetadataPP]
[kemonoparty][error] An unexpected error occurred: AttributeError - 'list' object has no attribute 'get'. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[kemonoparty][debug]
Traceback (most recent call last):
  File "gallery_dl\job.pyc", line 97, in run
  File "gallery_dl\job.pyc", line 145, in dispatch
  File "gallery_dl\job.pyc", line 314, in handle_directory
  File "gallery_dl\job.pyc", line 549, in _call_hook
  File "<expr>", line 1, in <module>
AttributeError: 'list' object has no attribute 'get'

conf

        "discord": {
            "#": "discord-specific settings",
            "directory": ["{subcategory} kemono.party {server}", "{channel_name[:25]} {channel}", "{date!s:.10} {id}"],
            "postprocessors": [{
                "name": "metadata",
                "event": "post",
                "filename": "{_now!s:.16}.txt",
                "filter": "embeds.get('url') or re.search(r'(?i)(redgifs|atomicloli|gfycat|google|drive|onedrive|1drv|mega|xgf|k00|koofr|gigafile|mediafire|porn3dx|gofile|dropbox)', content)",
                "mode": "custom",
                "format": "{content}\n{embeds:?/\n/}",
                "archive-format": "{service}_{user}_{id}_{content}_{embeds[url]}",
                "archive": "~/gallery-dl/test/test-kemono-discord-archive.db"
            }]
        }
mikf commented 1 year ago

Have you tried reading the error messages? It usually helps quite a lot. has no attribute 'get' means you are telling it to access a get where it shouldn't or can't.

Anyway, change "filter": "embeds.get('url') .... to "filter": "embeds ....

a84r7a3rga76fg commented 1 year ago

I changed it and it's saving content to the text file but it skips over embeds and I'm sometimes getting this error, what does it mean?

[kemonoparty][debug] Active postprocessor modules: [MtimePP, MetadataPP, MtimePP]
[kemonoparty][error] An unexpected error occurred: TypeError - can only concatenate str (not "list") to str. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[kemonoparty][debug]
Traceback (most recent call last):
  File "gallery_dl\job.pyc", line 97, in run
  File "gallery_dl\job.pyc", line 145, in dispatch
  File "gallery_dl\job.pyc", line 314, in handle_directory
  File "gallery_dl\job.pyc", line 550, in _call_hook
  File "gallery_dl\postprocessor\metadata.pyc", line 93, in run
  File "gallery_dl\util.pyc", line 911, in check
  File "<expr>", line 1, in <module>
TypeError: can only concatenate str (not "list") to str

Conf

   ```
 "discord": {
            "#": "discord-specific settings",
            "directory": ["{subcategory} kemono.party {server}", "{channel_name[:25]} {channel}", "{date!s:.10} {id}"],
            "postprocessors": [
            {
                "name": "mtime",
                "event": "post"
            },
            {
                "name": "metadata",
                "event": "post",
                "mode": "custom",
                "filename": "{_now!s:.16}.txt",
                "filter": "embeds or re.search(r'(?i)(redgifs|atomicloli|gfycat|google|drive|onedrive|1drv|mega|xgf|k00|koofr|gigafile|mediafire|porn3dx|gofile|dropbox)', content)",
                "format": "{content}\n{embeds:?/\n/}",
                "archive": "~/gallery-dl/archives-metadata/KEMONO-URL-DISCORD.sqlite",
                "archive-prefix": "\fF {subcategory}",
                "archive-format": "_{server}_{channel}_{id}_{hash_sha1(content + (embeds if embeds else ''))}",
                "mtime": true
            }
            ]
mikf commented 1 year ago

TypeError: can only concatenate str (not "list") to str

meaning you cannot do hash_sha1(content + embeds) since embeds is a list.

Use "archive-format": "_{server}_{channel}_{id}_{hash_sha1(content + ''.join(embeds))}",

a84r7a3rga76fg commented 1 year ago

Changed it to that, I'm getting this error with it

[kemonoparty][error] An unexpected error occurred: TypeError - sequence item 0: expected str instance, dict found. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[kemonoparty][debug]
Traceback (most recent call last):
  File "gallery_dl\job.pyc", line 97, in run
  File "gallery_dl\job.pyc", line 145, in dispatch
  File "gallery_dl\job.pyc", line 314, in handle_directory
  File "gallery_dl\job.pyc", line 550, in _call_hook
  File "gallery_dl\postprocessor\metadata.pyc", line 93, in run
  File "gallery_dl\util.pyc", line 911, in check
  File "<expr>", line 1, in <module>
TypeError: sequence item 0: expected str instance, dict found
mikf commented 1 year ago

embeds, when it does contain items, seems to have dicts in it rather then simple strings. I guess you could just convert to str and be done with it: "archive-format": "_{server}_{channel}_{id}_{hash_sha1(content + str(embeds))}",