mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.87k stars 976 forks source link

[help] with parent-directories and instagram tagged #2024

Closed CaptainJawZ closed 2 years ago

CaptainJawZ commented 2 years ago

Hello, I've been all day trying to get this to work by attempting several solutions i've found on the issues section, but while I've made it work with reddit for example, I can't make it work with instagram.

I want to be able to download tagged posts, but have all the files put in a single directory named "tagged" right under {username} (which should be the username of the instagram user whose link I provided, not the username of the person who uploaded and tagged them) So for instance from https://instagram.com/PERSON I want to have files downloaded in ./gallery-dl/PERSON/all the files from posts, reels, highlights, stories and chanels (currently this works) ./gallery-dl/PERSON/tagged/all the tagged posts

Instead what i get is: ./gallery-dl/TAGGEDUSER/tagged/pic

I hope that example is clear I managed to make it work on reddit, by following other tips found on here, altho with a different file structure to the one i want on instagram:

"reddit": {
      "directory": ["{category}", "{author}"],
      "parent-directory": true,
      "refresh-token": "redacted"
    },
    "redgifs": {
      "directory": ["{userName}"]
    },
    "imgur": {
      "mp4": true
    }

And here is what I got for instagram (there may be a few comas and stuff missing as i modified the json a bit for privacy reasons)

    "instagram": {
      "username": "redacted",
      "password": "redacted",
      "sleep-request": 5,
      "include": "all",
      "directory": ["{username}"],
      "parent-directory": true,
      "parent-metadata": "parent",
      "highlights": {
        "reverse": "true",
        "directory": ["{username}"]
      },
      "stories": {
        "reverse": "true",
        "directory": ["{username}"]
      },
      "tagged": {
        "directory": {
          "'parent' in locals()": [
            "{parent[username]}",
            "tagged"
          ]
        }
      }

The issue is that {parent[username]} returns none, also tired a wide array of variations {username}, {'parent'[username]}, etc, all of which either return none or the username of the user who tagged the post.


also this is unrelated, but while I'm askin for help, there is a bunch of extractors whom I use who use the same postprocessor settings, all of them being for manga websites, for instance i got this:

"mangahere": {
      "chapter-reverse": true,
      "directory": ["Manga", "{manga}", "{manga} - {chapter}{chapter_minor}"],
      "postprocessors": [
        {
          "name": "zip",
          "compression": "store",
          "mode": "safe",
          "extension": "cbz"
        }
      ]
    },

Just slightly modified with extractor name, and directory structure as some of them change a bit, my question is, is there a way to like, group all of these extractors, so they share the same postprocessor settings, so if I were to modify them i wouldnt have to be modifying with several entries.

Thanks for all the help in advance!

mikf commented 2 years ago

The instagram extractors do not use a parent-child relationship as for example reddit and imgur do, so all the parent-*** options don't have any effect here.

That said, it is currently also not possible to get the directory structure you want for instagram tagged files, although that would only need a small change to add the originating username (PERSON) to the available metadata entries.


is there a way to like, group all of these extractors, so they share the same postprocessor settings

There are at least two ways of doing that.

The first is placing the postprocessors settings under extractor and not extractor.mangahere etc., and adding a whitelist containing the category names you want this to be used with:

"extractor": {
    "postprocessors": [
        {
            "name": "zip",
            "compression": "store",
            "mode": "safe",
            "extension": "cbz",
            "whitelist": ["mangahere", "mangadex", "..."]
        }
    ],
    "mangahere": {
        "chapter-reverse": true,
        "directory": ["Manga", "{manga}", "{manga} - {chapter}{chapter_minor}"]
    }
}

The other is to create a cbz post processor in the global postprocessor block like here and then to only refer to it by name:

{
    "extractor": {
        "mangahere": {
            "chapter-reverse": true,
            "directory": ["Manga", "{manga}", "{manga} - {chapter}{chapter_minor}"]
            "postprocessors": ["cbz"]
        }
    },
    "postprocessor": {
        "cbz": {
            "name": "zip",
            "compression": "store",
            "mode": "safe",
            "extension": "cbz"
        }
    }
}
CaptainJawZ commented 2 years ago

Thank you for the clarification regarding instagram!, for the time being i'll disable tagged. Also thank you so much for expalining how to do the manga structure, tried the second solution and worked perfectly! its a very cool way to make the json config a bit modular, pretty appreciated.

mikf commented 2 years ago

There are now several tagged_* metadata fields referencing the original user (https://github.com/mikf/gallery-dl/commit/2aaac3c997b16b5a09eeac72ec27056e61f91eaf), so the following will now do what you want:

"tagged": {
    "directory": ["{tagged_username}", "tagged"]
}
CaptainJawZ commented 2 years ago

Much appreciated thank you very much!