Open Gigas002 opened 6 months ago
What exactly do you think is missing here?
The options are always used in the same way (That's why "filename"
is written as extractor.*.filename
, basically).
This is includes "cookies"
, "username"
& "password"
and so on..
Site specific options in configuration.rst
are only mentioned when they are, well, site-specific.
You don't have to search any source code here.
Everything you need is in gallery-dl -K <URL>
, basically, and gallery-dl -E <URL>
.
This is already mentioned in the documentation, btw.
The available replacement keys depend on the extractor used. A list of keys for a specific one can be acquired by calling gallery-dl with the -K/--list-keywords command-line option. For example: [..]
You can also turn on private fields with output.private
in the output here, not sure what you would need it for, though.
Ah, so the extractors there listed only if these have specific options, fair enough. Does that mean, that extractors, that are not listed have only properties, which a listed in docs with wildcard? I suspect something like idolcomplex is using sankaku's properties, but how would I know that beforehand? Someone who sees the code, and/or uses the website would suggest that it is using the sankaku's properties, but that's not said explicitly, which, I personally think, would be more user-friendly.
Also, I was looking at example config and found out that you can specify custom instances, but haven't found anything in detail about it in docs, so I thought these extractors are missing or something (though I still don't see where this part is coming from).
What about sqaure-braced (in doc file) extractors? e.g. extractor.[Danbooru].threshold
, what's the difference between just extractor.danbooru.%some_option%
?
Everything you need is in
gallery-dl -K <URL>
, basically, andgallery-dl -E <URL>
.
There's also --list-extractors
, which lists all extractors and their category values.
I suspect something like idolcomplex is using sankaku's properties
It does not. idolcomplex
has a quite different site layout than sankaku
(at least than https://sankaku.app
, which is what gallery-dl uses) and does not share any of its options.
It inherits from SankakuExtractor
, but that's most likely a "relic" of the past when both sites shared some code.
Also, I was looking at example config and found out that you can specify custom instances, but haven't found anything in detail about it in docs, so I thought these extractors are missing or something (though I still don't see where this part is coming from).
Yeah, that's not properly documented.
Some sites share the same basecategory
and can share config options by using that as key instead of the regular site category
and you can add additional instances for it.
What about sqaure-braced (in doc file) extractors? e.g.
extractor.[Danbooru].threshold
, what's the difference between justextractor.danbooru.%some_option%
?
Same thing. Square-bracketed values are the basecategory
and all extractors with it share this option.
For this specific case it'd be regular danbooru
, atfbooru
, aibooru
, booruvar
, and all Danbooru instances you added manually.
idolcomplex
has a quite different site layout thansankaku
My bad, meant chan.sankakucomplex.com
to be more precise (it's still not abandoned btw).
In any case, thanks for explaining all of this! If the inherited extractors have the parent's props then I guess I can find this out from code.
The reason I ask this question in the first place, is because I'm creating a custom serializing/deserializing tool for gallery-dl
's config (nothing really worth mentioning), and couldn't figure out all the options.
Some extractors configurations are missing docs. Searched the repo's gallery-dl.conf, gallery-dl-example.conf, configuration.rst and an html hosted version
Here's list of missing extractors in hosted html configuration:
I may be missing some things (like, I'm not sure all extractor properties and downloader/output/etc stuff listed too...) but used extractor's init for reference to list these
Personally, I don't mind searching for these in code, but it feels rather complicated, since extractor classes doesn't list serializable properties like this (sry if I'm saying stupid things, not an everyday python programmer, may be this is considered bad coding style):
and you have to figure out the properties by reading full code for each extractor