mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.96k stars 975 forks source link

[Enhancement] Unify the many keyword naming schemes #1646

Open Scripter17 opened 3 years ago

Scripter17 commented 3 years ago

(Split off from #1645 because having 2 things in one issue is dumb)

Different sites have wildly different naming conventions for the keywords in --list-keywords. This is only made worse by some keywords being ambiguous (such as author and user only being different in retweets and stuff)

Issues I've found so far:

This will completely shatter backwards-compatibility with existing databases, but it's something that needs to get addressed eventually and a hypothetical 2.X seems like a good opportunity

thatfuckingbird commented 3 years ago

A further difficulty with this is that in several cases, parts of the JSON returned from site APIs get added verbatim to the saved metadata. Otherwise some extractors would need a lot more code just to rename/restructure these (sometimes complex) JSON fragments. For this reason, completely unifying the keyword conventions would be much harder than just renaming keys already found in the code.

mikf commented 3 years ago

Deviantart has published_time for the unix timestamp of a post while twitter just... doesn't

Most sites that include some kind of date-time information provide them as date, which is a datetime object that can be formatted with strftime format codes ({date:%Y-%m-%d})

Timestamps can be converted to datetime with d and then formatted like above ({published_time!d:%Y-%m-%d}) and datetime objects can be converted to timestamp with T ({date!T})

github-account1111 commented 2 years ago

Timestamps can be converted to datetime

Would it be possible to format something like 8-3-2021 (what Behance use I think)? They don't seem to allow non-zero-padded months and days.

rautamiekka commented 2 years ago

8-3-2021

A pure maintenance and sorting nightmare right there.