Config file ignored; 'username', 'date' are not added to filename [SOLVED]

rudolphos commented 1 year ago

I have set up config file like this for reddit: But while downloading using this cmd line: gallery-dl.exe --download-archive archive.sqlite3 %input% It still downloads files with the basic/original file naming template, ignoring that I have added {user[name]} {created!d} in the template. The URL I downloaded was from a subreddit post and username nor date was included in the filename..

Config file is located alongside gallery-dl.exe and I made 2 exact config files gallery-dl.json and config.json since it's uncertain which name works with gdl (there are 2). And both of them were ignored.

afterdelight commented 1 year ago

provide full verbose log

rudolphos commented 1 year ago

Input URL: https://www.reddit.com/r/ChatGPT/comments/zuyi96/chatgpt_fails_to_create_chrome_plugin_due_to/ --verbose
[gallery-dl][debug] Version 1.24.2 - Executable
[gallery-dl][debug] Python 3.8.10 - Windows-10-10.0.19045
[gallery-dl][debug] requests 2.28.1 - urllib3 1.26.12
[gallery-dl][debug] Configuration Files ['C:\\Users\\....\\gallery-dl\\gallery-dl.conf']
[gallery-dl][debug] Starting DownloadJob for 'https://www.reddit.com/r/ChatGPT/comments/zuyi96/chatgpt_fails_to_create_chrome_plugin_due_to/'
[reddit][debug] Using RedditSubmissionExtractor for 'https://www.reddit.com/r/ChatGPT/comments/zuyi96/chatgpt_fails_to_create_chrome_plugin_due_to/'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): oauth.reddit.com:443
[urllib3.connectionpool][debug] https://oauth.reddit.com:443 "GET /comments/zuyi96/.json?limit=0&raw_json=1 HTTP/1.1" 200 6063
[reddit][debug] Using download archive 'archive.sqlite3'
# .\gallery-dl\reddit\ChatGPT\zuyi96 Chatgpt fails to create chrome plugin due to content violation !. Has anyone else seen this _.png

afterdelight commented 1 year ago

there is your gallery-dl config path [gallery-dl][debug] Configuration Files ['C:\\Users\\....\\gallery-dl\\gallery-dl.conf'] have you try to edit that config file?

AlttiRi commented 1 year ago

Rename it to config.json.

Read more carefully: https://github.com/mikf/gallery-dl#configuration

(C:/Users/rudolphos/gallery-dl/config.json)

However, why gallery-dl prints the file path in the log if it has the incorrect name? Or .... was not only one folder name?

rudolphos commented 1 year ago

@afterdelight @AlttiRi I said previously, both are identical files (I edited both). In that case config is not ignored, but when downloading a reddit post, file naming still doesn't work properly - {user[name]} {created!d} are not added to the filename as prefix for some reason.

AlttiRi commented 1 year ago

Put the content of subreddit just to reddit. Maybe your URL is not a subreddit category.

UPD:

Yes, your URL category is just reddit. So, it works as expected.

Use -K to check the category property.

enduser420 commented 1 year ago

this works

the subcategory for a post is submission (subreddit is for a whole subreddit)
since the filename is same for both we can place it in reddit
there is no user[name] key, use author

use date instead of created!d

"reddit": {
"filename": "{author} {date} {id}{num:?_//>02}_{title[:220]!t:R /_/}.{extension}",
"submission": {
    "directory": ["Reddit", "R", "{subreddit}"]
},
"user": {
    "directory": ["Reddit", "U", "{author}"]
}
}


$ gdl --no-download -vc cnf.json https://www.reddit.com/r/ChatGPT/comments/zuyi96/chatgpt_fails_to_create_chrome_plugin_due_to/ https://www.reddit.com/user/Super_Dentist_1094/        
[gallery-dl][debug] Version 1.24.2 - Git HEAD: 2d7d80d3
[gallery-dl][debug] Python 3.11.1 - Windows-10-10.0.19045-SP0
[gallery-dl][debug] requests 2.28.1 - urllib3 1.26.13
[gallery-dl][debug] Configuration Files ['%APPDATA%\\gallery-dl\\config.json', 'cnf.json']
[1/2] https://www.reddit.com/r/ChatGPT/comments/zuyi96/chatgpt_fails_to_create_chrome_plugin_due_to/
[gallery-dl][debug] Starting DownloadJob for 'https://www.reddit.com/r/ChatGPT/comments/zuyi96/chatgpt_fails_to_create_chrome_plugin_due_to/'
[reddit][debug] Using RedditSubmissionExtractor for 'https://www.reddit.com/r/ChatGPT/comments/zuyi96/chatgpt_fails_to_create_chrome_plugin_due_to/'
[reddit][info] Requesting public access token
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.reddit.com:443
[urllib3.connectionpool][debug] https://www.reddit.com:443 "POST /api/v1/access_token HTTP/1.1" 200 151
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): oauth.reddit.com:443
[urllib3.connectionpool][debug] https://oauth.reddit.com:443 "GET /comments/zuyi96/.json?limit=0&raw_json=1 HTTP/1.1" 200 6030

.\gallery-dl\Reddit\R\ChatGPT\Super_Dentist_1094 2022-12-25 13_52_49 zuyi96_Chatgpt_fails_to_create_chrome_plugin_due_to_contentviolation!._Has_anyone_else_seen_this__.png [2/2] https://www.reddit.com/user/Super_Dentist_1094/ [gallery-dl][debug] Starting DownloadJob for 'https://www.reddit.com/user/Super_Dentist_1094/' [reddit][debug] Using RedditUserExtractor for 'https://www.reddit.com/user/Super_Dentist_1094/' [urllib3.connectionpool][debug] https://oauth.reddit.com:443 "GET /user/Super_Dentist_1094/.json?limit=100&raw_json=1 HTTP/1.1" 200 2268
.\gallery-dl\Reddit\U\Super_Dentist_1094\Super_Dentist_1094 2022-12-25 13_5249 zuyi96..._fails_to_create_chrome_plugin_due_to_contentviolation!._Has_anyone_else_seen_this__.png

rudolphos commented 1 year ago

Thanks @enduser420, this works. I wonder why @mikf, @ImportTaste has not mentioned a single word about such simple keys such as author and date in this page: https://github.com/mikf/gallery-dl/blob/master/docs/formatting.md

There are mentions of how to get unix timestamps ({date!T}), but no mention of how to get easily YYYYMMDD {date} or {author}.

Why are many of these keys kept in secret and people have to make countless issue threads to find out about them?

AlttiRi commented 1 year ago

https://github.com/mikf/gallery-dl/blob/master/docs/formatting.md is about formatting, not about available properties. Not every service have the author property, for example.

Use -K/--list-keyword to list all available properties (keys) for a service. [*]

(gallery-dl https://... -K)

but no mention of how to get easily YYYYMMDD {date}

Use one of them: {date:%Y%m%d}/{date:%Y.%m.%d}/{date:%Y-%m-%d} (20221227/2022.12.27/2022-12-27).

rudolphos commented 1 year ago

@AlttiRi --list-keyword ok, thanks, didn't know there's so many. But it should be posted on github anyways since it's easier to search what you need using a search engine and quicker to understand the possibilities of gallery-dl which many are not even aware of from looking over the main page of this tool.

REDDIT - Keywords for directory names:
-----------------------------
allow_live_comments
approved_at_utc
approved_by
archived
author
author_flair_background_color
author_flair_css_class
author_flair_richtext[N][a]
author_flair_richtext[N][e]
author_flair_richtext[N][u]
author_flair_template_id
author_flair_text
author_flair_text_color
author_flair_type
author_fullname
author_is_blocked
author_patreon_flair
author_premium
banned_at_utc
banned_by
can_gild
can_mod_post
category
clicked
content_categories
contest_mode
created
created_utc
date
discussion_type
distinguished
domain
downs
edited
gilded
hidden
hide_score
id
is_created_from_ads_ui
is_crosspostable
is_meta
is_original_content
is_reddit_media_domain
is_robot_indexable
is_self
is_video
likes
link_flair_background_color
link_flair_css_class
link_flair_richtext[N][e]
link_flair_richtext[N][t]
link_flair_template_id
link_flair_text
link_flair_text_color
link_flair_type
locked
media
media_only
mod_note
mod_reason_by
mod_reason_title
name
no_follow
num_comments
num_crossposts
num_reports
over_18
parent_whitelist_status
permalink
pinned
pwls
quarantine
removal_reason
removed_by
removed_by_category
report_reasons
saved
score
secure_media
selftext
selftext_html
send_replies
spoiler
stickied
subcategory
subreddit
subreddit_id
subreddit_name_prefixed
subreddit_subscribers
subreddit_type
suggested_sort
thumbnail
thumbnail_height
thumbnail_width
title
top_awarded_type
total_awards_received
tournament_data[currency]
tournament_data[name]
tournament_data[predictions][N][body]
tournament_data[predictions][N][created_at]
tournament_data[predictions][N][id]
tournament_data[predictions][N][is_nsfw]
tournament_data[predictions][N][is_rtjson]
tournament_data[predictions][N][is_spoiler]
tournament_data[predictions][N][options][N][id]
tournament_data[predictions][N][options][N][image_url]
tournament_data[predictions][N][options][N][text]
tournament_data[predictions][N][options][N][total_amount]
tournament_data[predictions][N][options][N][user_amount]
tournament_data[predictions][N][options][N][vote_count]
tournament_data[predictions][N][resolved_option_id]
tournament_data[predictions][N][status]
tournament_data[predictions][N][title]
tournament_data[predictions][N][total_stake_amount]
tournament_data[predictions][N][total_vote_count]
tournament_data[predictions][N][user_selection]
tournament_data[predictions][N][user_won_amount]
tournament_data[predictions][N][vote_updates_remained]
tournament_data[predictions][N][voting_end_timestamp]
tournament_data[status]
tournament_data[subreddit_id]
tournament_data[theme_id]
tournament_data[total_participants]
tournament_data[tournament_id]
ups
upvote_ratio
url
url_overridden_by_dest
view_count
visited
whitelist_status
wls

mikf commented 1 year ago

$ gallery-dl --help
...
  -K, --list-keywords         Print a list of available keywords and example
                              values for the given URLs

or

https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst#extractorfilename

Both are mentioned in the README.

AlttiRi commented 1 year ago

I think it make sense to create help.txt with the content of gallery-dl --help.

Just gallery-dl --help > docs/help.txt. Currently, this content is "hidden" on GitHub in option.py.

~Also it makes sense to add in README.rst the link to docs/ folder. (Or just list the files of docs/ in README.rst as links.)~ https://github.com/mikf/gallery-dl#configuration Currently, here are 3 links of 5.

mikf commented 1 year ago

@AlttiRi I added a docs/options.md file and linked it in the README. You think this is good like that or should I change something?

AlttiRi commented 1 year ago

Even just a txt would be enough.

Looks nice, but...

width="8%" / width="10%" should help.

Technically, it's a wrong text:

(missed single quotes)

mikf / gallery-dl

Config file ignored; 'username', 'date' are not added to filename [SOLVED] #3464