AlphaSlayer1964 / kemono-dl

A simple kemono.party downloader using python.
507 stars 81 forks source link

KeyError: 'index' #92

Closed afterdelight closed 2 years ago

afterdelight commented 2 years ago

Version

Version: latest

Your Command

F:\downloader\kemono.party\kemono-dl-main>kemono-dl.py --cookies kemono.party_cookies.txt --from-file gumroad_aroma.txt --dirname-pattern "/{site}/{service}/{username} [{user_id}]/{title}" --filename-pattern "[{added}] [{id}] [{index}] {filename}.{ext}" --other-filename-pattern "[{added}] [{id}] [{index}] {filename}.{ext}" --extract-links --banner --icon --yt-dlp --archive gumroad_aroma_downloaded.txt --verbose

Description of bug

KeyError: 'index' whne running command

Error messages and tracebacks

``DEBUG:Getting creator json from https://kemono.party/api/creators/ DEBUG:Requesting user json from: https://kemono.party/api/gumroad/user/4487698848613?o=0 ERROR:Unable to get posts for https://kemono.party/gumroad/user/4487698848613 Traceback (most recent call last): File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 640, in start_download self.get_post(url) File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 162, in get_post post = self.clean_post(post, user, site) File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 355, in clean_post self.compile_content_links(new_post, content_soup) File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 284, in compile_content_links new_post['links']['file_path'] = compile_file_path(new_post['post_path'], new_post['post_variables'], new_post['links']['file_variables'], self.other_filename_template, self.restrict_ascii) File "F:\downloader\kemono.party\kemono-dl-main\src\helper.py", line 38, in compile_file_path cleaned_file = clean_file_name(file_split[-1].format(file_variables, post_variables)) KeyError: 'index' DEBUG:Requesting user json from: https://kemono.party/api/gumroad/user/1199729197439?o=0 ERROR:Unable to get posts for https://kemono.party/gumroad/user/1199729197439 Traceback (most recent call last): File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 640, in start_download self.get_post(url) File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 162, in get_post post = self.clean_post(post, user, site) File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 355, in clean_post self.compile_content_links(new_post, content_soup) File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 284, in compile_content_links new_post['links']['file_path'] = compile_file_path(new_post['post_path'], new_post['post_variables'], new_post['links']['file_variables'], self.other_filename_template, self.restrict_ascii) File "F:\downloader\kemono.party\kemono-dl-main\src\helper.py", line 38, in compile_file_path cleaned_file = clean_file_name(file_split[-1].format(file_variables, post_variables)) KeyError: 'index'

AlphaSlayer1964 commented 2 years ago

Index is not a pattern option for --other-filename-pattern. Look at output patterns in the readme.

afterdelight commented 2 years ago

now i got : TypeError: strptime() argument 1 must be str, not None

F:\downloader\kemono.party\kemono-dl-main>kemono-dl.py --cookies kemono.party_cookies.txt --from-file gumroad_aroma.txt --dirname-pattern "/{site}/{service}/{username} [{user_id}]/{title}" --filename-pattern "[{added}] [{id}] [{index}] {filename}.{ext}" --other-filename-pattern "[{added}] [{id}] {filename}.{ext}" --extract-links --yt-dlp --archive gumroad_aroma_downloaded.txt --verbose

DEBUG:Getting creator json from https://kemono.party/api/creators/ DEBUG:Requesting user json from: https://kemono.party/api/gumroad/user/4487698848613?o=0 ERROR:Unable to get posts for https://kemono.party/gumroad/user/4487698848613 Traceback (most recent call last): File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 640, in start_download self.get_post(url) File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 168, in get_post if self.skip_post(post): File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 195, in skip_post if check_date(datetime.datetime.strptime(post['post_variables']['published'], self.date_strf_pattern), self.date, self.datebefore, self.dateafter): TypeError: strptime() argument 1 must be str, not None DEBUG:Requesting user json from: https://kemono.party/api/gumroad/user/1199729197439?o=0 ERROR:Unable to get posts for https://kemono.party/gumroad/user/1199729197439 Traceback (most recent call last): File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 640, in start_download self.get_post(url) File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 168, in get_post if self.skip_post(post): File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 195, in skip_post if check_date(datetime.datetime.strptime(post['post_variables']['published'], self.date_strf_pattern), self.date, self.datebefore, self.dateafter): TypeError: strptime() argument 1 must be str, not None

afterdelight commented 2 years ago

this was my json and links format before update. After update i got the errors

image

AlphaSlayer1964 commented 2 years ago

I will have to look at this new error when at my PC. It shouldn't even be checking date since you don't have that option.

afterdelight commented 2 years ago

could you add index to other file and inline file as the pattern as the previous version has index in it. now my file orders are untidy.

AlphaSlayer1964 commented 2 years ago

now i got : TypeError: strptime() argument 1 must be str, not None

F:\downloader\kemono.party\kemono-dl-main>kemono-dl.py --cookies kemono.party_cookies.txt --from-file gumroad_aroma.txt --dirname-pattern "/{site}/{service}/{username} [{user_id}]/{title}" --filename-pattern "[{added}] [{id}] [{index}] {filename}.{ext}" --other-filename-pattern "[{added}] [{id}] {filename}.{ext}" --extract-links --yt-dlp --archive gumroad_aroma_downloaded.txt --verbose

DEBUG:Getting creator json from https://kemono.party/api/creators/ DEBUG:Requesting user json from: https://kemono.party/api/gumroad/user/4487698848613?o=0 ERROR:Unable to get posts for https://kemono.party/gumroad/user/4487698848613 Traceback (most recent call last): File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 640, in start_download self.get_post(url) File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 168, in get_post if self.skip_post(post): File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 195, in skip_post if check_date(datetime.datetime.strptime(post['post_variables']['published'], self.date_strf_pattern), self.date, self.datebefore, self.dateafter): TypeError: strptime() argument 1 must be str, not None DEBUG:Requesting user json from: https://kemono.party/api/gumroad/user/1199729197439?o=0 ERROR:Unable to get posts for https://kemono.party/gumroad/user/1199729197439 Traceback (most recent call last): File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 640, in start_download self.get_post(url) File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 168, in get_post if self.skip_post(post): File "F:\downloader\kemono.party\kemono-dl-main\src\main.py", line 195, in skip_post if check_date(datetime.datetime.strptime(post['post_variables']['published'], self.date_strf_pattern), self.date, self.datebefore, self.dateafter): TypeError: strptime() argument 1 must be str, not None

The issue is probably because gumroad doesn't record the published date. For a quick fix in src/main.py comment out the if check_date on line 195 if you don't use the date options at all.

afterdelight commented 2 years ago

yes, now it works

AlphaSlayer1964 commented 2 years ago

could you add index to other file and inline file as the pattern as the previous version has index in it. now my file orders are untidy.

I'd recommend deleting the old json/content/embeds and re-creating them while using --skip-attachments

afterdelight commented 2 years ago

ok, but the icons and banner not seperated to a subfolders? it downloaded to a post folder

afterdelight commented 2 years ago

image

afterdelight commented 2 years ago

now the json become [20210926] [hdoHE] json.json instead of [20210926] [hdoHE] post.json with {filename} naming

AlphaSlayer1964 commented 2 years ago

ok, but the icons and banner not seperated to a subfolders? it downloaded to a post folder

you are using --dirname-pattern "/{site}/{service}/{username} [{user_id}]/{title}"

AlphaSlayer1964 commented 2 years ago

now the json become [20210926] [hdoHE] json.json instead of [20210926] [hdoHE] post.json with {filename} naming

correct

afterdelight commented 2 years ago

ok, but the icons and banner not seperated to a subfolders? it downloaded to a post folder

you are using --dirname-pattern "/{site}/{service}/{username} [{user_id}]/{title}"

Yeah, thats because i want to download my files to a folder with post title. the banner and icon dont have specific folder to download to?

afterdelight commented 2 years ago

now the json become [20210926] [hdoHE] json.json instead of [20210926] [hdoHE] post.json with {filename} naming

correct isnt it supposed to be [20210926] [hdoHE] post.json because its a post.json?

AlphaSlayer1964 commented 2 years ago

ok, but the icons and banner not seperated to a subfolders? it downloaded to a post folder

you are using --dirname-pattern "/{site}/{service}/{username} [{user_id}]/{title}"

Yeah, thats because i want to download my files to a folder with post title. the banner and icon dont have specific folder to download to?

Add the title folder to the filename patterns like in the readme

AlphaSlayer1964 commented 2 years ago

isnt it supposed to be [20210926] [hdoHE] post.json because its a post.json?

If you want it to say post edit the write_json function in the mian.py and change filename to 'post' instead of 'json'. Honestly I don't imagine many people are going to use the --json option so I just changed the base filename when remaking the function. I made the filename json so it's consistent with the other non attachment filenames being what type of file it is.

afterdelight commented 2 years ago

the previous version was better in naming. i can see them tidy in order. what about adding back index number to non attachment filenames? how to do it?

AlphaSlayer1964 commented 2 years ago

non attachments would all just have an index of 0 so just put 0 in your filename pattern where you would want the index.

afterdelight commented 2 years ago

ok