ryelogheat / xpbot

Analyze a media file, identify the TMDB/IMDB, and generate all the necessary info to upload a torrent to a private tracker using the UNIT3D codebase
55 stars 27 forks source link

Getting Python KeyErrors running the script #21

Closed therealbungus closed 3 years ago

therealbungus commented 3 years ago

I have installed this script in my seedbox but I am getting a KeyError when it is querying the flask app to convert TMDB/IMDB IDs to MAL IDs,

putty_2021-06-09_01-38-42

If I comment out lines 1106 to 1110 to skip this if function, I also run into another KeyError when uploading screenshots.

putty_2021-06-09_01-52-34

Both of them seem to error out when receiving a response.

ryelogheat commented 3 years ago

Can I get the TMDB ID of the content that had the mal_id issue? I want to verify my fix works before pushing it :smiley:

therealbungus commented 3 years ago

64016, it doesn't have a mal_id

My seedbox is running Python 3.8.10, and I have installed ffmpeg, mediainfo, and everything in the requirements.txt

ryelogheat commented 3 years ago

Thanks, Also FYI I was unable to reproduce the KeyError you got when uploading to imgbb so instead I just added an Exception to that part of the code so if you get a KeyError again it'll try and upload to the next image host you have Enabled (If you only have imgbb Enabled then it'll just finish the upload without any screenshots)

therealbungus commented 3 years ago

When I run this script on my personal computer though with Python 3.9.5 and the same config.env file, I don't run into any errors at all.

ryelogheat commented 3 years ago

Hmm, I'm not really sure what could be causing issues then, Can you add a print statement to see what imgbb is returning?

Right under line #63 in upload_screenshots.py can you add the following line print(response) Then remove any private info from the response and post it here

therealbungus commented 3 years ago
{'data': {'id': 'n30Fyys', 'title': '869c1cea46be', 'url_viewer': 'https://ibb.co/n30Fyys', 'url': 'https://i.ibb.co/RhSqKKp/869c1cea46be.png',
'display_url': 'https://i.ibb.co/n30Fyys/869c1cea46be.png', 'size': 501201, 'time': '1623265707', 'expiration': '0', 'image': {'filename':
'869c1cea46be.png', 'name': '869c1cea46be', 'mime': 'image/png', 'extension': 'png', 'url': 'https://i.ibb.co/RhSqKKp/869c1cea46be.png'}, 'thumb':
{'filename': '869c1cea46be.png', 'name': '869c1cea46be', 'mime': 'image/png', 'extension': 'png', 'url': 'https://i.ibb.co/n30Fyys/869c1cea46be.png'},
'delete_url': 'xxxxxxx'}, 'success': True, 'status': 200}

Uploading... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% -:--:--
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /data/xpbot/auto_upload.py:1839 in <module>                                                      │
│                                                                                                  │
│   1836 │   media_info_duration = MediaInfo.parse(torrent_info["raw_video_file"] if "raw_video_f  │
│   1837 │   torrent_info["duration"] = str(media_info_duration.duration).split(".", 1)[0]  # Thi  │
│   1838 │   # Call function to actually take screenshots & upload them (different file)           │
│ ❱ 1839 │   console.print(take_upload_screens(duration=torrent_info["duration"],                  │
│   1840 │   │   │   │   │   │   │   │   │     upload_media_import=torrent_info["raw_video_file"]  │
│   1841 │   │   │   │   │   │   │   │   │     torrent_title_import=torrent_info["title"],         │
│   1842 │   │   │   │   │   │   │   │   │     base_path=working_folder,                           │
│                                                                                                  │
│ /data/xpbot/images/upload_screenshots.py:199 in take_upload_screens                              │
│                                                                                                  │
│   196 │   │   # Now we start the actual upload process                                           │
│   197 │   │   for host_site, host_api in upload_to_host_dict.items():                            │
│   198 │   │   │   # Call the function that actually uploads the images                           │
│ ❱ 199 │   │   │   upload_status = upload_screens(img_host=host_site, api_key=host_api, working   │
│   200 │   │   │   # Check "thumbs_links_dict" to verify images have uploaded and we have all t   │
│   201 │   │   │                                                                                  │
│   202 │   │   │   # if upload_status is equal to "failed" then we simply skip everything below   │
│                                                                                                  │
│ /data/xpbot/images/upload_screenshots.py:72 in upload_screens                                    │
│                                                                                                  │
│    69 │   │   │   │   │   console.print(f"upload to [bold]{img_host}[/bold] has failed!", styl   │
│    70 │   │   │   │   │   return "failed"                                                        │
│    71 │   │   │   │   # Save the thumb and full page links into a dict                           │
│ ❱  72 │   │   │   │   thumbs_links_dict[response["data"]["medium"]["url"]] = response['data'][   │
│    73 │   │   │                                                                                  │
│    74 │   │   │   if img_host == "freeimage":                                                    │
│    75 │   │   │   │   url = "https://freeimage.host/api/1/upload"                                │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯

KeyError: 'medium'

I opened a ticket with my seedbox provider basically saying the same thing as in my first post and this was their response.

Good day,

https://github.com/ryelogheat/xpbot/blob/master/auto_upload.py#L1076

We have manually connected to the URL listed in this line and successfully retrieved the response,

curl http://195.201.146.92:5000/api/?tmdb=10515 513

Since this is a coding related issue please get in touch with the devs to see if anything else besides mediainfo, ffmpeg and the python modules inside requirements.txt are required.

Thing is, I installed just those packages and programs on my local computer and the script runs alright.

ryelogheat commented 3 years ago

Perfect, thanks.

So the mal_api request is working like intended (When they made that curl request it returns literally just the mal ID and nothing else, if there is no match it just returns 0)

The imgbb response is pretty interesting, I noticed the screenshot/video aspect ratio is either 4x3 or something close to it & the resolution is relatively low so I'm thinking that imgbb isn't compressing & storing 3 copies of it (full, medium, & thumb), instead it seems they only host the full res image & thumbnail

I'll add a quick if/else statement to use the thumbnail link if the medium link doesn't exist, that should fix the issue

therealbungus commented 3 years ago

The original video aspect ratio is 4:3, and the resolution is 640x480

ryelogheat commented 3 years ago

Just pushed the fix https://github.com/ryelogheat/xpbot/commit/d3f6b9f1ffd2240e80e6e0572d3355230ef11488

Update the bot on your end & let me know if the image upload works now

therealbungus commented 3 years ago

One question while I have your attention, is there a way to manually specify the content type when manually entering the TMDB ID? I'm trying to manually enter the ID for a TV show and it is pulling the movie ID which is the same number instead.

ryelogheat commented 3 years ago

No not yet, There is an issue open right now asking about something similar so I'll get on that soon. Have you tried supplying the IMDB ID instead? It will look up the correct TMDB ID using the TMDB API endpoint /find/{external_id} so it should be accurate pretty much every time

I outline the available args here

therealbungus commented 3 years ago

I have, and both the IMDB ID and the TMDB ID incorrectly set it as a movie

-tmdb 14705 -imdb tt0282315

ryelogheat commented 3 years ago

OK, I'll add an option for content type probably over this weekend

therealbungus commented 3 years ago

Thank you very much, imgbb still fails to upload with the recent changes but it moves onto the next host and imgbox is handling the images just fine.

ryelogheat commented 3 years ago

Thank you very much, imgbb still fails to upload with the recent changes but it moves onto the next host and imgbox is handling the images just fine.

Wait, what? imgbb still isn't working? I don't have a 4:3 video on hand atm so I can't verify but looking at your print(response) output, it should be working now

Oh well, I was planning on re-writing the image screenshot/upload procedure at some point to be a standalone script & be more reliable so I'll take another look then

therealbungus commented 3 years ago

Yes, it still fails with the same print(response) output shown here

It successfully uploads but prints Upload to imgbb has failed! Going to try the backup now

Thank you for the work you have done on this project