fbacher / script.video.randomtrailers

A Kodi addon which plays movie trailers from various sources. Based on defunct addon screensaver.randomtrailers
GNU General Public License v2.0
1 stars 2 forks source link

Black screen in addon and repeated errors in kodi.log #15

Open bullwinkl opened 2 years ago

bullwinkl commented 2 years ago

Upon running all versions released so far I get a black screen after license screen and the kodi log grows to an unmanageable size with an error repeated thousands of times in it. The error has a different trailer listed each time Kodi runs, but always the same trailer for the duration of the log, if that makes sense. It repeats until the log is too big for me to view. I've attached just a small section of the kodi log to show you what I mean.

logselection.txt

fbacher commented 2 years ago

I will look at that specific bug. I may have already fixed it. For about the past week I noticed a related nasty bug that causes TMDb trailers to display very slowly.

I am clearly not doing enough testing. The main culprit in my testing is that I am not 1) blowing away my entire trailer cache 2) not spending enough time testing each trailer source. I am too eager to release.

That said, I am working on the issue. I am working about 1/2 speed for the next few weeks on this.

The general problem with black screen has been around a few months, actually longer. I haven't noticed because I haven't tested long enough on the scenario that triggers it. The easy way for me to trigger it is to clear my trailer cache and then download TMDb trailers. I generally use two machines for testing. One, which is also my main movie box for home use, I don't tend to blow away my downloaded trailers. On the other, I do blow away the trailers periodically. However, if I also have it configured to download from multiple sources, then the problem doesn't stick out so much, since trailers play from the other sources.

The fundamental problem is that with no cached trailers, you have to discover everything and since there are a lot of movies without trailers you can see a blank screen for a bit. It can also take longer to download the trailer than it takes to play it. Normally I have seen this as a startup issue that goes away after a decent number of trailers have been cached. Over time I tweaked things to reduce the problem when downloading TMDb trailers. These include: at startup, first find all of the TMDb movies with trailers that are cached and feed those to the "random player". After that, resume searching for more movies with trailers. However, due to a change, this is not working so well, and hasn't for a while. I should have caught it sooner.

I also need to test your specific scenario, which I did a while back, but apparently not long enough.

Do you see any trailers being played?

bullwinkl commented 2 years ago

I've not seen any trailers yet--just get the black screen and then if I don't exit after a while, Kodi becomes unresponsive--I'd imagine due to the event log filling.

FYI, this is Kodi 19.2 running on an Nvidia Shield with Kodi installed on an external SSD, and with a shared Kodi database on another server. All movies and local trailers are on SMB shares from the server.

On Sat, Oct 23, 2021 at 3:57 AM fbacher @.***> wrote:

I will look at that specific bug. I may have already fixed it. For about the past week I noticed a related nasty bug that causes TMDb trailers to display very slowly.

I am clearly not doing enough testing. The main culprit in my testing is that I am not 1) blowing away my entire trailer cache 2) not spending enough time testing each trailer source. I am too eager to release.

That said, I am working on the issue. I am working about 1/2 speed for the next few weeks on this.

The general problem with black screen has been around a few months, actually longer. I haven't noticed because I haven't tested long enough on the scenario that triggers it. The easy way for me to trigger it is to clear my trailer cache and then download TMDb trailers. I generally use two machines for testing. One, which is also my main movie box for home use, I don't tend to blow away my downloaded trailers. On the other, I do blow away the trailers periodically. However, if I also have it configured to download from multiple sources, then the problem doesn't stick out so much, since trailers play from the other sources.

The fundamental problem is that with no cached trailers, you have to discover everything and since there are a lot of movies without trailers you can see a blank screen for a bit. It can also take longer to download the trailer than it takes to play it. Normally I have seen this as a startup issue that goes away after a decent number of trailers have been cached. Over time I tweaked things to reduce the problem when downloading TMDb trailers. These include: at startup, first find all of the TMDb movies with trailers that are cached and feed those to the "random player". After that, resume searching for more movies with trailers. However, due to a change, this is not working so well, and hasn't for a while. I should have caught it sooner.

I also need to test your specific scenario, which I did a while back, but apparently not long enough.

Do you see any trailers being played?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/fbacher/script.video.randomtrailers/issues/15#issuecomment-950113255, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGCVCQMHDPIWIEO6NN5XECLUIJTGZANCNFSM5GPOGW2A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- [image: created with MySignature.io] https://mysignature.io/?utm_source=logo

Matt Smeltzer mobile: 812-698-0373 email: @.*** [image: created with MySignature.io] https://www.facebook.com/mattrsmeltzer https://mysignature.io/editor/?utm_source=freepixel

fbacher commented 2 years ago

Actively working on. I have it working, but I want to test it at least a few days. Are you open to using a pre-build in a few days so I might be able to avoid making a fool out of myself again?

bullwinkl commented 2 years ago

Absolutely. Just give a yell whenever you need me.

On Wed, Nov 3, 2021 at 6:05 PM fbacher @.***> wrote:

Actively working on. I have it working, but I want to test it at least a few days. Are you open to using a pre-build in a few days so I might be able to avoid making a fool out of myself again?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/fbacher/script.video.randomtrailers/issues/15#issuecomment-960198829, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGCVCQMOFGWMGDTZX4YIG2LUKG54JANCNFSM5GPOGW2A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- [image: created with MySignature.io] https://mysignature.io/?utm_source=logo

Matt Smeltzer mobile: 812-698-0373 email: @.*** [image: created with MySignature.io] https://www.facebook.com/mattrsmeltzer https://mysignature.io/editor/?utm_source=freepixel

fbacher commented 2 years ago

Can you try 2.0.7.beta.5 ? It is on it's own branch. I did not create .zip for it. Let me know if you need some other format.

Sorry that it took me so long. I had several commitments that got in the way. I expect that trailers will be slow to download (about 2x time to play the trailer), but this may not be the case for you. Seems youtube is throttling downloads by youtube-dl, but this may happen only after some threshold is met. If you cache your trailers, then it should not take long before you have a decent number of trailers local.

Since you have no local trailers, be sure to have your settings configured to: Include trailers from library (if you don't select then no library trailers will be shown) Include movies with remote trailers Include movies with no trailer info

So... I expect that 1) you will probably see some black screen, probably lasting five minutes or more since: 1) there are no local trailers to play 2) In random order, a movie from the library will be picked to find a trailer for a) TMDb will be queried for the movie b) if the movie is found and it passes whatever filters you have configured (MPAA, etc..) AND it has a trailer, then the trailer will be downloaded 3) Only one download from youtube will occur at a time

Trailer downloads occur in a separate thread from discovery of data from TMDb, therefore even while trailer is downloading, more will be being searched for. Up to two additional trailers will be queued up to download while another is downloading.

While playing a trailer, up to three other trailers will be located for downloading and they will be serially downloaded in the background.


The code will try to play 3/4 of the trailers from local or downloaded trailers, with 1/4 of them being downloaded. This should guarantee that you don't have black-screen for very long, even though for a while you will see duplicates. It should not take very long before a sufficient number of downloaded trailers exist so that the duplicates are not noticed so much.

If you play trailers for multiple sources (Library, TMDb, TFH...) then the above search and download rules apply to each source (there is a separate set of threads for each source).

I am very interested to hear what your experience is: 1) Do you see any trailers (give 10 minutes or so) 2) Do you see errors in log 3) Do you see evidence that it is stuck (like you saw before) 4) CPU load. I expect that the first few minutes of every run of Kodi that CPU load will be pretty heavy for perhaps a minute, and then quickly lower. If audio normalization is enabled, then CPU usage is pretty heavy as a movie is normalized. Since normalized trailers are cached, then the normalization costs should fairly quickly decrease. 5) Disk usage. By default the cache is in /userdata/addon_data/script.video.randomtrailers/cache. How large does it grow and is this a problem? No attempt is made to compress the trailers; it is simply a copy of whatever the original trailer source is (plus any audio normalization).

Should I change the default settings to include downloading remote trailers (the path in Kodi is a URL instead of a local file)? Same for when no trailer info exists for local trailers? I think that I should. Originally the cost was substantial for enabling these, but with the caching changes this is not so bad.

File Descriptor Limit There is a bug that I found ages ago that appears to be in Kodi that I need to follow up with. There is a file descriptor leak. The leak occurs for output files for the video device. Eventually your file descriptor limit is reached and Kodi stops playing and must be forceably stopped and restarted. You can reduce how often this occurs by increasing your file descriptor limit (mine is 1024, which allows me to play trailers for over a day). I primarily use Ubuntu running on an Intel CPU using the integrated GPU. Perhaps the behavior is different on other configurations.

bullwinkl commented 2 years ago
  1. Got first trailer after 7 minutes of black screen on first run
  2. After a couple of trailers, got permanent black screen. Kodi remained responsive, but I didn't grab log from then. Uninstalled/Reinstalled and I tailed the log to see what was going on. This error kept popping up every 10 seconds or so.

2021-12-14 15:51:45.174 T:6953 ERROR <general>: Skipped 159 duplicate messages.. 2021-12-14 15:51:45.174 T:6953 ERROR <general>: [rt_backend.discovery.discover_tmdb_movies.DiscoverTmdbMovies.process_page:1942:] LEAK Traceback StackTrace StackDump (most recent call last) File "/data/user/0/org.xbmc.kodi/cache/apk/assets/python3.8/lib/python3.8/threading.py", line 890, in _bootstrap self._bootstrap_inner() File "/data/user/0/org.xbmc.kodi/cache/apk/assets/python3.8/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.video.randomtrailers/resources/lib/discovery/discover_tmdb_movies.py", line 136, in run self.run_worker() File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.video.randomtrailers/resources/lib/discovery/discover_tmdb_movies.py", line 228, in run_worker self.discover_movies( File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.video.randomtrailers/resources/lib/discovery/discover_tmdb_movies.py", line 336, in discover_movies more_to_get = self.discover_movies_using_search_pages( File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.video.randomtrailers/resources/lib/discovery/discover_tmdb_movies.py", line 1289, in discover_movies_using_search_pages self.get_movies(url=url, data=data, File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.video.randomtrailers/resources/lib/discovery/discover_tmdb_movies.py", line 1810, in get_movies movies = self.process_page(info_string, url=url) File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.video.randomtrailers/resources/lib/discovery/discover_tmdb_movies.py", line 1942, in process_page movie_summary_parser: ParseTMDbPageData = ParseTMDbPageData(movie_entry) File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.video.randomtrailers/resources/lib/discovery/utils/parse_tmdb_page_data.py", line 27, in __init__ self._tmdb_movie: TMDbMoviePageData = TMDbMoviePageData() File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.video.randomtrailers/resources/lib/common/movie.py", line 385, in __init__ super().__init__(movie_id) File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.video.randomtrailers/resources/lib/common/movie.py", line 352, in __init__ self.set_tmdb_id(int(movie_id)) TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

  1. Kodi remains responsive now, but I haven't sit and let it run for an hour to see if it stays yet.
  2. I've not noticed any slowdown on the machine. I haven't run anything to keep cpu into on the screen while it runs.
  3. I'll let you know about cache size once I have the opportunity to let it sit for quite a while.

Perhaps coincidentally, nearly every remote trailer that has run has been a TFH trailer, so I'm guessing something is hooey with TMDB discovery?

bullwinkl commented 2 years ago

...or maybe I could've just read my own error message to see that there was something up with TMDB discovery it didn't like. Derp.

bullwinkl commented 2 years ago

I sat and let it run for 90 minutes or so. I got a few TFH trailers, but mostly just extended periods of black screens. Here's the log kodi.zip .

Cache grew to about 250mb. missingtrailers.playlist is empty. Viewed.Playlist contents are: 2021-12-14 15:50 START There's Something About Mary (1998) - TFH Trailer https://youtu.be/QInh4LhZAFY 2021-12-14 15:55 The Americanization of Emily (1964) - TFH Trailer https://youtu.be/eYaG8WSHLpA 2021-12-14 16:04 Ski Troop Attack (1960) - TFH Trailer https://youtu.be/rTSVDfPsHZo 2021-12-14 16:19 Thunderbolt and Lightfoot (1974) - TFH Trailer https://youtu.be/3gxEdQcMEt0 2021-12-14 16:26 Four Friends (1981) - TFH Trailer https://youtu.be/557-pQcUv70 2021-12-14 16:34 Touch of Evil (1958) - TFH Trailer https://youtu.be/fmYsBZTkFzo 2021-12-14 16:49

So yeah, all TFH trailers. This is with the settings here: settings.zip .

fbacher commented 2 years ago

I committed several changes to the 2.0.7 beta 5 branch that you should try. Among the things I did were:   1) There is a bug in Kodi that shows up post 19.0 that causes my code to determine the locale/country code to fail. I patched it to default to en_US.  2) I switched from using youtube-dl to yt-dlp. This means you must manually install sendtokodi: see https://github.com/firsttris/plugin.video.sendtokodi. (I'll provide yt-dlp by some other means later).

Frank

On 12/14/21 4:32 PM, bullwinkl wrote:

I sat and let it run for 90 minutes or so. I got a few TFH trailers, but mostly just extended periods of black screens. Here's the log kodi.zip https://github.com/fbacher/script.video.randomtrailers/files/7715129/kodi.zip .

Cache grew to about 250mb. missingtrailers.playlist is empty. Viewed.Playlist contents are: |2021-12-14 15:50 START There's Something About Mary (1998) - TFH Trailer https://youtu.be/QInh4LhZAFY 2021-12-14 15:55 The Americanization of Emily (1964) - TFH Trailer https://youtu.be/eYaG8WSHLpA 2021-12-14 16:04 Ski Troop Attack (1960)

So yeah, all TFH trailers. This is with the settings here: settings.zip https://github.com/fbacher/script.video.randomtrailers/files/7715160/settings.zip .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/fbacher/script.video.randomtrailers/issues/15#issuecomment-994104641, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACRK6CK4OH4EYMIHIZNUMNDUQ7AW5ANCNFSM5GPOGW2A. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

bullwinkl commented 2 years ago

Cloned latest branch and installed sendtokodi. Same situation. Repeated error every 10 seconds 2021-12-18 19:52:10.931 T:20420 ERROR <general>: [rt_backend.discovery.abstract_trailer_fetcher.AbstractTrailerFetcher.run_worker:243:] LEAK Traceback StackTrace StackDump (most recent call last) File "/data/user/0/org.xbmc.kodi/cache/apk/assets/python3.8/lib/python3.8/threading.py", line 890, in _bootstrap self._bootstrap_inner() File "/data/user/0/org.xbmc.kodi/cache/apk/assets/python3.8/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.video.randomtrailers/resources/lib/discovery/abstract_trailer_fetcher.py", line 198, in run self.run_worker() File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.video.randomtrailers/resources/lib/discovery/abstract_trailer_fetcher.py", line 243, in run_worker self.fetch_trailer_to_play(movie) File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.video.randomtrailers/resources/lib/discovery/abstract_trailer_fetcher.py", line 385, in fetch_trailer_to_play self._fetch_trailer_to_play_worker(movie) File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.video.randomtrailers/resources/lib/discovery/abstract_trailer_fetcher.py", line 594, in _fetch_trailer_to_play_worker LibraryTrailerIndex.add(movie) # Update File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.video.randomtrailers/resources/lib/cache/library_trailer_index.py", line 67, in add movie = movie.get_as_movie_id_type() File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.video.randomtrailers/resources/lib/common/movie.py", line 1610, in get_as_movie_id_type movie_id: LibraryMovieId = LibraryMovieId(self.get_id()) File "/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/addons/script.video.randomtrailers/resources/lib/common/movie.py", line 591, in __init__ self.set_library_id(int(movie_id)) ValueError: invalid literal for int() with base 10: 'None'

Full log here: https://paste.kodi.tv/yopisorita.kodi