search

CastagnaIT / plugin.video.netflix

InputStream based Netflix plugin for Kodi
MIT License
1.89k stars 258 forks source link

Solve issue with missing art #1668

Open adamhotep opened 6 months ago

adamhotep commented 6 months ago

Check if this PR fulfills these requirements:

Types of changes

Description

When encountering a movie with missing image data, the plugin refuses to show the list of movies. This is because that's sometimes an empty dict rather than a string. This converts {} to "" to resolve the issue.

In case of Feature change / Breaking change:

Steps to reproduce the bug

Open the Netflix plugin and perform a search for "The Meg". At least at the moment, this will result in an error:

EXCEPTION: argument "value" for method "setArt" must be unicode or str

This is because there is missing data in the state['art'] dictionary for The Meg. Even if you're searching for something else, this particular entry causes the problem. If you alter the code to dump the JSON to a log, you'll see this:

{
  "poster": "",
  "fanart": "https://occ-0-2705-2706.1.nflxso.net/dnm/api/v6/…OwQw.jpg?r=b36",
  "thumb": {},
  "landscape": {},
  "clearlogo": "https://occ-0-2705-2706.1.nflxso.net/dnm/api/v6/…klWQ.png?r=3ae"
}

(I loaded the clearlogo to determine which movie's metadata was broken. Apologies for the truncated paths, I'm not sure if there's anything in there that would identify my account.)

As a result, you cannot see any search results that happen to include the broken item, even if it's quite a bit down the list.

Describe the new behavior

The new code simply replaces empty dict entries with empty str entries. You get the placeholder icon in place of the missing content (the poster in my view). Suboptimal, but I don't know how we can get better (I assume it's a Netflix bug).

Note, I didn't dig too hard to see how this dict was populated. Maybe there's a better place for the fix, but my solution certainly solves the issue (though it doesn't find replacements for the missing images).

CastagnaIT commented 6 months ago

there is nothing that identify your account in to this addon profiles are also encoded with uuid and are useless values individually because "related" to account important account values are NEVER printed to log

this PR is not correct, the wrapper must NOT change the data, the existing data adaptations in wrapper are only to support API change in Kodi python modules

im not able to find "The Meg" video maybe wrong translated name or its geolocked however i want understand better this problem and track down the raw data to see the situation

please add a new debug LOG to "response" here: https://github.com/CastagnaIT/plugin.video.netflix/blob/bb132cd4a7856e499e7a4e56394dfa28e425db65/resources/lib/services/nfsession/session/path_requests.py#L33-L34 and so open addon to load the problematic list, then attach here the raw data as txt file or the full debug log

Falke-Design commented 6 months ago

Also not available in Austria: grafik

Falke-Design commented 6 months ago

Would it not be better to fix it at this place?

https://github.com/CastagnaIT/plugin.video.netflix/blob/bb132cd4a7856e499e7a4e56394dfa28e425db65/resources/lib/kodi/infolabels.py#L263-L276

adamhotep commented 6 months ago

Yeah, I figured I was hacking in the wrong place. Thanks for the tips, but I'm not sufficiently up to speed on the codebase (and python is not my primary language). I'm not sure what to add to path_requests.py. I added a LOG.debug(…) line between the previous lines 33 & 34 and restarted the Netflix addon, but it wasn't triggered in my test. Since I was unable to get the built-in logging working, my tests yesterday involved dumping directly to a file, bypassing the debug system. I can supply the unredacted JSON dump, but I don't see what you'd gain from the full paths to the fanart and clearlogo.

Here's a dump of my log upon disabling and then reenabling the Netflix addon, then searching for "The Meg": kodi-netflix-pull-1668.log

The Meg is the only entry I've encountered with this problem, so it's unfortunate that it's not available in your region for you to verify the bug.

@Falke-Design: isn't _best_art() a better place to do this?

https://github.com/CastagnaIT/plugin.video.netflix/blob/bb132cd4a7856e499e7a4e56394dfa28e425db65/resources/lib/kodi/infolabels.py#L279-L281

Again, I'm not sure where the {} value is coming from. I wasn't able to resolve it with a quick pass at altering _best_art(), but I'm somewhat stabbing in the dark here.

As this isn't the appropriate fix, what next step should I take? Close this PR and replace it with an Issue reporting the bug? Or should we just treat this as the bug report?

CastagnaIT commented 6 months ago

but it wasn't triggered in my test

heh how do you think to debug something if you leave kodi and the addon debug settings disabled? you cant... infact your attached log there is no useful data, this is well explained on github readme.

however with the link https://www.netflix.com/title/80237937 i understood that this title isnt shown to me because i have ADS account and this title is banned from these accounts... thanks to netflix sh#t but i forced the addon to get title data: pathReq.json

how we can see on each "art" data contents, suggests that there is something that need to be fixed in parse_art and _assign_art of infolabels.py that currently they generate bad data

adamhotep commented 6 months ago

heh how do you think to debug something if you leave kodi and the addon debug settings disabled? you cant...

Yeah, I had the Netflix plugin's settings to enable debug but not Kodi's. Sorry about that, I had assumed that large base64 blob had what you wanted. Here's an updated dump with Kodi debug enabled: kodi-netflix-pull-1668-20240106_1437.log. (It complains about kodi_wrappers.py line 87, but that's really line 82 since I commented out my fix to reproduce this bug.)

Rather than searching for "The Meg", this time I navigated to my Favourites area, which has a link straight into Netflix Search, and happened to remember I was previously looking at the results of that search. The custom debug line you requested starts with Path request dump: and I can verify The Meg's metadata is indeed there, including what appears to be the appropriate poster image at size _342x684, formatted just like other titles that have their posters correctly populated. If it were just a missing entry, this would be a simpler issue. I can't see anything wrong with that data, though I don't have a trained eye to artifacts here.

I doubt I can get to a better solution from here on my own. Let me know if I can do more to help. Since you don't have access to the title, I can help test once you've got something you think will work.

this title isnt shown to me because i have ADS account and this title is banned from these accounts

At least with this title, you're not missing much 😉

Thanks for all your hard work, by the way. I am very happy with this plugin.

CastagnaIT commented 6 months ago

thanks for your attempt and the new log, but for now i block the PR because its a wrong solution will be required better investigations since having an empty dict as returned value dont sound right and so there is to understand what happens behind this i will try investigate when i will have free time for this (please dont close the PR that its a reminder to me)

Falke-Design commented 6 months ago

I intercepted the requests and then returned the responses from the log. Saddly the error is not thrown. Tested on Windows 10, Kodi 20 Nexus

grafik

grafik

grafik

fake_response.py

CastagnaIT commented 6 months ago

i also i have check whole json art data from log and all data its correct

i think that a previously parsing for some reason failed and the wrong data has been stored to the cache therefore now your addon load the wrong data from the cache not the fresh data that we are checking from log

so first you have to delete addon cache, go to addon Expert settings and delete cache from memory and disk after that open again the menu that caused the error, if the problem happens again, provide the new log at that moment without reloading the menu, so that if we are lucky contains the problematic json data in log

adamhotep commented 6 months ago

I generated a new log with these steps:

  1. Enable system-wide debug (Settings → System → Logging → Enable debug logging)
  2. Disable the Netflix addon
  3. Edit the addon's code to comment out my fix and add the aforementioned debug line
  4. Clear Netflix cache in memory and on disk (Addons → Netflix → Expert → Clear in-memory and on-disk cache
  5. Repeat the above step (there's no visual confirmation that it worked)
  6. Enable the Netflix addon
  7. Go to my Favourites shortcut for Netflix Search (it auto-visited my last request, for "Wormwood", which in my geo, includes "The Meg")
  8. See the error
  9. Disable debug logging

This reproduced the problem, though the dict for The Meg still includes a proper 80237937.boxarts._342x684 entry. I'm not 100% the cache flushed properly. See kodi-netflix-pull-1668-20240107_1529.log.

I'm on Ubuntu 20.04 LTS (amd64) and everything is fully up to date, including Kodi 6:20.2+git20230630.0528-5f418d0b13-0~focal and plugin.video.netflix 1.23.1+matrix.1. A few items are upgraded beyond Ubuntu Focal but they hopefully don't matter here. Apologies for not mentioning that earlier.

CastagnaIT commented 6 months ago

Disable the Netflix addon

why you disabled the addon? if you disable the addon, you cannot delete the cache... when the cache is successfully deleted a message is shown on screen so in the new log nothing is changed

adamhotep commented 5 months ago

Okay, good control. I figured the configuration was separate and was trying to spare myself some debug output to read through and redact (there are a lot of verbose addons that do a lot of nothing, like the Youtube addon, whose logs I've left alone).

Again!

Steps:

  1. Revert the plugin's alterations (retaining the debug line)
  2. Disable the addon
  3. Enable Kodi debug
  4. Enable the addon
  5. Clear the addon cache (in-memory and on-disk)
  6. Go to Favourites, Netflix Search
  7. Go to saved search for "The Meg" (it works fine)
  8. Go to saved search for "The Batman", possibly another title that suffers this issue (it works fine)
  9. Disable Kodi debug

So clearing the cache removed the issue and prevented us from seeing what the true cause may have been.

Log saved as kodi-netflix-pull-1668-20240109_2150.log

CastagnaIT commented 5 months ago

thanks for the patience i confirm also the current data is good

So clearing the cache removed the issue and prevented us from seeing what the true cause may have been.

in any case the cache data cannot help to identify the problem, because we lack the original raw data that we can use to understand the root of problem

you can change the debug line i have added in the code, from LOG.debug('Path request dump:... to: LOG.error('Path request dump:...

so that also if you have the kodi/addon debug settings disabled it will be always printed on the log, and next time that the error happens you will have the raw data in the log