ZeroQI / Hama.bundle

Plex HTTP Anidb Metadata Agent (HAMA)
GNU General Public License v3.0
1.21k stars 113 forks source link

HAMA is no longer working! #547

Closed JShiesty-dev closed 1 year ago

JShiesty-dev commented 1 year ago

HAMA no longer fetches metadata. HAMA needs to be updated. Tried everything under the sun including a total reinstall of Plex, ASS, and HAMA. Multiple others in the past week have said they are having the same issue. Likely a new Plex update that broke HAMA?

ZeroQI commented 1 year ago

If several are impacted without any update to Hama, then that is a very likely scenario

Logs missing despite full path listed in troubleshooting section...

JShiesty-dev commented 1 year ago

com.plexapp.system.log com.plexapp.agents.hama.log

ZeroQI commented 1 year ago

HAMA log show clear error with Anidb title xml: unknown ssl error

Place manually anime-list-master.xml in agent data folder / AniDB

Le sam. 8 avr. 2023 à 02:01, JShiesty @.***> a écrit :

com.plexapp.system.log https://github.com/ZeroQI/Hama.bundle/files/11182139/com.plexapp.system.log com.plexapp.agents.hama.log https://github.com/ZeroQI/Hama.bundle/files/11182140/com.plexapp.agents.hama.log

— Reply to this email directly, view it on GitHub https://github.com/ZeroQI/Hama.bundle/issues/547#issuecomment-1500730741, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHMWZKSNGUOAJZFD366XK3XACTEFANCNFSM6AAAAAAWW5E37I . You are receiving this because you commented.Message ID: <ZeroQI/Hama. @.***>

JShiesty-dev commented 1 year ago

Also I am unsure why it would throw an error since I can navigate to "https://raw.githubusercontent.com/Anime-Lists/anime-lists/master/anime-list-master.xml" perfectly fine. Is this an issue on HAMA's end?

ZeroQI commented 1 year ago

Hama can't could be Hama, plex, firewall, antyspyware, http cache from plex, ... Save that file manually in agent data folder / AniDB

JShiesty-dev commented 1 year ago

Hama can't could be Hama, plex, firewall, antyspyware, http cache from plex, ... Save that file manually in agent data folder / AniDB

Where exactly is the agent data folder / AniDB?

ZeroQI commented 1 year ago

From the Readme: Hama auto-creates some folders in the agent data folder ("Plug-In Support/Data/com.plexapp.agents.hama/DataItems")

JShiesty-dev commented 1 year ago

I see only a logs folder in that directory. Moving anime-list-master.xml into "Plug-In Support/Data/com.plexapp.agents.hama/DataItems/AniDB" and restarting PMS still results in no metadata population.

ZeroQI commented 1 year ago

2023-04-07 16:35:07,227 (1d80) : ERROR (sandbox:19) - ERROR: common.LoadFile() - issue loading url: 'https://raw.githubusercontent.com/Anime-Lists/anime-lists/master/anime-list-master.xml', filename: 'anime-list-master.xml', Headers: {'Content-type': 'application/json', 'User-agent': 'Plex/HAMA'}, Exception: 'unknown error (_ssl.c:2938)'

It is missing both the anidb and anime-list xmls

If you can open the url and plex/Hama can't, there is an issue there... Probably rights issue in agent data folder or antivirus/firewall

Le sam. 8 avr. 2023 à 02:54, JShiesty @.***> a écrit :

I see only a logs folder in that directory. Moving anime-list-master.xml into "Plug-In Support/Data/com.plexapp.agents.hama/DataItems/AniDB" and restarting PMS still results in no metadata population.

— Reply to this email directly, view it on GitHub https://github.com/ZeroQI/Hama.bundle/issues/547#issuecomment-1500747166, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHMWZOZHTOC5KFZ5GPW3ITXACZNVANCNFSM6AAAAAAWW5E37I . You are receiving this because you commented.Message ID: <ZeroQI/Hama. @.***>

JShiesty-dev commented 1 year ago

2023-04-07 16:35:07,227 (1d80) : ERROR (sandbox:19) - ERROR: common.LoadFile() - issue loading url: 'https://raw.githubusercontent.com/Anime-Lists/anime-lists/master/anime-list-master.xml', filename: 'anime-list-master.xml', Headers: {'Content-type': 'application/json', 'User-agent': 'Plex/HAMA'}, Exception: 'unknown error (_ssl.c:2938)' It is missing both the anidb and anime-list xmls If you can open the url and plex/Hama can't, there is an issue there... Probably rights issue in agent data folder or antivirus/firewall Le sam. 8 avr. 2023 à 02:54, JShiesty @.> a écrit : I see only a logs folder in that directory. Moving anime-list-master.xml into "Plug-In Support/Data/com.plexapp.agents.hama/DataItems/AniDB" and restarting PMS still results in no metadata population. — Reply to this email directly, view it on GitHub <#547 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHMWZOZHTOC5KFZ5GPW3ITXACZNVANCNFSM6AAAAAAWW5E37I . You are receiving this because you commented.Message ID: <ZeroQI/Hama. @.>

Not sure how to go about fixing this. Could you tell me precisely what files need to go where? I am not familiar with where the agent data folder is. The path ""Plug-In Support/Data/com.plexapp.agents.hama/DataItems" only contains the _Logs folder.

JShiesty-dev commented 1 year ago

I highly doubt it would be firewall related as I have been using HAMA for a year with no problems. I would say it is likely something with a recent Plex update. If you could try recreating this issue that would be helpful. If you cannot recreate this issue then that at least proves its something on my end.

JShiesty-dev commented 1 year ago

Might be also worth noting that I found this post on Reddit about another one of your agents: https://www.reddit.com/r/PleX/comments/12c5se2/did_anyones_zeroqis_youtube_plex_agent_stop/ This may be an indication that this is indeed some sort of a change in a recent Plex update that broke something.

seguraf16 commented 1 year ago

Would like to chime in been having some issues with the scanner lately running the latest plex pass beta with it not grabbing metadata correctly on brand new shows being imported, it's not consistent but just now it refused to even discover a show unless I switched it manualy with movie database, got some debug logs but not sure where else to poke at to help

Apr 09, 2023 23:32:46.043 [1744] DEBUG - [HttpClient/HCl#5f0b] HTTP/1.1 (0.0s) 200 response from GET http://127.0.0.1:50282/:/plugins/com.plexapp.agents.hama/messaging/function/"" Apr 09, 2023 23:32:46.043 [6396] DEBUG - [Req#bbaac] [com.plexapp.agents.hama] HTTP reply status 200, with 320 bytes of content. Apr 09, 2023 23:32:46.043 [5844] DEBUG - Completed: [127.0.0.1:50703] 200 GET /:/plugins/com.plexapp.agents.hama/messaging/function/"" (13 live) #bbaac GZIP 19ms 663 bytes Apr 09, 2023 23:32:46.044 [1744] DEBUG - [HttpClient/HCl#5f0a] HTTP/1.1 (0.0s) 200 response from GET http://127.0.0.1:59871/system/agents/search?mediaType=2&id=247117&identifier=com.plexapp.agents.hama&lang=en&show=Demon%20Slayer%3A%20Kimetsu%20no%20Yaiba&year=2019&manual=1 (reused) Apr 09, 2023 23:32:46.045 [3976] DEBUG - [Req#bbaaa] [com.plexapp.system] HTTP reply status 200, with 130 bytes of content. Apr 09, 2023 23:32:46.045 [4424] DEBUG - Completed: [127.0.0.1:50671] 200 GET /system/agents/search?mediaType=2&id=247117&identifier=com.plexapp.agents.hama&lang=en&show=Demon%20Slayer%3A%20Kimetsu%20no%20Yaiba&year=2019&manual=1 (12 live) #bbaaa GZIP 25ms 437 bytes (pipelined: 6) Apr 09, 2023 23:32:46.045 [1744] DEBUG - [HttpClient/HCl#5f09] HTTP/1.1 (0.0s) 200 response from GET http://127.0.0.1:32400/system/agents/search?mediaType=2&id=247117&identifier=com.plexapp.agents.hama&lang=en&show=Demon%20Slayer%3A%20Kimetsu%20no%20Yaiba&year=2019&manual=1 (reused) Apr 09, 2023 23:32:46.045 [7588] WARN - [Req#bbaab] No matches for search.

krisvega commented 1 year ago

I also had this issue after updating PMS to version 1.32.0.6918. Logs were showing ssl error and metadata wasn't being fetched properly. After downgrading PMS back to version 1.31.3.6868, the agent is working normally. Not sure if this is a bug on the Plex side and will be fixed by another update, or if it is an intended change that Hama will have to support going forward. Not updating PMS for now.

mkoreen commented 1 year ago

I would also like to chime in. I am running plex server (newest plex pass version) on a windows server 2022 machine. The log files for hama are showing variations of this: ERROR: common.LoadFile() - issue loading url: 'https://api.thetvdb.com/series/386572?en', filename: 'series_en.json', Headers: {'Content-type': 'application/json', 'Accept-Language': 'en', 'User-agent': 'Plex/HAMA'}, Exception: 'unknown error (_ssl.c:2938)'

This is the only real error I'm seeing. I was wondering if it had to do with certificate fails or some such, but I've tried some hacks to turn off verification and that didn't help. I've also noticed that if you load them subsequent times, you get locally cached files that load without issues, maybe? Plex is using a super old python 2.7, and I don't know any way to get a look at the actual _ssl.c error line it's pointing out. Would be nice if we had the source code.

Second time I refreshed after changing the filename slightly just so it would refresh the metadata: common.LoadFile() - File cached locally - Filename: 'TheTVDB\json\386572\series_en.json', Age: '14.76 days', Limit: '6 days', url: 'https://api.thetvdb.com/series/386572?en' ERROR: common.LoadFile() - issue loading url: 'https://api.thetvdb.com/series/386572?en', filename: 'series_en.json', Headers: {'Content-type': 'application/json', 'Accept-Language': 'en', 'User-agent': 'Plex/HAMA'}, Exception: 'unknown error (_ssl.c:2938)'

ZeroQI commented 1 year ago

Or TheTVDB started blocking the old API... Is it just TheTVDB showing the error?

mkoreen commented 1 year ago

Yes, it's always TheTVDB. The anidb ones are working. If I load the URL in a browser it works too, at least I get json data. But maybe it doesn't like something about the user agent or has specifically blocked requests from hama's agent.

mkoreen commented 1 year ago

Hrm, doesn't look like they are blocking the agent/headers. I sent the same request with the same headers via postman and that appears to work. Image here

mkoreen commented 1 year ago

Sorry for multiple messages, but reading that reddit post people posted above, it appears that an older version of plex still works, 1.31.3.6868. If true, I wonder if plex updated or changed their python version somewhat. The most recent version is using python 2.7.17. I wonder if the earlier plex version is using a different version. I may have to roll back and check, and/or diff the python environment files and see if changes happened. Either way this is pretty shitty since I use hama for all my anime and I'd like to make it work again. Good luck figuring it out ZeroQI.

TChilderhose commented 1 year ago

One of the undocumented changes Plex made with 1.32.X is they dropped OpenSSL 1.1.1 support, so now it's OpenSSL 3+ only. It really sucks because Plex uses Python 2.7 which doesn't have OpenSSL 3. I've asked about any plans to include python3 but worried that this might be the end of plugins.

TChilderhose commented 1 year ago

They have no plans on Python 3. You might be able to get around the cert issues, but having it use an OpenSSL library that the Python version doesnt support is going to cause a lot of other issues with https. Other than sticking with PMS 1.31.X I dont really have a great solution for the plugin unfortunately

mkoreen commented 1 year ago

wow so they basically broke all plugins that use https. I wonder if there's a way to patch openssl 3 support into python 2.7.

mkoreen commented 1 year ago

You know what I'd consider doing.. bundle a compiled curl.exe in the mod and have that do your https calls via system calls.

fawzib commented 1 year ago

yeah i'm getting ERROR: TVDB Loading search XML failed, Exception: 'unknown error (_ssl.c:2938)' maybe we need to ignore the cert or something. for XML.ElementFromURL method. how can we ignore ssl validation or maybe replace it altogether and use some method that allows us to bypass cert validations

TChilderhose commented 1 year ago

Doesn't look as dire as I thought and seems to likely a Windows only issue.

Plex said they are going to take a look at it. From what I can tell I dont think its necessarily an issue on their end. They might be able to tweak their foundation library a bit to loosen some cert verification, but yeah hard to say.

I'm hoping to have some time this weekend to try a few things

TChilderhose commented 1 year ago

Ended up having some time today and worked on a fix. I posted it in the thread so hopefully they can just use it or at least give them a good starting point.

Dingmatt commented 1 year ago

The issue is caused by the Proxy.Preview (image thumbnail proxy) function. You can bypass it by avoiding thumbnail handling for the time being and using the Proxy.Media for full-resolution images instead

TChilderhose commented 1 year ago

Plex Media Server 1.32.1.6999 fixes the issue

Xeno426 commented 1 year ago

Can confirm, just updated to the latest beta and I'm getting search results again.