Metadata not scraping at all with latest Hama + ASS

EarthBoundX5 commented 2 years ago

I recently upgraded Hama and ASS to the latest versions about a week ago (wanted to try out the 4 digit numbering with Detective Conan!). Immediately upon update, I found that metadata scraping was no longer working. I tried restarting Plex and manually seeking metadata to no avail. I don't have a library that I can recreate without wasting a couple days; but it looks like others have already seen, tried this, and reported this issue in the forums. Now that there are multiple reports in the forums, it's likely not user error and rather some sort of bug.

https://forums.plex.tv/t/rel-http-anidb-metadata-agent-hama/40470/2094 https://forums.plex.tv/t/rel-http-anidb-metadata-agent-hama/40470/2096

My specific issue us documented here: https://forums.plex.tv/t/rel-http-anidb-metadata-agent-hama/40470/2095

I’ll try and manually fix matches, and they just seem to hang/spin…eventually timing out after 5 minutes with no results.

Given that it's been about 3 weeks since the first report in the forums, and the subsequent similar reports, filing this with some confidence there is a bug in our midst after the last change to Hama.

My library of blank show tiles is growing as my hoarding continues, please halp! k thanks!

==================================================================================

Platform

Operating system and version: Windows Server 2019 (Version 1809, Build 17763.2452) Plex version: 1.25.6.5577

Expected Behavior

I would expect at the minimum manual metadata searching to return results. At best, pull this metadata automatically.

Current Behavior

See above description. Logs: Plex Media Server Logs_2022-03-02_01-28-01.zip HamaLogs.zip

Steps to Reproduce

Update to latest Hama, ASS, and Plex
Scan library for new items
Observe new shows populated without metadata
Manually select Match for shows
Observe Match fail to produce results

Additional information

See above.

ZeroQI commented 2 years ago

hama.log has one plex error and one source code error:

  File "E:\PlexLocalAppData\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\__init__.py", line 7
    <!DOCTYPE html>
    ^
SyntaxError: invalid syntax

Python script code will NOT include html tags <!DOCTYPE html> and this is not HAMA code as per https://github.com/ZeroQI/Hama.bundle/blob/master/Contents/Code/__init__.py line 7 which is "### Imports ###"

Please update propertly the code so it doesn't have html tags and report, and indicate if you did manually update Hama.bundle/tree/master/Contents/Code)/init.py with the github html page of the source code?

EarthBoundX5 commented 2 years ago

hama.log has one plex error and one source code error:
  File "E:\PlexLocalAppData\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\__init__.py", line 7
    <!DOCTYPE html>
    ^
SyntaxError: invalid syntax
Python script code will NOT include html tags <!DOCTYPE html> and this is not HAMA code as per https://github.com/ZeroQI/Hama.bundle/blob/master/Contents/Code/__init__.py line 7 which is "### Imports ###"

Please update propertly the code so it doesn't have html tags and report, and indicate if you did manually update Hama.bundle/tree/master/Contents/Code)/init.py with the github html page of the source code?

This is most definitely the issue.

I had followed the updating steps here: https://github.com/ZeroQI/Hama.bundle#updating

Given that the link for init.py was not a hyperlink, I simply selected to save as and move on; not checking the content within init.py. Subsequently seeing 2x other reports in the support thread with the same issue as me provide confirmation bias that there was an issue and to wait for further support instructions.

I should be familiar enough with git to have known better, but I'm betting this isn't the first time. Maybe a slight adjustment I'd suggest would be to hyperlink the path to each file or better yet hyperlink to the raw file like in the ASS updating documentation? I've learned that if documentation is clear to you, it's no good, because some idiot can come along and screw it all up...in this case, I (and I assume the same for the others with the same issue) are the idiot.

ZeroQI commented 2 years ago

Not the first time indeed. My issue is if I put too much in the readme people won't read it so it gets tricky... I do approve any pr on the readme though...

EarthBoundX5 commented 2 years ago

Scratch that, after correcting my mistake, I'm able to search for metadata...but metadata still doesn't populate...I'll fool around and see if there is something I can figure out.

EarthBoundX5 commented 2 years ago

Not the first time indeed. My issue is if I put too much in the readme people won't read it so it gets tricky... I do approve any pr on the readme though...

It's just hard, never had to use version control for more than just consumption. Huge mistake on my part, but I'm stuck in the old days of self managing versions due to how infrequent I make changes to what little scripting or coding I do and lack of coworkers in my position to reach consensus with changes. Didn't formally ever have to make my first PR until about a year and a half ago using Azure DevOps and that was for a simple basic update to reference a new Visual C++ redist for our instance of Chef. No excuse there, it's just a process I've yet to need to learn...maybe a good a excuse to start simple with contributing to a Readme, but I really worry about not following the right process and screwing something up.

ZeroQI commented 2 years ago

To install Hama, it's like other Plex agents: Get the latest source zip in https://github.com/ZeroQI/Hama.bundle > "Code" > "Download Zip". Inside the Hama.bundle-master.zip you will have the folder "Hama.bundle-master" to rename to "Hama.bundle" and place in Plex plugin folder [https://support.plex.tv/hc/en-us/articles/201106098-How-do-I-find-the-Plug-Ins-folder-] or Plex main folder / "Plug-ins"

I really can't think of a better way to explain the install process... The update was from before there was one file per metadata source, need to remove it from the readme

I would need the per series custom title.update.log file to troubleshoot...

EarthBoundX5 commented 2 years ago

Alright, just added and imported a new series into Plex and while it showed the ability to search for metadata, none was scraped.

Here are the series logs for the one Sabikui Bisco pictured above: Sabikui Bisco (2022) [anidb-16103].agent-update.log Sabikui Bisco (2022) [anidb-16103].agent-search.log

And here are the logs for Astro Boy (2003) noted a the previous post. Astro Boy (2003) (2003) [anidb-807].agent-update.log Astro Boy (2003) (2003) [anidb-807].agent-search.log

Astro Boy was imported prior to fixing the bad update files, and Sabikui Bisco was just imported now after having fixed said files.

EDIT: It looks like the metadata is shown in the update logs...but just not making it to Plex?

EDIT 2: After I replaced the bad files for Hama, I should note that I did a full restart of the server, not just bounced the service.

EDIT 3: Episode titles are showing in Plex, just no other metadata. Same for Astro Boy (2003). Confirmed the same with a couple other series. Metadata search is fine, pulls in Episode titles, but nothing else from what I see in the UI.

EDIT 4: I made a new test library and produced the same results. Test Library - Anime TV.zip

EarthBoundX5 commented 2 years ago

To install Hama, it's like other Plex agents: Get the latest source zip in https://github.com/ZeroQI/Hama.bundle > "Code" > "Download Zip". Inside the Hama.bundle-master.zip you will have the folder "Hama.bundle-master" to rename to "Hama.bundle" and place in Plex plugin folder [https://support.plex.tv/hc/en-us/articles/201106098-How-do-I-find-the-Plug-Ins-folder-] or Plex main folder / "Plug-ins"

I really can't think of a better way to explain the install process... The update was from before there was one file per metadata source, need to remove it from the readme

I would need the per series custom title.update.log file to troubleshoot...

Just noticed the edit, ya, the install process is perfectly fine. No issues there! It's just the update process and me making assumptions of similarity to the ASS update process.

Note, I made a few edits of my own to my previous post (noted with "EDIT").

ZeroQI commented 2 years ago

Last log is in anidb part after ratings... https://github.com/ZeroQI/Hama.bundle/blob/53ac8759694eb85f93d6fd983722bbef7da058ad/Contents/Code/AniDB.py#L193

Please check standard hama log to see if there is an hama error in there, otherwise code needs altering for debugging by outputting text to locate issue...

EarthBoundX5 commented 2 years ago

Last log is in anidb part after ratings...

https://github.com/ZeroQI/Hama.bundle/blob/53ac8759694eb85f93d6fd983722bbef7da058ad/Contents/Code/AniDB.py#L193

Please check standard hama log to see if there is an hama error in there, otherwise code needs altering for debugging by outputting text to locate issue...

There is a Critical error in the Hama logs.

2022-03-12 21:58:30,832 (13d8) : CRITICAL (core:574) - Exception in the update function of agent named 'HamaTV', called with guid 'com.plexapp.agents.hama://anidb-16103?lang=en' (most recent call last): File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-c8bd13540\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\api\agentkit.py", line 1093, in _update agent.update(obj, media, lang, **kwargs) File "E:\PlexLocalAppData\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\__init__.py", line 166, in update def update (self, metadata, media, lang, force ): Update (metadata, media, lang, force, False) File "E:\PlexLocalAppData\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\__init__.py", line 141, in Update dict_AniDB, ANNid, MALid = AniDB.GetMetadata(media, movie, error_log, source, AniDBid, TVDBid, AnimeLists.AniDBMovieSets, mappingList) File "E:\PlexLocalAppData\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\AniDB.py", line 315, in GetMetadata elif epNumType in ['1', '2']: SaveDict([episode], missing, season) File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-c8bd13540\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\code\sandbox.py", line 108, in <lambda> _getitem_ = lambda x, y: x.__getitem__(y), File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-c8bd13540\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\api\runtimekit.py", line 432, in __getitem__ return self._sandbox.preferences.get()[name] File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-c8bd13540\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\code\preferences.py", line 332, in __getitem__ raise KeyError("No preference named '%s' found." % name) KeyError: "No preference named 'AnidbGenresMainOnly' found."

PMS Plugin Logs.zip

EDIT: I should have done this sooner, but wanted to wait for direction from you before messing with my setup; I went ahead and replaced the AniDB.py file mentioned in the error above with the latest from github and metadata is not scraping and I'm not seeing the error in the main Hama logs.

So, either what I did with the malformed html file somehow broke the update process or going from the version I had to the current has a missing piece to the update process? Please let me know what I can do to help if this is indeed something worth investigating (I'd think it is?).

For reference, I would have likely been upgrading from the version of Hama from 7/25/2021...or 2/15/2020; as those are the copies I've retained and would have been from one of my last rebuilds of my Plex server.

EDIT 2: Just to add unrelated, the 4 digit episode count bug I reported around Detective Conan a few weeks ago is fixed and working after having just scraped with this release. Thanks!

ZeroQI commented 2 years ago

File "E:\PlexLocalAppData\Plex Media Server\Plug-ins\Hama.bundle\Contents\Code\AniDB.py", line 315, in GetMetadata elif epNumType in ['1', '2']: SaveDict([episode], missing, season)

https://github.com/ZeroQI/Hama.bundle/blob/master/Contents/DefaultPrefs.json does not have a variable named "AnidbGenresMainOnly"

https://github.com/ZeroQI/Hama.bundle/blob/master/Contents/Code/AniDB.py#L315 does not have "AnidbGenresMainOnly"

I searched ALL files from latest code in in notepad++ for this string, it doesn't exist You do not have the latest code and the DefaultPrefs.json file does not match the python code, so you have old files from different versions, closing

ZeroQI / Hama.bundle