Closed rg9400 closed 6 months ago
Seems like they're storing some new data in the extra data that breaks the decoding process.
I'll be able to take a closer look at it once I'm back in my home country at January 10th.
Same Problem here. Thanks for your work mynttt :). Grüße aus Deutschland.
thanks just noticed it myself..
Same here.
@mrjohnpoz Could you assist me by sending me a sample database of where this problem occurs to my mail? (marc@herschel.io)
Yeah have sent it you before - just want a copy of the database. I never looked to deep into where it was happening.. But noticed my docker was sucking cpu, and was like why it still running, it was crashing and then restarting, and then repeat..
I will drop you a copy here in a bit.
ok db sent.. Here is a crash
[INFO ] - 2024-01-11 03:04:53 @ TvdbApiV4.auth: Test passed. API Key is valid. [INFO ] - 2024-01-11 03:04:53 @ ImdbDockerImplementation$ImdbBatchJob.run: [MOVIE] Movies has 4066 item(s) [INFO ] - 2024-01-11 03:04:53 @ ImdbDockerImplementation$ImdbBatchJob.run: [SERIES] TV Shows has 25941 item(s) [INFO ] - 2024-01-11 03:04:53 @ ImdbDockerImplementation$ImdbBatchJob.run: Processing [MOVIE] Movies with UUID c47513ed-aae2-4a5d-91b9-a87b23d64061 at stage: CREATED [INFO ] - 2024-01-11 03:04:53 @ ImdbPipeline.analyseDatabase: Resolving IMDB identifiers for items. Only warnings and errors will show up... [INFO ] - 2024-01-11 03:04:53 @ ImdbPipeline.analyseDatabase: Items that show up here will not be processed by further stages of the pipeline. [INFO ] - 2024-01-11 03:04:53 @ ImdbPipeline.analyseDatabase: Starting watchdog to print progress to std::out with a delay and interval of 1 minute... [INFO ] - 2024-01-11 03:04:53 @ ImdbPipeline.analyseDatabase: Progress printing watchdog has been stopped. Cancelation status: true [INFO ] - 2024-01-11 03:04:53 @ ImdbPipeline.analyseDatabase: Save point: Persisting caches to keep queried look-up data in case of crashes or hang-ups. [INFO ] - 2024-01-11 03:04:54 @ ImdbPipeline.analyseDatabase: Filtered 2 invalid item(s). [ERROR] - 2024-01-11 03:04:54 @ ExtraData.of: Tried to decode [KEY] := '{"at:audienceRatingImage":"imdb://image.rating","ex:hasPremiumPrimaryExtra":"1","ex:primaryExtraKey":"/library/metadata/79343","pv:artBlurHash":"LQLy+US58^s8?HofV?o19FRk%gX8","pv:blurHashesChangedAt":"140717343446400","pv:cloudAssetsRefreshedAt":"1606011225","pv:thumbBlurHash":"TgO3qbxD~q~WkCIV_NS4IUIUjG%2","url":"at%3AaudienceRatingImage' [ERROR] - 2024-01-11 03:04:54 @ ExtraData.of: Tried to decode [VALUE] := 'imdb%3A%2F%2Fimage%2Erating' [INFO ] - 2024-01-11 03:04:54 @ ImdbDockerImplementation$ImdbBatchJob.run: Job returned ERROR : URLDecoder: Illegal hex characters in escape (%) pattern - Error at index 0 in: "gX" [ERROR] - 2024-01-11 03:04:54 @ ImdbDockerImplementation$ImdbBatchJob.run: IllegalArgumentException exception encountered... [ERROR] - 2024-01-11 03:04:54 @ ImdbDockerImplementation$ImdbBatchJob.run: Please contact the maintainer of the application with the stacktrace below if you think this is unwanted behavior. [ERROR] - 2024-01-11 03:04:54 @ ImdbDockerImplementation$ImdbBatchJob.run: ======================================== [ERROR] - 2024-01-11 03:04:54 @ ImdbDockerImplementation$ImdbBatchJob.run: java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in escape (%) pattern - Error at index 0 in: "gX" at java.base/java.net.URLDecoder.decode(Unknown Source) at updatetool.common.ExtraData.of(ExtraData.java:101) at updatetool.imdb.ImdbTransformer.needsUpdate(ImdbTransformer.java:39) at java.base/java.util.function.Predicate.lambda$negate$1(Unknown Source) at java.base/java.util.stream.ReferencePipeline$2$1.accept(Unknown Source) at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(Unknown Source) at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source) at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source) at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source) at updatetool.imdb.ImdbPipeline.transformMetadata(ImdbPipeline.java:190) at updatetool.imdb.ImdbPipeline.transformMetadata(ImdbPipeline.java:45) at updatetool.api.Pipeline.invoke(Pipeline.java:12) at updatetool.imdb.ImdbJobRunner.run(ImdbJobRunner.java:19) at updatetool.imdb.ImdbDockerImplementation$ImdbBatchJob.run(ImdbDockerImplementation.java:285) at updatetool.TaskWrapper.run(Main.java:252) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [ERROR] - 2024-01-11 03:04:54 @ ImdbDockerImplementation$ImdbBatchJob.run: ======================================== [ERROR] - 2024-01-11 03:04:54 @ ImdbDockerImplementation$ImdbBatchJob.run: The application will terminate now. budman@NAS:/volume3/docker/imdpupdaterconfig$
mine also does not work
[WARN ] - 2024-01-13 19:10:14 @ DefaultResolvement.resolve: Item: 'Ani Kuri15' has no matching IMDB resolver and will be ignored. (guid=local://5) [INFO ] - 2024-01-13 19:10:14 @ ImdbPipeline.analyseDatabase: Progress printing watchdog has been stopped. Cancelation status: true [INFO ] - 2024-01-13 19:10:14 @ ImdbPipeline.analyseDatabase: Save point: Persisting caches to keep queried look-up data in case of crashes or hang-ups. [INFO ] - 2024-01-13 19:10:14 @ ImdbPipeline.analyseDatabase: Filtered 1 invalid item(s). [INFO ] - 2024-01-13 19:10:14 @ ImdbDockerImplementation$ImdbBatchJob.run: Job returned ERROR : URLDecoder: Illegal hex characters in escape (%) pattern - Error at index 1 in: "4." [ERROR] - 2024-01-13 19:10:14 @ ExtraData.of: Tried to decode [KEY] := '{"at:audienceRatingImage":"imdb://image.rating","pv:artBlurHash":"LIC?1I^%4.R*~U?HaKInWVa{xat7","pv:blurHashesChangedAt":"90348","pv:thumbBlurHash":"TaJR2F0KXn_NITxspInlRQSiRjs.","url":"at%3AaudienceRatingImage' [ERROR] - 2024-01-13 19:10:14 @ ExtraData.of: Tried to decode [VALUE] := 'imdb%3A%2F%2Fimage%2Erating' [ERROR] - 2024-01-13 19:10:14 @ ImdbDockerImplementation$ImdbBatchJob.run: IllegalArgumentException exception encountered... [ERROR] - 2024-01-13 19:10:14 @ ImdbDockerImplementation$ImdbBatchJob.run: Please contact the maintainer of the application with the stacktrace below if you think this is unwanted behavior. [ERROR] - 2024-01-13 19:10:14 @ ImdbDockerImplementation$ImdbBatchJob.run: ======================================== [ERROR] - 2024-01-13 19:10:14 @ ImdbDockerImplementation$ImdbBatchJob.run: java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in escape (%) pattern - Error at index 1 in: "4." at java.base/java.net.URLDecoder.decode(Unknown Source) at updatetool.common.ExtraData.of(ExtraData.java:101) at updatetool.imdb.ImdbTransformer.needsUpdate(ImdbTransformer.java:39) at java.base/java.util.function.Predicate.lambda$negate$1(Unknown Source) at java.base/java.util.stream.ReferencePipeline$2$1.accept(Unknown Source) at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(Unknown Source) at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source) at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source) at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source) at updatetool.imdb.ImdbPipeline.transformMetadata(ImdbPipeline.java:190) at updatetool.imdb.ImdbPipeline.transformMetadata(ImdbPipeline.java:45) at updatetool.api.Pipeline.invoke(Pipeline.java:12) at updatetool.imdb.ImdbJobRunner.run(ImdbJobRunner.java:19) at updatetool.imdb.ImdbDockerImplementation$ImdbBatchJob.run(ImdbDockerImplementation.java:285) at updatetool.TaskWrapper.run(Main.java:252) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) [ERROR] - 2024-01-13 19:10:14 @ ImdbDockerImplementation$ImdbBatchJob.run: ======================================== [ERROR] - 2024-01-13 19:10:14 @ ImdbDockerImplementation$ImdbBatchJob.run: The application will terminate now. **** Binary has crashed. Restart in 10 seconds... ****
Hi, Just want to say i like the tool a lot. Great work here. Do you have any update about the fix for that problem? @mynttt
Adding my name to the list. Been using GUI on windows and it doesn’t even open anymore. Hopefully the fix for new database isn’t far off
Thanks for the greate tool, hoping for a fix soon :)
Would appreciate a a fix as well, can provide debugging logs or similar if needed
@mynttt any update on this? Any more details needed from our end to help you fix the issue?
My take is he is busy with other stuff - maybe real life stuff? Or this is more of an issue than he first thought. I did hear back from him that he got my db, etc.
This is a bit odd though, in the past stuff has been fixed right away.. I am starting to notice and missing this great tool keeping my ratings updated and current..
I'm really busy with real life stuff right now, so until now I've not found time to look at it. I assume it's just a small fix but I'm currently unable to invest any time into it - I hope that next month onwards when it all settles I'll finde some time for it again
No worries, take your time. It's a great tool.
Real life issues should always take precedence over hobbies, etc, and I want to express my deepest gratitude to the continued maintenance of this tool. Updatetool has been a valuable tool for my Plex installation. I do hope that whatever issues mynttt is faced with, come to a happy ending.
That being said, anyone else looking for a similar tool in the meantime can check out https://github.com/meisnate12/Plex-Meta-Manager
It can mass update ratings (user, audience, critic), but the configuration can be overwhelming, as it's designed to do much more.
I am certainly NOT trying to steer people away from updatetool. And I hope it does not come across this way, but wanted to offer a different solution until this issue gets resolved. Apologies if it is frowned upon doing so.
I fully agree with @BrodyBuster. Thanks for the valuable tool, but personal life is always more important then hobbies that bring mainly value to others 😄
To help ppl a bit in the right direction for PMM:
libraries:
Movies:
overlay_files:
- pmm: ratings
template_variables:
rating1: critic
rating2: audience
rating3: user
rating1_image: imdb
rating2_image: rt_popcorn
rating3_image: tmdb
operations:
mass_critic_rating_update: imdb
mass_audience_rating_update: mdb_tomatoesaudience
mass_user_rating_update: tmdb
TV Shows:
overlay_files:
- pmm: ratings
template_variables:
rating1: critic
rating2: audience
rating3: user
rating1_image: imdb
rating2_image: rt_popcorn
rating3_image: tmdb
- pmm: ratings
template_variables:
builder_level: episode
rating1: critic
rating2: audience
rating1_image: imdb
rating2_image: tmdb
operations:
mass_critic_rating_update: imdb
mass_audience_rating_update: mdb_tomatoesaudience
mass_user_rating_update: tmdb
mass_episode_critic_rating_update: imdb
mass_episode_audience_rating_update: tmdb
playlist_files:
- pmm: playlist
template_variables:
libraries: Movies, TV Shows
settings:
run_order:
- operations
- metadata
- collections
- overlays
cache: true
cache_expiration: 60
asset_directory: config/assets
asset_folders: true
asset_depth: 0
create_asset_folders: false
prioritize_assets: false
dimensional_asset_rename: false
download_url_assets: false
show_missing_season_assets: false
show_missing_episode_assets: false
show_asset_not_needed: true
sync_mode: append
minimum_items: 1
default_collection_order:
delete_below_minimum: true
delete_not_scheduled: false
run_again_delay: 2
missing_only_released: false
only_filter_missing: false
show_unmanaged: true
show_unconfigured: true
show_filtered: false
show_options: true
show_missing: true
show_missing_assets: true
save_report: false
tvdb_language: eng
ignore_ids:
ignore_imdb_ids:
item_refresh_delay: 0
playlist_sync_to_user: all
playlist_exclude_users:
playlist_report: false
verify_ssl: true
custom_repo:
check_nightly: false
plex: # REQUIRED for the script to run
url: http://<Plex server>:32400
token: # <In Plex Web, open a movie and open info as an xml presentation. At the end of the URL there is the Plex token>
timeout: 60
db_cache:
clean_bundles: true
empty_trash: true
optimize: true
tmdb: # REQUIRED for the script to run
apikey: # <Create an account ad tmdb and generate an API token>
language: en
cache_expiration: 60
region:
ignore_cache: false
Okay I have some update on the matter, the data format for extra_data has simply changed that's why the parser is not working anymore. I'm working on a fix right now that I can hopefully publish in an hour or so. In order to keep compatibility with the old format the class responsible for parsing will detect the new format and then behave differently when parsing and exporting it.
That's great news. Thank you for the update.
So there has been couple of updates since 7775, we are now on 7997, with how lazy the release notes are, its hard to tell if there has been any actual changes to the db at all, but there was also a 7996 update day before 7997..
New docker container is out! Please let me know how it works out. 32-bit container is not supported anymore because the java base image is deprecated and it's package sources don't work anymore. To see that it works look at the data below taken from the test database!
Test Data:
Operation: Add Imdb Badge (No Badge)
title = "Hank Chen: I'm Not Supposed to Be Here"
OLD: {"pv:artBlurHash":"LZKtvD{}iJWC[WOCOrR*-CI:Sgoe","pv:blurHashesChangedAt":"140717350514102","pv:thumbBlurHash":"TnKRb8+^Rj}aaeV[v#SgSM-pS2Sg","url":"pv%3AartBlurHash=LZKtvD%7B%7DiJWC%5BWOCOrR%2A-CI%3ASgoe&pv%3AblurHashesChangedAt=140717350514102&pv%3AthumbBlurHash=TnKRb8%2B%5ERj%7DaaeV%5Bv%23SgSM-pS2Sg"}
NEW: {"at:audienceRatingImage":"imdb://image.rating","pv:artBlurHash":"LZKtvD{}iJWC[WOCOrR*-CI:Sgoe","pv:blurHashesChangedAt":"140717350514102","pv:thumbBlurHash":"TnKRb8+^Rj}aaeV[v#SgSM-pS2Sg","url":"pv%3AartBlurHash=LZKtvD%7B%7DiJWC%5BWOCOrR%2A-CI%3ASgoe&pv%3AblurHashesChangedAt=140717350514102&pv%3AthumbBlurHash=TnKRb8%2B%5ERj%7DaaeV%5Bv%23SgSM-pS2Sg"}
Operation: Remove Junk Badge (No Imdb), Add Imdb Badge
title = "Justice League: Crisis on Infinite Earths - Part One"
OLD: {"at:audienceRatingImage":"themoviedb://image.rating","pv:artBlurHash":"LPBL-qR.0#fhA0og-PR*xWWVRixG","pv:blurHashesChangedAt":"140717350597760","pv:cloudAssetRefreshedAt":"1704757404","pv:thumbBlurHash":"TPE.0Cs;0g23j?-3-.R*M{EMsoxt","url":"at%3AaudienceRatingImage=themoviedb%3A%2F%2Fimage%2Erating&pv%3AartBlurHash=LPBL-qR%2E0%23fhA0og-PR%2AxWWVRixG&pv%3AblurHashesChangedAt=140717350597760&pv%3AcloudAssetRefreshedAt=1704757404&pv%3AthumbBlurHash=TPE%2E0Cs%3B0g23j%3F-3-%2ER%2AM%7BEMsoxt"}
NEW: {"at:audienceRatingImage":"imdb://image.rating","pv:artBlurHash":"LPBL-qR.0#fhA0og-PR*xWWVRixG","pv:blurHashesChangedAt":"140717350597760","pv:cloudAssetRefreshedAt":"1704757404","pv:thumbBlurHash":"TPE.0Cs;0g23j?-3-.R*M{EMsoxt","url":"at%3AaudienceRatingImage=themoviedb%3A%2F%2Fimage%2Erating&pv%3AartBlurHash=LPBL-qR%2E0%23fhA0og-PR%2AxWWVRixG&pv%3AblurHashesChangedAt=140717350597760&pv%3AcloudAssetRefreshedAt=1704757404&pv%3AthumbBlurHash=TPE%2E0Cs%3B0g23j%3F-3-%2ER%2AM%7BEMsoxt"}
@mrjohnpoz I'd assume that it probably will work just fine and they have not changed anything else. Otherwise we'll find out :D
@mynttt Was able to run the latest version without an issue, thank you very much for the fix :heart:
@leepeuker Glad to hear it works well again! 🎆
yeah sure looks to be working - thank you so much.. And I hope whatever was going on in RL was something good, or was resolved with everyone happy and healthy..
INFO ] - 2024-02-11 18:53:17 @ ImdbTransformer.updateMetadata: Adjust rating: 5.8 -> 5.9 for Under the Christmas Sky [INFO ] - 2024-02-11 18:53:17 @ ImdbTransformer.updateMetadata: Adjust rating: 5.2 -> 5.3 for Hellboy [INFO ] - 2024-02-11 18:53:17 @ ImdbTransformer.updateMetadata: Adjust rating: 7.4 -> 7.3 for Another Thin Man [INFO ] - 2024-02-11 18:53:17 @ ImdbTransformer.updateMetadata: Adjust rating: 5.4 -> 5.5 for One True Loves [INFO ] - 2024-02-11 18:53:17 @ ImdbTransformer.updateMetadata: Adjust rating: 6.8 -> 6.7 for Migration [INFO ] - 2024-02-11 18:53:17 @ ImdbTransformer.updateMetadata: Adjust rating: 6.8 -> 6.7 for Beau Is Afraid [INFO ] - 2024-02-11 18:53:17 @ ImdbTransformer.updateMetadata: Adjust rating: 7.0 -> 6.9 for Barbie [INFO ] - 2024-02-11 18:53:17 @ ImdbTransformer.updateMetadata: Adjust rating: 7.8 -> 7.7 for Captain America: The Winter Soldier [INFO ] - 2024-02-11 18:53:17 @ ImdbTransformer.updateMetadata: Adjust rating: 6.5 -> 6.4 for Navigating Christmas [INFO ] - 2024-02-11 18:53:17 @ ImdbTransformer.updateMetadata: Adjust rating: 5.9 -> 5.6 for The Marvels [INFO ] - 2024-02-11 18:53:17 @ ImdbTransformer.updateMetadata: Adjust rating: 8.2 -> 8.1 for To Be or Not to Be [INFO ] - 2024-02-11 18:53:17 @ ImdbTransformer.updateMetadata: Adjust rating: 7.3 -> 7.2 for The Last Repair Shop [INFO ] - 2024-02-11 18:53:17 @ ImdbTransformer.updateMetadata: Adjust rating: 5.4 -> 5.3 for Run & Gun
It seems like the latest Plex Pass beta update to PMS v1.40+ made some database changes. Those changes are now causing the below error when UpdateTool tries to run. From the Plex changelogs