search

mynttt / UpdateTool

A tool to update the IMDB ratings for Plex libraries that contain movies/series and use the IMDB agent to receive ratings
GNU General Public License v3.0
250 stars 12 forks source link

URLDecoder: Illegal hex characters in escape (%) pattern #123

Closed YovelD closed 1 month ago

YovelD commented 2 months ago

Hi,

Thank you for this great tool. I'm running into some issue that I cannot quite troubleshoot. You'll find the logs below.

I'm running Plex in my Synology NAS, and using UpdateTool GUI 1.0.8. I have several French and Spanish movies and tv shows with special characters - maybe this is what broke it?

I can see that the TV Show "L'amour est dans le pré" (https://www.imdb.com/title/tt1255201/) hasn't been updated for instance.

Thanks for your help.

Started UpdateTool:

[INFO ] - 2024-06-11 11:28:49 @ Main.loadImplementation: Loaded implementation << imdb-docker << updatetool.imdb.ImdbDockerImplementation
[INFO ] - 2024-06-11 11:28:49 @ Main.main: Running version: 1.8.2
[INFO ] - 2024-06-11 11:28:49 @ Main.main: Args: {imdb-docker={schedule=12}}
[INFO ] - 2024-06-11 11:28:49 @ Main.main: << INIT: imdb-docker @ {schedule=12} >>
[INFO ] - 2024-06-11 11:28:49 @ TmdbApiV3.<init>: Testing TMDB API (v3) key: XXX
[INFO ] - 2024-06-11 11:28:50 @ TmdbApiV3.<init>: Test passed. API Key is valid.
[INFO ] - 2024-06-11 11:28:50 @ ImdbDockerImplementation.bootstrap: TMDB API key enabled TMDB <=> IMDB matching. Will process TMDB backed Movie and TV Series libraries and TMDB orphans.
[INFO ] - 2024-06-11 11:28:50 @ TvdbApiV4.auth: Testing TMDB API (v4) pin: XXX
[INFO ] - 2024-06-11 11:28:50 @ TvdbApiV4.auth: Test passed. API Key is valid.
[INFO ] - 2024-06-11 11:28:50 @ ImdbDockerImplementation.bootstrap: TVDB API (TVDB_V4) authorization enabled IMDB rating update for Movies and TV Series with the TVDB agent.
[INFO ] - 2024-06-11 11:28:50 @ ImdbDockerImplementation.bootstrap: Starting IMDB Watchdog
[INFO ] - 2024-06-11 11:28:50 @ ImdbDockerImplementation.bootstrap: Plex data dir: J:\
[INFO ] - 2024-06-11 11:28:50 @ ImdbDockerImplementation.bootstrap: Unlocking library for IMDB TV Show rating update for the new TV Show agent: 1
[INFO ] - 2024-06-11 11:28:50 @ ImdbDockerImplementation.bootstrap: Unlocking library for IMDB TV Show rating update for the new TV Show agent: 2
[INFO ] - 2024-06-11 11:28:50 @ ImdbDockerImplementation.bootstrap: Capabilities: [TMDB, TVDB]
[INFO ] - 2024-06-11 11:28:50 @ Main.main: << INIT SUCCESS >>
[INFO ] - 2024-06-11 11:28:50 @ Scheduler.go: Scheduler is loading tasks... Blocking until completely set-up and ready to go.
[INFO ] - 2024-06-11 11:28:50 @ Scheduler.go: Scheduling tasks...
[INFO ] - 2024-06-11 11:28:50 @ Scheduler.lambda$go$0: Scheduled imdb-docker task to run @ every 12 hour(s).
[INFO ] - 2024-06-11 11:28:50 @ Scheduler.lambda$go$1: Queued task imdb-docker for immediate execution.
[INFO ] - 2024-06-11 11:28:50 @ Scheduler.go: Running supplied tasks immediately NOW!
[INFO ] - 2024-06-11 11:28:50 @ TaskWrapper.run: ================================================
[INFO ] - 2024-06-11 11:28:50 @ TaskWrapper.run: Starting task: imdb-docker | Execution count: 0
[INFO ] - 2024-06-11 11:28:50 @ TaskWrapper.run: ================================================
[INFO ] - 2024-06-11 11:28:50 @ ImdbDockerImplementation$ImdbBatchJob.run: LIBRARIES => PRE LIBRARY FILTERING
[INFO ] - 2024-06-11 11:28:50 @ ImdbDockerImplementation$ImdbBatchJob.lambda$run$0: Found library [MOVIE] Films (ID=1) with agent: tv.plex.agents.movie and 293 item(s).
[INFO ] - 2024-06-11 11:28:50 @ ImdbDockerImplementation$ImdbBatchJob.lambda$run$0: Found library [SERIES] S�ries TV (ID=2) with agent: tv.plex.agents.series and 1469 item(s).
[INFO ] - 2024-06-11 11:28:50 @ ImdbDockerImplementation$ImdbBatchJob.run: LIBRARIES => POST LIBRARY FILTERING
[INFO ] - 2024-06-11 11:28:50 @ ImdbDockerImplementation$ImdbBatchJob.lambda$run$3: Will process library Films (ID=1) with agent: tv.plex.agents.movie and 293 item(s).
[INFO ] - 2024-06-11 11:28:50 @ ImdbDockerImplementation$ImdbBatchJob.lambda$run$3: Will process library S�ries TV (ID=2) with agent: tv.plex.agents.series and 1469 item(s).
[INFO ] - 2024-06-11 11:28:50 @ ImdbDatabaseSupport.<init>: NewExtraDataFormat has been identified as: true
[INFO ] - 2024-06-11 11:28:53 @ ImdbDockerImplementation$ImdbBatchJob.run: Library IDs on ignore list: []
[INFO ] - 2024-06-11 11:28:53 @ ImdbRatingDatasetFactory.readData: Reading data...
[INFO ] - 2024-06-11 11:28:54 @ ImdbRatingDatasetFactory.readData: 1445124 lines read.
[INFO ] - 2024-06-11 11:28:54 @ TmdbApiV3.<init>: Testing TMDB API (v3) key: XXX
[INFO ] - 2024-06-11 11:28:54 @ TmdbApiV3.<init>: Test passed. API Key is valid.
[INFO ] - 2024-06-11 11:28:54 @ TvdbApiV4.auth: Testing TMDB API (v4) pin: XXX
[INFO ] - 2024-06-11 11:28:54 @ TvdbApiV4.auth: Test passed. API Key is valid.
[INFO ] - 2024-06-11 11:28:54 @ ImdbDockerImplementation$ImdbBatchJob.run: [MOVIE] Films has 293 item(s)
[INFO ] - 2024-06-11 11:28:54 @ ImdbDockerImplementation$ImdbBatchJob.run: [SERIES] S�ries TV has 1469 item(s)
[INFO ] - 2024-06-11 11:28:54 @ ImdbDockerImplementation$ImdbBatchJob.run: Processing [MOVIE] Films with UUID c64b4893-8353-4eed-966b-1ee630c0d09a at stage: CREATED
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.analyseDatabase: Resolving IMDB identifiers for items. Only warnings and errors will show up...
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.analyseDatabase: Items that show up here will not be processed by further stages of the pipeline.
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.analyseDatabase: Starting watchdog to print progress to std::out with a delay and interval of 1 minute...
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.analyseDatabase: Progress printing watchdog has been stopped. Cancelation status: true
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.analyseDatabase: Save point: Persisting caches to keep queried look-up data in case of crashes or hang-ups.
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.analyseDatabase: Filtered 0 invalid item(s).
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.transformMetadata: 291 item(s) need no update.
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.transformMetadata: Transforming 0 item(s)
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.transformMetadata: Transformed entries for 0 items(s).
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.transformMetadata: Save point: Persisting caches to keep queried look-up data in case of crashes or hang-ups.
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.updateDatabase: Nothing to update. Skipping...
[INFO ] - 2024-06-11 11:28:54 @ ImdbDockerImplementation$ImdbBatchJob.run: Job returned PASS : Job finished correctly
[INFO ] - 2024-06-11 11:28:54 @ ImdbDockerImplementation$ImdbBatchJob.run: Job finished successfully for [MOVIE] Films with UUID c64b4893-8353-4eed-966b-1ee630c0d09a
[INFO ] - 2024-06-11 11:28:54 @ ImdbDockerImplementation$ImdbBatchJob.run: Processing [SERIES] S�ries TV with UUID f197e064-1710-4c16-8b8a-1061ed8f1999 at stage: CREATED
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.analyseDatabase: Resolving IMDB identifiers for items. Only warnings and errors will show up...
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.analyseDatabase: Items that show up here will not be processed by further stages of the pipeline.
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.analyseDatabase: Starting watchdog to print progress to std::out with a delay and interval of 1 minute...
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://4484)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://291148)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://291400)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://294047)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://294810)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://294956)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://294957)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://294958)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://295022)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://295024)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://295079)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://295085)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://295093)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://295094)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://295095)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://295096)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://295132)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://295269)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '49e c�r�monie des C�sar' has no matching IMDB resolver and will be ignored. (guid=local://295511)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://295655)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://295656)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://295657)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: 'Zone Interdite' has no matching IMDB resolver and will be ignored. (guid=local://294808)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://294046)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://294809)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: '' has no matching IMDB resolver and will be ignored. (guid=local://294955)
[WARN ] - 2024-06-11 11:28:54 @ DefaultResolvement.resolve: Item: 'C�r�monie des C�sar' has no matching IMDB resolver and will be ignored. (guid=local://295510)
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.analyseDatabase: Progress printing watchdog has been stopped. Cancelation status: true
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.analyseDatabase: Save point: Persisting caches to keep queried look-up data in case of crashes or hang-ups.
[INFO ] - 2024-06-11 11:28:54 @ ImdbPipeline.analyseDatabase: Filtered 434 invalid item(s).
[ERROR] - 2024-06-11 11:28:55 @ OldExtraData.<init>: Tried to decode [KEY] := 'ler tasksGenerateMediaIndexFilesfg?fg??d??
1SGmedia.generate.bifGenerating video preview thumbnailsEl guardi�n invisible (2017)fg??fg?
>??
%;^?butlerBut]??
1S9media.generate.bifGenerating video preview thumbnailsThe Peripheral S01 E07fg??fg??b??
1SCmedia.generate'
[ERROR] - 2024-06-11 11:28:55 @ OldExtraData.<init>: Tried to decode [VALUE] := ''
[INFO ] - 2024-06-11 11:28:55 @ ImdbDockerImplementation$ImdbBatchJob.run: Job returned ERROR : URLDecoder: Illegal hex characters in escape (%) pattern - Error at index 0 in: ";"
[ERROR] - 2024-06-11 11:28:55 @ ImdbDockerImplementation$ImdbBatchJob.run: IllegalArgumentException exception encountered...
[ERROR] - 2024-06-11 11:28:55 @ ImdbDockerImplementation$ImdbBatchJob.run: Please contact the maintainer of the application with the stacktrace below if you think this is unwanted behavior.
[ERROR] - 2024-06-11 11:28:55 @ ImdbDockerImplementation$ImdbBatchJob.run: ========================================
[ERROR] - 2024-06-11 11:28:55 @ ImdbDockerImplementation$ImdbBatchJob.run: java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in escape (%) pattern - Error at index 0 in: ";"
    at java.base/java.net.URLDecoder.decode(URLDecoder.java:243)
    at updatetool.common.OldExtraData.<init>(OldExtraData.java:40)
    at updatetool.common.ExtraData.of(ExtraData.java:20)
    at updatetool.imdb.ImdbTransformer.needsUpdate(ImdbTransformer.java:40)
    at java.base/java.util.function.Predicate.lambda$negate$1(Predicate.java:80)
    at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:178)
    at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1858)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
    at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
    at updatetool.imdb.ImdbPipeline.transformMetadata(ImdbPipeline.java:190)
    at updatetool.imdb.ImdbPipeline.transformMetadata(ImdbPipeline.java:45)
    at updatetool.api.Pipeline.invoke(Pipeline.java:12)
    at updatetool.imdb.ImdbJobRunner.run(ImdbJobRunner.java:19)
    at updatetool.imdb.ImdbDockerImplementation$ImdbBatchJob.run(ImdbDockerImplementation.java:286)
    at updatetool.TaskWrapper.run(Main.java:252)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1583)
[ERROR] - 2024-06-11 11:28:55 @ ImdbDockerImplementation$ImdbBatchJob.run: ========================================
[ERROR] - 2024-06-11 11:28:55 @ ImdbDockerImplementation$ImdbBatchJob.run: The application will terminate now.

UpdateTool stopped.
mynttt commented 1 month ago

So there is a field in the database called extra_data which Plex uses to store some metadata related to each entry. This is used to store the IMDB badges that then get rendered in Plex itself.

Back in the days this was encoded as a URL with ?key=value&key2=value2 pairs as seen here nowadays they have a new JSON like (I think it even is JSON) approach for that which is decoded here.

In your case the error message comes because the data for the item is malformed: 

ler tasksGenerateMediaIndexFilesfg?fg??d??
1SGmedia.generate.bifGenerating video preview thumbnailsEl guardi�n invisible (2017)fg??fg?
>??
%;^?butlerBut]??
1S9media.generate.bifGenerating video preview thumbnailsThe Peripheral S01 E07fg??fg??b??
1SCmedia.generate

So the Java Internal URLEncoder fails when being called from OldExtraData. Only way to solve this is go into the database and set the offending field to an empty string.

Let me know if the issue still persists. Another way to resolve this is deleting the Library and recrating it; since then it will 100% use the new format and rebuild all extra_data strings anyways.