OFark / Compressarr

37 stars 0 forks source link

Docker constantly crashes with reload page ribbon at the bottom #5

Closed ItsRainingHP closed 2 years ago

ItsRainingHP commented 2 years ago

Seems like a really cool docker if it works. Would be nice to have a friendly GUI over trying to use Radarr-SMA/Sonarr-SMA which is a pain to setup and updates will break the converter without you knowing and then nothing is converted until you notice.

I think my issue with your docker has been the large size of my library. Every time I connect Sonarr/Radarr to Compressar, the webpage has a large "Loading" text across the screen and eventually it crashes my Unraid dashboard. My library is currently just over 40 TBs. I do not want to convert everything but it would be great if I could run jobs.

Another thing that seems lacking is your GUI doesn't display the file extension. Converting media from mkv/avi to mp4 is my primary use case but your current filters do not make that easy.

The Compressarr logs show numerous get calls from Sonarr. Assuming its trying to populate media into its Database. If that is the case, having a progress bar and/or notifications when populating Compressarr's database would be a huge improvement. At this time I have not been able to run a single job and Compressarr is still running Get calls in the logs.

The last improvement I hope to see is the ability to add multiple media folders to Compressarr. For example I split my Anime and TV Shows folders. Anime is its own library so Plex uses AniDB and HAMA for the media.

Some of these suggestions might be planned, requested, or in development and I apologize if this is duplicate information.

OFark commented 2 years ago

Thank you for the feedback. My library is about 40TB and it handles it with no issues. I would need a copy of the logs to see if I can find out what's causing the problem, but it certainly should not be affecting the Unraid Dashboard, that sounds like intense CPU usage, and at the point you're describing all it's doing is fetching a JSON payload from the Radarr and Sonarr API's.

As for the file extensions, this isn't how FFmpeg works. Compressarr is basically a UI for setting up batch jobs with FFmpeg, I thought about writing specific tasks to take one extension to another, but that would mean I would have to filter what can be done in FFmpeg to what I have the time to write. The other feature is that FFmpeg doesn't always use the extensions by name, for example, it calls mkv Matroska, and you have to use that name, which is why these things are in dropdowns, I did kinda try to put the popular ones at the top of the list, but then what I think is popular and what other people use doesn't always line up.

Compressarr has a database to track what it's done, but it doesn't keep a local database of videos, it fetches them from Radarr/Sonarr as required, this way the list is always up to date. It shouldn't take much more than a second or two to parse the API results.

When you say multiple folders, do you mean multiple instances of Radarr/Sonarr? Because it doesn't matter what folder it's in, Radarr/Sonarr just pass back a path to the file, as long as you have mapped the same directories as you did for Radarr/Sonarr it should just pick them up, no matter where they are.

ItsRainingHP commented 2 years ago

I will take another look tonight. Try and pin Compressarr to some cores to avoid my Unraid GUI freezing. I have a 5950x in the server so I thought it was weird to crash the Unraid GUI like that. I am hoping to utilize my P2000 for conversion again. I believe I have debug mode on in the log level unless you need Trace or a different log level.

What I meant by the folders is that my Sonarr has two base folders. The UI only lets me select one unless I am doing it wrong and I can select two folders.

My only desire is to convert media to MP4 containers because most devices do not like MKV containers and I have noticed the Plex is far more responsive and quick to load with MP4 containers. It is the recommended format on plex forums. I would continue to just use Radarr-Sonarr SMA versions except Sonarr/Radarr are moving their dockers to Alpine only. Which according to the developer RandomNinjaAtk does not support Nvidia ffmpeg codecs.

OFark commented 2 years ago

Yeah, Alpine doesn't support Nvidia, which is why my Nvidia supporting version is so big.

The connection to Radarr or Sonarr is literally just a request to the API and reading that data, it's not CPU intensive at all. It would be interesting to see which Docker is consuming the CPU in Unraid. It could be Radarr/Sonarr trying to piece together the data. Though I've not seen that before.

The Base folders option is there to interest something before the paths from Radarr/Sonarr. In Linux they pass back a linux path which can be translated to a drive in Windows. In Windows the path (IIRC) can be relative, so you may need a base folder in front of that. Unraid Docker to Docker shouldn't need a base folder at all, as long as the path that Radarr/Sonarr can see makes sense to Compressarr.

OFark commented 2 years ago

A note on what it sounds like your doing based on my experience: MP4 doesn't support image-based subtitles, MKV does. Any subtitles may have to be extracted to SRT files to work. Hardware-based encoding (I too have a P2000) is a lot faster but this is at the cost of quality AND file size. If you convert to MP4 and encode the video you may find the video's look a lot worse and take up more space. Now, iirc MP4 and MKV share video & audio codecs, so you may be able to just change the container, Compressarr can handle this, and it won't do any re-encodes and will take a little longer than it takes to read the binary data and re-write it to your HDD's.

ItsRainingHP commented 2 years ago

Funny how you have similar hardware to mine. I understand that MKV containers are better quality and have subtitle features, etc. but there is no point in keeping them if the videos are slow to load. I personally haven't seen any difference in quality between MKV and MP4 containers.

I would appreciate the chance to discuss and compare setups. Do you have a discord server? I can DM you my account name too. It really sounds like you have a grasp on optimized media formats and have similar hardware.

OFark commented 2 years ago

I'm saying a Hardware encode will cause a reduction in quality, the container (MP4/MKV/etc) doesn't make any difference to quality.

A thought has occurred, the MKV's that you refer to may be slow to load because the header information is at the end of the stream. You can correct that, with a simple flag you pass to FFmpeg, but this is a maybe and I can't remember what that flag is off the top of my head.

ItsRainingHP commented 2 years ago

After some attempts at using your program I am still unable to run jobs. When I go to the Sonarr tab I still incur an Unraid dashboard crash. I do notice my Sonarr cores maxed out before the crash. After a little bit I can relog to the dashboard. Most of the UI options when selected also results in "An error has occurred. This application may no longer respond until reloaded."

image

My logs (most recent has an error):


{"Timestamp":"23:15:00 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Services.RadarrService","Message":"LinkURL: http://192.168.50.6:7878/api/v3/system/status?apikey=ee54efcbb28c444289e63c3f8ad397c2","State":{"Message":"LinkURL: http://192.168.50.6:7878/api/v3/system/status?apikey=ee54efcbb28c444289e63c3f8ad397c2","{OriginalFormat}":"LinkURL: http://192.168.50.6:7878/api/v3/system/status?apikey=ee54efcbb28c444289e63c3f8ad397c2"},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"},"Test Connection"]}
{"Timestamp":"23:15:00 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Services.RadarrService","Message":"Connecting.","State":{"Message":"Connecting.","{OriginalFormat}":"Connecting."},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"},"Test Connection"]}
{"Timestamp":"23:15:00 ","EventId":0,"LogLevel":"Information","Category":"Compressarr.Services.RadarrService","Message":"Success.","State":{"Message":"Success.","{OriginalFormat}":"Success."},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"},"Test Connection"]}
{"Timestamp":"23:15:00 ","EventId":0,"LogLevel":"Information","Category":"Compressarr.Services.SonarrService","Message":"Test Sonarr Connection.","State":{"Message":"Test Sonarr Connection.","{OriginalFormat}":"Test Sonarr Connection."},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"},"Test Connection"]}
{"Timestamp":"23:15:00 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Services.SonarrService","Message":"LinkURL: http://192.168.50.6:8990/api/system/status?apikey=aa79ace81f5448cc95a91d8763a33352","State":{"Message":"LinkURL: http://192.168.50.6:8990/api/system/status?apikey=aa79ace81f5448cc95a91d8763a33352","{OriginalFormat}":"LinkURL: http://192.168.50.6:8990/api/system/status?apikey=aa79ace81f5448cc95a91d8763a33352"},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"},"Test Connection"]}
{"Timestamp":"23:15:00 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Services.SonarrService","Message":"Connecting.","State":{"Message":"Connecting.","{OriginalFormat}":"Connecting."},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"},"Test Connection"]}
{"Timestamp":"23:15:00 ","EventId":0,"LogLevel":"Information","Category":"Compressarr.Services.SonarrService","Message":"Success.","State":{"Message":"Success.","{OriginalFormat}":"Success."},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"},"Test Connection"]}
{"Timestamp":"23:15:05 ","EventId":0,"LogLevel":"Information","Category":"Compressarr.Services.RadarrService","Message":"Test Radarr Connection.","State":{"Message":"Test Radarr Connection.","{OriginalFormat}":"Test Radarr Connection."},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"},"Test Connection"]}
{"Timestamp":"23:15:05 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Services.RadarrService","Message":"LinkURL: http://192.168.50.6:7878/api/v3/system/status?apikey=ee54efcbb28c444289e63c3f8ad397c2","State":{"Message":"LinkURL: http://192.168.50.6:7878/api/v3/system/status?apikey=ee54efcbb28c444289e63c3f8ad397c2","{OriginalFormat}":"LinkURL: http://192.168.50.6:7878/api/v3/system/status?apikey=ee54efcbb28c444289e63c3f8ad397c2"},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"},"Test Connection"]}
{"Timestamp":"23:15:05 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Services.RadarrService","Message":"Connecting.","State":{"Message":"Connecting.","{OriginalFormat}":"Connecting."},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"},"Test Connection"]}
{"Timestamp":"23:15:05 ","EventId":0,"LogLevel":"Information","Category":"Compressarr.Services.RadarrService","Message":"Success.","State":{"Message":"Success.","{OriginalFormat}":"Success."},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"},"Test Connection"]}
{"Timestamp":"23:15:05 ","EventId":0,"LogLevel":"Information","Category":"Compressarr.Services.SonarrService","Message":"Test Sonarr Connection.","State":{"Message":"Test Sonarr Connection.","{OriginalFormat}":"Test Sonarr Connection."},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"},"Test Connection"]}
{"Timestamp":"23:15:05 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Services.SonarrService","Message":"LinkURL: http://192.168.50.6:8990/api/system/status?apikey=aa79ace81f5448cc95a91d8763a33352","State":{"Message":"LinkURL: http://192.168.50.6:8990/api/system/status?apikey=aa79ace81f5448cc95a91d8763a33352","{OriginalFormat}":"LinkURL: http://192.168.50.6:8990/api/system/status?apikey=aa79ace81f5448cc95a91d8763a33352"},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"},"Test Connection"]}
{"Timestamp":"23:15:05 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Services.SonarrService","Message":"Connecting.","State":{"Message":"Connecting.","{OriginalFormat}":"Connecting."},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"},"Test Connection"]}
{"Timestamp":"23:15:05 ","EventId":0,"LogLevel":"Information","Category":"Compressarr.Services.SonarrService","Message":"Success.","State":{"Message":"Success.","{OriginalFormat}":"Success."},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"},"Test Connection"]}
{"Timestamp":"23:17:08 ","EventId":0,"LogLevel":"Warning","Category":"Compressarr.Application.ApplicationService","Message":"File does not exist","State":{"Message":"File does not exist","{OriginalFormat}":"File does not exist"},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"},"ReadAppSettings"]}
{"Timestamp":"23:17:08 ","EventId":111,"LogLevel":"Error","Category":"Microsoft.AspNetCore.Components.Server.Circuits.CircuitHost","Message":"Unhandled exception in circuit \u0027qCQH-8kpQSySMs2Au24GXpFB3FXjT0g8YpAyjRwy09g\u0027.","Exception":"Microsoft.CSharp.RuntimeBinder.RuntimeBinderException: Cannot perform runtime binding on a null reference at CallSite.Target(Closure , CallSite , Object , String ) at System.Dynamic.UpdateDelegates.UpdateAndExecute2[T0,T1,TRet](CallSite site, T0 arg0, T1 arg1) at Compressarr.Application.ApplicationService.UpdateLogLevel(LogLevel level) in /src/Compressarr/Compressarr/Application/ApplicationService.cs:line 156 at Compressarr.Pages.Options.saveSettings() in /src/Compressarr/Compressarr/Pages/Options.razor:line 235 at System.Threading.Tasks.Task.\u003C\u003Ec.\u003CThrowAsync\u003Eb__128_0(Object state) at Microsoft.AspNetCore.Components.Rendering.RendererSynchronizationContext.ExecuteSynchronously(TaskCompletionSource\u00601 completion, SendOrPostCallback d, Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state) --- End of stack trace from previous location --- at Microsoft.AspNetCore.Components.Rendering.RendererSynchronizationContext.ExecuteBackground(WorkItem item)","State":{"Message":"Unhandled exception in circuit \u0027qCQH-8kpQSySMs2Au24GXpFB3FXjT0g8YpAyjRwy09g\u0027.","CircuitId":"qCQH-8kpQSySMs2Au24GXpFB3FXjT0g8YpAyjRwy09g","{OriginalFormat}":"Unhandled exception in circuit \u0027{CircuitId}\u0027."},"Scopes":[{"Message":"SpanId:0ab2d64b2e925d90, TraceId:670d19fab7732dca6944df5b928aaead, ParentId:0000000000000000","SpanId":"0ab2d64b2e925d90","TraceId":"670d19fab7732dca6944df5b928aaead","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG5LPBTF0IG","ConnectionId":"0HMG5LPBTF0IG"},{"Message":"RequestPath:/_blazor RequestId:0HMG5LPBTF0IG:00000002","RequestId":"0HMG5LPBTF0IG:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:VxjwY14iQVHveRaMPVsHWw","TransportConnectionId":"VxjwY14iQVHveRaMPVsHWw"}]}```
OFark commented 2 years ago

Roughly how many videos do you have in Sonarr? That error was a problem setting the loglevel, not sure why I'll have a look tomorrow, but it certainly won't be causing anything like CPU thrashing. Are there any errors in Sonarr Logs?

ItsRainingHP commented 2 years ago

Roughly 45k episodes. Sonarr and Radarr show no errors.

OFark commented 2 years ago

Yeah, I got half that, I do see an all core CPU spike, but it lasts 2 seconds at the most. I'm loading the data in parallel, I'll see what I can do to put in a limiter.

OFark commented 2 years ago

Right, under the options page you'll find an option for "Maximum simultaneous API calls". For Radarr it does nothing, as Radarr hands over everything in one dump. Sonarr, however, has to do multiple calls to get series, then episodes. This setting will limit the number of simultaneous calls to Sonarr to get the episode files. It was hardcoded to 20 before. So you might want to start with 10, see how that goes?

ItsRainingHP commented 2 years ago

Edit: I see it was hardcoded and you've updated this docker. I will redownload the image and try again tonight. Thanks for the effort developing!

OFark commented 2 years ago

No worries, let me know how you get on.

ItsRainingHP commented 2 years ago

I am still having issues loading the Compressarr Sonarr page. Crashes my Unraid dashboard.

image

image

{"Timestamp":"00:25:53 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Services.SonarrService","Message":"Link: http://192.168.50.6:8989/api/episodefile?seriesid=593\u0026apikey=aa79ace81f5448cc95a91d8763a33352","State":{"Message":"Link: http://192.168.50.6:8989/api/episodefile?seriesid=593\u0026apikey=aa79ace81f5448cc95a91d8763a33352","{OriginalFormat}":"Link: http://192.168.50.6:8989/api/episodefile?seriesid=593\u0026apikey=aa79ace81f5448cc95a91d8763a33352"},"Scopes":[{"Message":"SpanId:72d93393c6c3726a, TraceId:9c7a025e40213de1ba5f164fe3e298d8, ParentId:0000000000000000","SpanId":"72d93393c6c3726a","TraceId":"9c7a025e40213de1ba5f164fe3e298d8","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG97QR3418Q","ConnectionId":"0HMG97QR3418Q"},{"Message":"RequestPath:/_blazor RequestId:0HMG97QR3418Q:00000002","RequestId":"0HMG97QR3418Q:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:6tb5lDh2W7V2a1oi1M5WEQ","TransportConnectionId":"6tb5lDh2W7V2a1oi1M5WEQ"},"Get Series","Requesting Series","Requesting Episode files for Series 593"]}
{"Timestamp":"00:25:53 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Services.SonarrService","Message":"Creating new HTTP client.","State":{"Message":"Creating new HTTP client.","{OriginalFormat}":"Creating new HTTP client."},"Scopes":[{"Message":"SpanId:72d93393c6c3726a, TraceId:9c7a025e40213de1ba5f164fe3e298d8, ParentId:0000000000000000","SpanId":"72d93393c6c3726a","TraceId":"9c7a025e40213de1ba5f164fe3e298d8","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG97QR3418Q","ConnectionId":"0HMG97QR3418Q"},{"Message":"RequestPath:/_blazor RequestId:0HMG97QR3418Q:00000002","RequestId":"0HMG97QR3418Q:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:6tb5lDh2W7V2a1oi1M5WEQ","TransportConnectionId":"6tb5lDh2W7V2a1oi1M5WEQ"},"Get Series","Requesting Series","Requesting Episode files for Series 593"]}
{"Timestamp":"00:25:53 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Services.SonarrService","Message":"Downloading Series List.","State":{"Message":"Downloading Series List.","{OriginalFormat}":"Downloading Series List."},"Scopes":[{"Message":"SpanId:72d93393c6c3726a, TraceId:9c7a025e40213de1ba5f164fe3e298d8, ParentId:0000000000000000","SpanId":"72d93393c6c3726a","TraceId":"9c7a025e40213de1ba5f164fe3e298d8","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG97QR3418Q","ConnectionId":"0HMG97QR3418Q"},{"Message":"RequestPath:/_blazor RequestId:0HMG97QR3418Q:00000002","RequestId":"0HMG97QR3418Q:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:6tb5lDh2W7V2a1oi1M5WEQ","TransportConnectionId":"6tb5lDh2W7V2a1oi1M5WEQ"},"Get Series","Requesting Series","Requesting Episode files for Series 593"]}
{"Timestamp":"00:25:53 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Services.SonarrService","Message":"Success.","State":{"Message":"Success.","{OriginalFormat}":"Success."},"Scopes":[{"Message":"SpanId:72d93393c6c3726a, TraceId:9c7a025e40213de1ba5f164fe3e298d8, ParentId:0000000000000000","SpanId":"72d93393c6c3726a","TraceId":"9c7a025e40213de1ba5f164fe3e298d8","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG97QR3418Q","ConnectionId":"0HMG97QR3418Q"},{"Message":"RequestPath:/_blazor RequestId:0HMG97QR3418Q:00000002","RequestId":"0HMG97QR3418Q:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:6tb5lDh2W7V2a1oi1M5WEQ","TransportConnectionId":"6tb5lDh2W7V2a1oi1M5WEQ"},"Get Series","Requesting Series","Requesting Episode files for Series 593"]}
{"Timestamp":"00:25:53 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Services.SonarrService","Message":"Success.","State":{"Message":"Success.","{OriginalFormat}":"Success."},"Scopes":[{"Message":"SpanId:72d93393c6c3726a, TraceId:9c7a025e40213de1ba5f164fe3e298d8, ParentId:0000000000000000","SpanId":"72d93393c6c3726a","TraceId":"9c7a025e40213de1ba5f164fe3e298d8","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG97QR3418Q","ConnectionId":"0HMG97QR3418Q"},{"Message":"RequestPath:/_blazor RequestId:0HMG97QR3418Q:00000002","RequestId":"0HMG97QR3418Q:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:6tb5lDh2W7V2a1oi1M5WEQ","TransportConnectionId":"6tb5lDh2W7V2a1oi1M5WEQ"},"Get Series","Requesting Series","Requesting Episode files for Series 270"]}
{"Timestamp":"00:25:53 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Services.SonarrService","Message":"Success.","State":{"Message":"Success.","{OriginalFormat}":"Success."},"Scopes":[{"Message":"SpanId:72d93393c6c3726a, TraceId:9c7a025e40213de1ba5f164fe3e298d8, ParentId:0000000000000000","SpanId":"72d93393c6c3726a","TraceId":"9c7a025e40213de1ba5f164fe3e298d8","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG97QR3418Q","ConnectionId":"0HMG97QR3418Q"},{"Message":"RequestPath:/_blazor RequestId:0HMG97QR3418Q:00000002","RequestId":"0HMG97QR3418Q:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:6tb5lDh2W7V2a1oi1M5WEQ","TransportConnectionId":"6tb5lDh2W7V2a1oi1M5WEQ"},"Get Series","Requesting Series","Requesting Episode files for Series 255"]}
{"Timestamp":"00:25:53 ","EventId":0,"LogLevel":"Error","Category":"Compressarr.Services.SonarrService","Message":"Error understanding output from Sonarr","Exception":"System.NullReferenceException: Object reference not set to an instance of an object. at Compressarr.Services.Models.Season..ctor(SeasonJSON seasonJSON, EpisodeFile ef) in /src/Compressarr/Compressarr/Services/Models/Series.cs:line 29 at Compressarr.Services.SonarrService.\u003C\u003Ec__DisplayClass26_0.\u003C\u003CRequestSeries\u003Eb__2\u003Ed.MoveNext() in /src/Compressarr/Compressarr/Services/SonarrService.cs:line 541 --- End of stack trace from previous location --- at Compressarr.Services.SonarrService.\u003C\u003Ec__DisplayClass26_0.\u003C\u003CRequestSeries\u003Eb__2\u003Ed.MoveNext() in /src/Compressarr/Compressarr/Services/SonarrService.cs:line 537 --- End of stack trace from previous location --- at Compressarr.Services.SonarrService.RequestSeries() in /src/Compressarr/Compressarr/Services/SonarrService.cs:line 535","State":{"Message":"Error understanding output from Sonarr","{OriginalFormat}":"Error understanding output from Sonarr"},"Scopes":[{"Message":"SpanId:72d93393c6c3726a, TraceId:9c7a025e40213de1ba5f164fe3e298d8, ParentId:0000000000000000","SpanId":"72d93393c6c3726a","TraceId":"9c7a025e40213de1ba5f164fe3e298d8","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG97QR3418Q","ConnectionId":"0HMG97QR3418Q"},{"Message":"RequestPath:/_blazor RequestId:0HMG97QR3418Q:00000002","RequestId":"0HMG97QR3418Q:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:6tb5lDh2W7V2a1oi1M5WEQ","TransportConnectionId":"6tb5lDh2W7V2a1oi1M5WEQ"},"Get Series","Requesting Series"]}
{"Timestamp":"00:25:53 ","EventId":0,"LogLevel":"Warning","Category":"Compressarr.Services.SonarrService","Message":"Error understanding output from Sonarr. Dumping output to: series.json","State":{"Message":"Error understanding output from Sonarr. Dumping output to: series.json","{OriginalFormat}":"Error understanding output from Sonarr. Dumping output to: series.json"},"Scopes":[{"Message":"SpanId:72d93393c6c3726a, TraceId:9c7a025e40213de1ba5f164fe3e298d8, ParentId:0000000000000000","SpanId":"72d93393c6c3726a","TraceId":"9c7a025e40213de1ba5f164fe3e298d8","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG97QR3418Q","ConnectionId":"0HMG97QR3418Q"},{"Message":"RequestPath:/_blazor RequestId:0HMG97QR3418Q:00000002","RequestId":"0HMG97QR3418Q:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:6tb5lDh2W7V2a1oi1M5WEQ","TransportConnectionId":"6tb5lDh2W7V2a1oi1M5WEQ"},"Get Series","Requesting Series"]}
{"Timestamp":"00:25:53 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Application.FileService","Message":"Getting lock for: /config/debug/series.json","State":{"Message":"Getting lock for: /config/debug/series.json","{OriginalFormat}":"Getting lock for: /config/debug/series.json"},"Scopes":[{"Message":"SpanId:72d93393c6c3726a, TraceId:9c7a025e40213de1ba5f164fe3e298d8, ParentId:0000000000000000","SpanId":"72d93393c6c3726a","TraceId":"9c7a025e40213de1ba5f164fe3e298d8","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG97QR3418Q","ConnectionId":"0HMG97QR3418Q"},{"Message":"RequestPath:/_blazor RequestId:0HMG97QR3418Q:00000002","RequestId":"0HMG97QR3418Q:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:6tb5lDh2W7V2a1oi1M5WEQ","TransportConnectionId":"6tb5lDh2W7V2a1oi1M5WEQ"},"Get Series","Requesting Series",{"Message":"WriteFileAsyncfrom /config/debug/series.json","path":"/config/debug/series.json","{OriginalFormat}":"WriteFileAsyncfrom {path}"}]}
{"Timestamp":"00:25:53 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Application.FileService","Message":"Writing all text to file: /config/debug/series.json","State":{"Message":"Writing all text to file: /config/debug/series.json","{OriginalFormat}":"Writing all text to file: /config/debug/series.json"},"Scopes":[{"Message":"SpanId:72d93393c6c3726a, TraceId:9c7a025e40213de1ba5f164fe3e298d8, ParentId:0000000000000000","SpanId":"72d93393c6c3726a","TraceId":"9c7a025e40213de1ba5f164fe3e298d8","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG97QR3418Q","ConnectionId":"0HMG97QR3418Q"},{"Message":"RequestPath:/_blazor RequestId:0HMG97QR3418Q:00000002","RequestId":"0HMG97QR3418Q:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:6tb5lDh2W7V2a1oi1M5WEQ","TransportConnectionId":"6tb5lDh2W7V2a1oi1M5WEQ"},"Get Series","Requesting Series",{"Message":"WriteFileAsyncfrom /config/debug/series.json","path":"/config/debug/series.json","{OriginalFormat}":"WriteFileAsyncfrom {path}"}]}
{"Timestamp":"00:25:53 ","EventId":0,"LogLevel":"Debug","Category":"Compressarr.Application.FileService","Message":"Getting lock for: /config/debug/series.json","State":{"Message":"Getting lock for: /config/debug/series.json","{OriginalFormat}":"Getting lock for: /config/debug/series.json"},"Scopes":[{"Message":"SpanId:72d93393c6c3726a, TraceId:9c7a025e40213de1ba5f164fe3e298d8, ParentId:0000000000000000","SpanId":"72d93393c6c3726a","TraceId":"9c7a025e40213de1ba5f164fe3e298d8","ParentId":"0000000000000000"},{"Message":"ConnectionId:0HMG97QR3418Q","ConnectionId":"0HMG97QR3418Q"},{"Message":"RequestPath:/_blazor RequestId:0HMG97QR3418Q:00000002","RequestId":"0HMG97QR3418Q:00000002","RequestPath":"/_blazor"},{"Message":"TransportConnectionId:6tb5lDh2W7V2a1oi1M5WEQ","TransportConnectionId":"6tb5lDh2W7V2a1oi1M5WEQ"},"Get Series","Requesting Series",{"Message":"WriteFileAsyncfrom /config/debug/series.json","path":"/config/debug/series.json","{OriginalFormat}":"WriteFileAsyncfrom {path}"}]}
OFark commented 2 years ago

Seems like the models in Compressarr don't agree with the data you're getting back from Sonarr. Can you get me some more info: In Compressarr, in Options, next to the Sonarr URL should be a green tick, click that and it'll give you information about Sonarr, can you copy and paste, or screenshot that here, please?

ItsRainingHP commented 2 years ago

I tried two different Sonarr dockers. At first, I thought maybe the sonarr package I am using is the cause since I am using randomninjaatk/sonarr-sma but I switched to lscr.io/linuxserver/sonarr with the same results.

randomninjaatk/sonarr-sma:

{
  "Success": true,
  "version": "3.0.7.1479",
  "buildTime": "2022-03-06T23:16:23Z",
  "isDebug": false,
  "isProduction": true,
  "isAdmin": false,
  "isUserInteractive": false,
  "startupPath": "/app/sonarr/bin",
  "appData": "/config",
  "isMonoRuntime": true,
  "isMono": true,
  "isLinux": true,
  "isOsx": false,
  "isWindows": false,
  "branch": "develop",
  "authentication": "forms",
  "sqliteVersion": "3.36.0",
  "urlBase": "",
  "runtimeVersion": "6.12.0.122"
}

lscr.io/linuxserver/sonarr

{
  "Success": true,
  "version": "3.0.7.1477",
  "buildTime": "2022-03-04T19:57:49Z",
  "isDebug": false,
  "isProduction": true,
  "isAdmin": false,
  "isUserInteractive": false,
  "startupPath": "/app/sonarr/bin",
  "appData": "/config",
  "osVersion": "18.04",
  "isMonoRuntime": true,
  "isMono": true,
  "isLinux": true,
  "isOsx": false,
  "isWindows": false,
  "branch": "main",
  "authentication": "forms",
  "sqliteVersion": "3.22.0",
  "urlBase": "",
  "runtimeVersion": "5.20.1.34"
}
OFark commented 2 years ago

That's the same version I've been testing with. The problem seems to be that Sonarr is reporting a number of episode files but not reporting any seasons. Which just isn't right,m but I've put in a handler for it now. So it should pull in all the other series and ignore any matching that issue. The update is building now.

ItsRainingHP commented 2 years ago

Switched to Tdarr. Thank you!