search

MediaBrowser / Emby

Emby Server is a personal media server with apps on just about every device.
https://emby.media
GNU General Public License v2.0
3.99k stars 792 forks source link

Crash During Playback: NullReferenceException: Emby.Api.Images.ImageService.GetImage #3790

Closed Noah-Huppert closed 8 months ago

Noah-Huppert commented 10 months ago

Recently while playing the last 3 episodes of a TV show Emby has crashed 10-20 minutes in. In the logs I see a NullReferenceException, the first method in the stack trace is Emby.Api.Images.ImageService.GetImage. After about 5 minutes the server recovers and I can start viewing media again.

I am running Emby on Kubernetes via the emby/embyserver:4.8.0.39 image. The media and config directories are provided by Rook Ceph. I am viewing media via the Nvidia Shield Emby app.

Full Logs 
2023-09-04 00:00:00.224 Info App: Emby Server Version: 4.8.0.39
2023-09-04 00:00:00.270 Info App: Emby
    Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
    Operating system: Linux version 6.1.0-11-amd64 (debian-kernel@lists.debian.org) (gcc-12 (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT
    Framework: .NET 6.0.14
    OS/Process: x64/x64
    Runtime: system/System.Private.CoreLib.dll
    Processor count: 2
    Data path: /config
    Application path: /system
2023-09-04 00:00:00.270 Info App: Logs path: /config/logs
2023-09-04 00:00:00.270 Info App: Cache path: /config/cache
2023-09-04 00:00:00.270 Info App: Internal metadata path: /config/metadata
2023-09-04 00:00:00.270 Info App: Transcoding temporary files path: /config/transcoding-temp
2023-09-04 00:00:00.272 Info App: Plugins:
    Bluray Folder Support 1.0.0.0
    Cinema Intros 1.0.44.0
    DLNA 1.2.8.0
    Dvd Folder Support 1.0.0.0
    Emby Guide Data 1.0.12.0
    Fanart.tv 1.0.15.0
    M3U TV Tuner 1.0.17.0
    MovieDb 1.6.9.0
    MusicBrainz 1.0.22.0
    Nfo Metadata 1.0.76.0
    OMDb 1.0.20.0
    Open Subtitles 1.0.35.0
    Port Mapper 1.2.3.0
    Statistics 3.0.10.0
    Studio Images 1.0.3.0
    TheAudioDb 1.0.18.0
    TheTVDB 1.4.1.0
    Webhooks 1.0.30.0
    XmlTV 1.1.3.0
2023-09-04 00:10:04.351 Error Server: Error processing request
    *** Error Report ***
    Version: 4.8.0.39
    Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
    Operating system: Linux version 6.1.0-11-amd64 (debian-kernel@lists.debian.org) (gcc-12 (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT
    Framework: .NET 6.0.14
    OS/Process: x64/x64
    Runtime: system/System.Private.CoreLib.dll
    Processor count: 2
    Data path: /config
    Application path: /system
    System.NullReferenceException: System.NullReferenceException: Object reference not set to an instance of an object.
       at Emby.Api.Images.ImageService.GetImage(ImageRequest request, Int64 itemId, BaseItem item, Boolean isHeadRequest)
       at Emby.Server.Implementations.Services.ServiceController.GetTaskResult(Task task)
       at Emby.Server.Implementations.Services.ServiceHandler.ProcessRequestAsync(HttpListenerHost httpHost, IServerApplicationHost appHost, IRequest httpReq, IResponse httpRes, RestPath restPath, String responseContentType, CancellationToken cancellationToken)
       at Emby.Server.Implementations.HttpServer.HttpListenerHost.RequestHandler(IRequest httpReq, ReadOnlyMemory`1 urlString, ReadOnlyMemory`1 localPath, CancellationToken cancellationToken)
    Source: Emby.Api
    TargetSite: Void MoveNext()

2023-09-04 00:10:04.351 Error Server: Error processing request
    *** Error Report ***
    Version: 4.8.0.39
    Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
    Operating system: Linux version 6.1.0-11-amd64 (debian-kernel@lists.debian.org) (gcc-12 (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT
    Framework: .NET 6.0.14
    OS/Process: x64/x64
    Runtime: system/System.Private.CoreLib.dll
    Processor count: 2
    Data path: /config
    Application path: /system
    System.NullReferenceException: System.NullReferenceException: Object reference not set to an instance of an object.
       at Emby.Api.Images.ImageService.GetImage(ImageRequest request, Int64 itemId, BaseItem item, Boolean isHeadRequest)
       at Emby.Server.Implementations.Services.ServiceController.GetTaskResult(Task task)
       at Emby.Server.Implementations.Services.ServiceHandler.ProcessRequestAsync(HttpListenerHost httpHost, IServerApplicationHost appHost, IRequest httpReq, IResponse httpRes, RestPath restPath, String responseContentType, CancellationToken cancellationToken)
       at Emby.Server.Implementations.HttpServer.HttpListenerHost.RequestHandler(IRequest httpReq, ReadOnlyMemory`1 urlString, ReadOnlyMemory`1 localPath, CancellationToken cancellationToken)
    Source: Emby.Api
    TargetSite: Void MoveNext()

2023-09-04 00:10:04.354 Error Server: Error processing request
    *** Error Report ***
    Version: 4.8.0.39
    Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
    Operating system: Linux version 6.1.0-11-amd64 (debian-kernel@lists.debian.org) (gcc-12 (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT
    Framework: .NET 6.0.14
    OS/Process: x64/x64
    Runtime: system/System.Private.CoreLib.dll
    Processor count: 2
    Data path: /config
    Application path: /system
    System.NullReferenceException: System.NullReferenceException: Object reference not set to an instance of an object.
       at Emby.Api.Images.ImageService.GetImage(ImageRequest request, Int64 itemId, BaseItem item, Boolean isHeadRequest)
       at Emby.Server.Implementations.Services.ServiceController.GetTaskResult(Task task)
       at Emby.Server.Implementations.Services.ServiceHandler.ProcessRequestAsync(HttpListenerHost httpHost, IServerApplicationHost appHost, IRequest httpReq, IResponse httpRes, RestPath restPath, String responseContentType, CancellationToken cancellationToken)
       at Emby.Server.Implementations.HttpServer.HttpListenerHost.RequestHandler(IRequest httpReq, ReadOnlyMemory`1 urlString, ReadOnlyMemory`1 localPath, CancellationToken cancellationToken)
    Source: Emby.Api
    TargetSite: Void MoveNext()

2023-09-04 00:10:04.354 Error Server: Error processing request
    *** Error Report ***
    Version: 4.8.0.39
    Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
    Operating system: Linux version 6.1.0-11-amd64 (debian-kernel@lists.debian.org) (gcc-12 (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT
    Framework: .NET 6.0.14
    OS/Process: x64/x64
    Runtime: system/System.Private.CoreLib.dll
    Processor count: 2
    Data path: /config
    Application path: /system
    System.NullReferenceException: System.NullReferenceException: Object reference not set to an instance of an object.
       at Emby.Api.Images.ImageService.GetImage(ImageRequest request, Int64 itemId, BaseItem item, Boolean isHeadRequest)
       at Emby.Server.Implementations.Services.ServiceController.GetTaskResult(Task task)
       at Emby.Server.Implementations.Services.ServiceHandler.ProcessRequestAsync(HttpListenerHost httpHost, IServerApplicationHost appHost, IRequest httpReq, IResponse httpRes, RestPath restPath, String responseContentType, CancellationToken cancellationToken)
       at Emby.Server.Implementations.HttpServer.HttpListenerHost.RequestHandler(IRequest httpReq, ReadOnlyMemory`1 urlString, ReadOnlyMemory`1 localPath, CancellationToken cancellationToken)
    Source: Emby.Api
    TargetSite: Void MoveNext()

2023-09-04 00:10:04.354 Error Server: Error processing request
    *** Error Report ***
    Version: 4.8.0.39
    Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
    Operating system: Linux version 6.1.0-11-amd64 (debian-kernel@lists.debian.org) (gcc-12 (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT
    Framework: .NET 6.0.14
    OS/Process: x64/x64
    Runtime: system/System.Private.CoreLib.dll
    Processor count: 2
    Data path: /config
    Application path: /system
    System.NullReferenceException: System.NullReferenceException: Object reference not set to an instance of an object.
       at Emby.Api.Images.ImageService.GetImage(ImageRequest request, Int64 itemId, BaseItem item, Boolean isHeadRequest)
       at Emby.Server.Implementations.Services.ServiceController.GetTaskResult(Task task)
       at Emby.Server.Implementations.Services.ServiceHandler.ProcessRequestAsync(HttpListenerHost httpHost, IServerApplicationHost appHost, IRequest httpReq, IResponse httpRes, RestPath restPath, String responseContentType, CancellationToken cancellationToken)
       at Emby.Server.Implementations.HttpServer.HttpListenerHost.RequestHandler(IRequest httpReq, ReadOnlyMemory`1 urlString, ReadOnlyMemory`1 localPath, CancellationToken cancellationToken)
    Source: Emby.Api
    TargetSite: Void MoveNext()

2023-09-04 00:10:04.450 Error Server: Error processing request
    *** Error Report ***
    Version: 4.8.0.39
    Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
    Operating system: Linux version 6.1.0-11-amd64 (debian-kernel@lists.debian.org) (gcc-12 (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT
    Framework: .NET 6.0.14
    OS/Process: x64/x64
    Runtime: system/System.Private.CoreLib.dll
    Processor count: 2
    Data path: /config
    Application path: /system
    System.NullReferenceException: System.NullReferenceException: Object reference not set to an instance of an object.
       at Emby.Api.Images.ImageService.GetImage(ImageRequest request, Int64 itemId, BaseItem item, Boolean isHeadRequest)
       at Emby.Server.Implementations.Services.ServiceController.GetTaskResult(Task task)
       at Emby.Server.Implementations.Services.ServiceHandler.ProcessRequestAsync(HttpListenerHost httpHost, IServerApplicationHost appHost, IRequest httpReq, IResponse httpRes, RestPath restPath, String responseContentType, CancellationToken cancellationToken)
       at Emby.Server.Implementations.HttpServer.HttpListenerHost.RequestHandler(IRequest httpReq, ReadOnlyMemory`1 urlString, ReadOnlyMemory`1 localPath, CancellationToken cancellationToken)
    Source: Emby.Api
    TargetSite: Void MoveNext()
Noah-Huppert commented 9 months ago

I'm looking into it but I'm thinking that this could be an issue with my infrastructure.

I am running Ceph and it looks like issues with your Ceph cluster can cause permission denied errors: https://access.redhat.com/solutions/7004783

LukePulverenti commented 8 months ago

Hi, what did you find? Are you still having an issue with this?

Noah-Huppert commented 8 months ago

Hi, what did you find? Are you still having an issue with this?

I found that my Ceph OSDs were running out of memory, crashing, and causing the Ceph cluster to thrash and restart loop. This then caused the Emby error because it couldn't access files. I never found the root cause but this mailing list post has some good ideas I was going to try. I ended up resorting to using plain old Kubernetes PVCs and placement affinity instead of a distributed storage cluster.

LukePulverenti commented 8 months ago

Thanks for the info.