search

MediaBrowser / Emby

Emby Server is a personal media server with apps on just about every device.
https://emby.media
GNU General Public License v2.0
3.99k stars 792 forks source link

Unhandled exception / System.InvalidCastException #3793

Closed mikabytes closed 9 months ago

mikabytes commented 9 months ago

I had a bad case of a Gluster-based storage system going particularly nasty on me. For some time some calls to fileExist() or stat() system calls would respond with false until a listing was made on the containing directory. This made Emby behave quite weirdly. So I restored the system from a backup and now I have a strange exception on startup that I just cannot get rid of.

I'm running docker container emby/embyserver:4.7.14.0 and /config directory has been fully restored to a known "good" state. Also tried container emby/embyserver:4.7.13.0. Anyone got ideas what I could try next?

Edit: Now have also tried moving it to another server. No change.

Logs:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
Info Main: Application path: /system/EmbyServer.dll
Info Main: Emby
    Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
    Operating system: Linux version 5.15.0-84-generic (buildd@lcy02-amd64-005) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #93-Ubuntu
    Framework: .NET 6.0.20
    OS/Process: x64/x64
    Runtime: system/System.Private.CoreLib.dll
    Processor count: 8
    Data path: /config
    Application path: /system
Info Main: Logs path: /config/logs
Info Main: Cache path: /config/cache
Info Main: Internal metadata path: /config/metadata
Info Main: Transcoding temporary files path: /config/transcoding-temp
Info App: Application version: 4.7.14.0
Info App: Loading assemblies
Info App: File /config/plugins/Emby.Server.CinemaMode.dll has version 1.0.40.0
Info App: File /system/plugins/Emby.Server.CinemaMode.dll has version 1.0.40.0
Info App: File /config/plugins/Fanart.dll has version 1.0.15.0
Info App: File /system/plugins/Fanart.dll has version 1.0.13.0
Info App: File /config/plugins/Emby.PortMapper.dll has version 1.2.1.0
Info App: File /system/plugins/Emby.PortMapper.dll has version 1.1.8.0
Info App: File /config/plugins/StudioImages.dll has version 1.0.3.0
Info App: File /system/plugins/StudioImages.dll has version 1.0.3.0
Info App: File /config/plugins/Tvdb.dll has version 1.4.1.0
Info App: File /system/plugins/Tvdb.dll has version 1.3.3.0
Info App: File /config/plugins/DvdMounter.dll has version 1.0.0.0
Info App: File /system/plugins/DvdMounter.dll has version 1.0.0.0
Info App: File /config/plugins/MovieDb.dll has version 1.7.3.0
Info App: File /system/plugins/MovieDb.dll has version 1.6.2.0
Info App: File /config/plugins/OpenSubtitles.dll has version 1.0.34.0
Info App: File /system/plugins/OpenSubtitles.dll has version 1.0.31.0
Info App: File /config/plugins/MusicBrainz.dll has version 1.0.22.0
Info App: File /system/plugins/MusicBrainz.dll has version 1.0.22.0
Info App: File /config/plugins/Emby.Webhooks.dll has version 1.0.25.0
Info App: File /system/plugins/Emby.Webhooks.dll has version 1.0.25.0
Info App: File /config/plugins/OMDb.dll has version 1.0.20.0
Info App: File /system/plugins/OMDb.dll has version 1.0.18.0
Info App: File /config/plugins/EmbyGuideData.dll has version 1.0.12.0
Info App: File /system/plugins/EmbyGuideData.dll has version 1.0.10.0
Info App: File /config/plugins/AudioDb.dll has version 1.0.18.0
Info App: File /system/plugins/AudioDb.dll has version 1.0.16.0
Info App: File /config/plugins/NfoMetadata.dll has version 1.0.77.0
Info App: File /system/plugins/NfoMetadata.dll has version 1.0.70.0
Info App: File /config/plugins/Emby.Dlna.dll has version 1.2.7.0
Info App: File /system/plugins/Emby.Dlna.dll has version 1.0.91.0
Info App: File /config/plugins/BlurayMounter.dll has version 1.0.0.0
Info App: File /system/plugins/BlurayMounter.dll has version 1.0.0.0
Info App: File /config/plugins/Emby.M3UTuner.dll has version 1.0.13.0
Info App: File /system/plugins/Emby.M3UTuner.dll has version 1.0.13.0
Info App: File /config/plugins/Emby.XmlTV.dll has version 1.1.3.0
Info App: File /system/plugins/Emby.XmlTV.dll has version 1.0.9.0
Info App: Loading AudioDb, Version=1.0.18.0, Culture=neutral, PublicKeyToken=null from /config/plugins/AudioDb.dll
Info App: Loading DvdMounter, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null from /config/plugins/DvdMounter.dll
Info App: Loading Emby.Dlna, Version=1.2.7.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Emby.Dlna.dll
Info App: Loading Emby.Webhooks, Version=1.0.25.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Emby.Webhooks.dll
Info App: Loading Emby.XmlTV, Version=1.1.3.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Emby.XmlTV.dll
Info App: Loading MusicBrainz, Version=1.0.22.0, Culture=neutral, PublicKeyToken=null from /config/plugins/MusicBrainz.dll
Info App: Loading OpenSubtitles, Version=1.0.34.0, Culture=neutral, PublicKeyToken=null from /config/plugins/OpenSubtitles.dll
Info App: Loading Tvdb, Version=1.4.1.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Tvdb.dll
Info App: Loading BlurayMounter, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null from /config/plugins/BlurayMounter.dll
Info App: Loading OMDb, Version=1.0.20.0, Culture=neutral, PublicKeyToken=null from /config/plugins/OMDb.dll
Info App: Loading Emby.Server.CinemaMode, Version=1.0.40.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Emby.Server.CinemaMode.dll
Info App: Loading Fanart, Version=1.0.15.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Fanart.dll
Info App: Loading StudioImages, Version=1.0.3.0, Culture=neutral, PublicKeyToken=null from /config/plugins/StudioImages.dll
Info App: Loading MovieDb, Version=1.7.3.0, Culture=neutral, PublicKeyToken=null from /config/plugins/MovieDb.dll
Info App: Loading EmbyGuideData, Version=1.0.12.0, Culture=neutral, PublicKeyToken=null from /config/plugins/EmbyGuideData.dll
Info App: Loading NfoMetadata, Version=1.0.77.0, Culture=neutral, PublicKeyToken=null from /config/plugins/NfoMetadata.dll
Info App: Loading Emby.M3UTuner, Version=1.0.13.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Emby.M3UTuner.dll
Info App: Loading Emby.Server.CinemaMode, Version=1.0.40.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Emby.Server.CinemaMode.dll
Info App: Loading Emby.M3UTuner, Version=1.0.13.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Emby.M3UTuner.dll
Info App: Loading Fanart, Version=1.0.15.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Fanart.dll
Info App: Loading NfoMetadata, Version=1.0.77.0, Culture=neutral, PublicKeyToken=null from /config/plugins/NfoMetadata.dll
Info App: Loading EmbyGuideData, Version=1.0.12.0, Culture=neutral, PublicKeyToken=null from /config/plugins/EmbyGuideData.dll
Info App: Loading MovieDb, Version=1.7.3.0, Culture=neutral, PublicKeyToken=null from /config/plugins/MovieDb.dll
Info App: Loading Emby.PortMapper, Version=1.2.1.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Emby.PortMapper.dll
Info App: Loading Emby.PortMapper, Version=1.2.1.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Emby.PortMapper.dll
Info App: Loading StudioImages, Version=1.0.3.0, Culture=neutral, PublicKeyToken=null from /config/plugins/StudioImages.dll
Info App: Loading Tvdb, Version=1.4.1.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Tvdb.dll
Info App: Loading MusicBrainz, Version=1.0.22.0, Culture=neutral, PublicKeyToken=null from /config/plugins/MusicBrainz.dll
Info App: Loading AudioDb, Version=1.0.18.0, Culture=neutral, PublicKeyToken=null from /config/plugins/AudioDb.dll
Info App: Loading Emby.Dlna, Version=1.2.7.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Emby.Dlna.dll
Info App: Loading Emby.XmlTV, Version=1.1.3.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Emby.XmlTV.dll
Info App: Loading DvdMounter, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null from /config/plugins/DvdMounter.dll
Info App: Loading OpenSubtitles, Version=1.0.34.0, Culture=neutral, PublicKeyToken=null from /config/plugins/OpenSubtitles.dll
Info App: Loading Emby.Webhooks, Version=1.0.25.0, Culture=neutral, PublicKeyToken=null from /config/plugins/Emby.Webhooks.dll
Info App: Loading OMDb, Version=1.0.20.0, Culture=neutral, PublicKeyToken=null from /config/plugins/OMDb.dll
Info App: Loading BlurayMounter, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null from /config/plugins/BlurayMounter.dll
Info App: Loading Emby.Api, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading Emby.Web, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading MediaBrowser.Model, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading MediaBrowser.Common, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading MediaBrowser.Controller, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading Emby.Providers, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading Emby.Photos, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading Emby.Server.Implementations, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading Emby.LiveTV, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading Emby.ActivityLog, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading Emby.Server.MediaEncoding, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading Emby.LocalMetadata, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading Emby.Notifications, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading Emby.Codecs.Dxva, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading Emby.Codecs, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading Emby.Server.Connect, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading Emby.Server.Sync, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info App: Loading EmbyServer, Version=4.7.14.0, Culture=neutral, PublicKeyToken=null
Info SqliteUserRepository: Sqlite version: 3.39.2
Info SqliteUserRepository: Sqlite compiler options: ATOMIC_INTRINSICS=1,COMPILER=gcc-10.3.0,DEFAULT_AUTOVACUUM,DEFAULT_CACHE_SIZE=-2000,DEFAULT_FILE_FORMAT=4,DEFAULT_JOURNAL_SIZE_LIMIT=-1,DEFAULT_MMAP_SIZE=0,DEFAULT_PAGE_SIZE=4096,DEFAULT_PCACHE_INITSZ=20,DEFAULT_RECURSIVE_TRIGGERS,DEFAULT_SECTOR_SIZE=4096,DEFAULT_SYNCHRONOUS=2,DEFAULT_WAL_AUTOCHECKPOINT=1000,DEFAULT_WAL_SYNCHRONOUS=2,DEFAULT_WORKER_THREADS=0,ENABLE_COLUMN_METADATA,ENABLE_DBSTAT_VTAB,ENABLE_FTS3,ENABLE_FTS3_PARENTHESIS,ENABLE_FTS3_TOKENIZER,ENABLE_FTS4,ENABLE_FTS5,ENABLE_GEOPOLY,ENABLE_MATH_FUNCTIONS,ENABLE_PREUPDATE_HOOK,ENABLE_RTREE,ENABLE_SESSION,ENABLE_UNLOCK_NOTIFY,ENABLE_UPDATE_DELETE_LIMIT,LIKE_DOESNT_MATCH_BLOBS,MALLOC_SOFT_LIMIT=1024,MAX_ATTACHED=10,MAX_COLUMN=2000,MAX_COMPOUND_SELECT=500,MAX_DEFAULT_PAGE_SIZE=8192,MAX_EXPR_DEPTH=1000,MAX_FUNCTION_ARG=127,MAX_LENGTH=1000000000,MAX_LIKE_PATTERN_LENGTH=50000,MAX_MMAP_SIZE=0x7fff0000,MAX_PAGE_COUNT=1073741823,MAX_PAGE_SIZE=65536,MAX_SCHEMA_RETRY=25,MAX_SQL_LENGTH=1000000000,MAX_TRIGGER_DEPTH=1000,MAX_VARIABLE_NUMBER=250000,MAX_VDBE_OP=250000000,MAX_WORKER_THREADS=8,MUTEX_PTHREADS,OMIT_LOOKASIDE,SECURE_DELETE,SYSTEM_MALLOC,TEMP_STORE=1,THREADSAFE=1
Info SqliteUserRepository: Opening sqlite connection to /config/data/users.db
Info SqliteUserRepository: Default journal_mode for /config/data/users.db is wal
Info ActivityRepository: Opening sqlite connection to /config/data/activitylog.db
Info ActivityRepository: Default journal_mode for /config/data/activitylog.db is wal
Info SqliteDisplayPreferencesRepository: Opening sqlite connection to /config/data/displaypreferences.db
Info NetworkManager: Detecting local network addresses
Info NetworkManager: networkInterface: Ethernet eth0, Speed: 10000000000, Description: eth0
Info NetworkManager: GatewayAddresses: 169.254.1.1
Info NetworkManager: UnicastAddresses: 10.1.106.99,fe80::387f:5aff:feba:74a2%3
Info NetworkManager: networkInterface: Loopback lo, Speed: -1, Description: lo
Info NetworkManager: GatewayAddresses: 
Info NetworkManager: UnicastAddresses: 127.0.0.1,::1
Info NetworkManager: Detected local ip addresses: 10.1.106.99, fe80::387f:5aff:feba:74a2%3, 127.0.0.1, ::1
Info SqliteDisplayPreferencesRepository: Default journal_mode for /config/data/displaypreferences.db is wal
Info App: Adding HttpListener prefix http://+:8096/
Info AuthenticationRepository: Opening sqlite connection to /config/data/authentication.db
Info AuthenticationRepository: Default journal_mode for /config/data/authentication.db is wal
Info SqliteItemRepository: Opening sqlite connection to /config/data/library.db
Info SqliteItemRepository: Default journal_mode for /config/data/library.db is wal
Info SqliteItemRepository: cache_size is -98304
Info SqliteItemRepository: page_size is 4096
Info FfmpegManager: FFMpeg: /bin/ffmpeg
Info FfmpegManager: FFProbe: /bin/ffprobe
Info FfmpegManager: FFDetect: /bin/ffdetect
Info Skia: SkiaSharp version: 2.80.0.0
Info TaskManager: Daily trigger for Thumbnail image extraction set to fire at 09/30/2023 02:00:00, which is 679.6945035083334 minutes from now.
Info TaskManager: Daily trigger for Rotate log file set to fire at 09/30/2023 00:00:00, which is 559.69084622 minutes from now.
Info App: ServerId: 98f3745d96414296961a3fe6e43b49a1
Info App: Starting entry point Emby.Dlna.Main.DlnaEntryPoint
Info App: Entry point completed: Emby.Dlna.Main.DlnaEntryPoint. Duration: 0.0527346 seconds
Info App: Starting entry point Emby.Dlna.Main.DlnaEntryPoint
Info App: Entry point completed: Emby.Dlna.Main.DlnaEntryPoint. Duration: 0.0187081 seconds
Info App: Starting entry point Emby.Server.Implementations.Networking.RemoteAddressEntryPoint
Info App: Loading data from /config/data/wan.dat
Error GlobalExceptionHandler: UnhandledException
    *** Error Report ***
    Version: 4.7.14.0
    Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
    Operating system: Linux version 5.15.0-84-generic (buildd@lcy02-amd64-005) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #93-Ubuntu
    Framework: .NET 6.0.20
    OS/Process: x64/x64
    Runtime: system/System.Private.CoreLib.dll
    Processor count: 8
    Data path: /config
    Application path: /system
    System.InvalidCastException: System.InvalidCastException: [A]Emby.Dlna.DlnaOptions cannot be cast to [B]Emby.Dlna.DlnaOptions. Type A originates from 'Emby.Dlna, Version=1.2.7.0, Culture=neutral, PublicKeyToken=null' in the context 'Default' in a byte array. Type B originates from 'Emby.Dlna, Version=1.2.7.0, Culture=neutral, PublicKeyToken=null' in the context 'Default' in a byte array.
       at MediaBrowser.Common.Configuration.ConfigurationManagerExtensions.GetConfiguration[T](IConfigurationManager manager, String key)
       at Emby.Dlna.ConfigurationExtension.GetDlnaConfiguration(IConfigurationManager manager)
       at Emby.Dlna.Main.DlnaEntryPoint.ReloadComponents(CancellationToken cancellationToken)
       at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_0(Object state)
       at System.Threading.SynchronizationContext.<>c.<Post>b__8_0(ValueTuple`2 s)
       at System.Threading.QueueUserWorkItemCallbackDefaultContext`1.Execute()
       at System.Threading.ThreadPoolWorkQueue.Dispatch()
       at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
       at System.Threading.Thread.StartCallback()
    Source: System.Private.CoreLib
    TargetSite: System.Object ChkCastAny(Void*, System.Object)

UnhandledException
System.InvalidCastException: System.InvalidCastException: [A]Emby.Dlna.DlnaOptions cannot be cast to [B]Emby.Dlna.DlnaOptions. Type A originates from 'Emby.Dlna, Version=1.2.7.0, Culture=neutral, PublicKeyToken=null' in the context 'Default' in a byte array. Type B originates from 'Emby.Dlna, Version=1.2.7.0, Culture=neutral, PublicKeyToken=null' in the context 'Default' in a byte array.
   at MediaBrowser.Common.Configuration.ConfigurationManagerExtensions.GetConfiguration[T](IConfigurationManager manager, String key)
   at Emby.Dlna.ConfigurationExtension.GetDlnaConfiguration(IConfigurationManager manager)
   at Emby.Dlna.Main.DlnaEntryPoint.ReloadComponents(CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_0(Object state)
   at System.Threading.SynchronizationContext.<>c.<Post>b__8_0(ValueTuple`2 s)
   at System.Threading.QueueUserWorkItemCallbackDefaultContext`1.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback()
Source: System.Private.CoreLib
TargetSite: System.Object ChkCastAny(Void*, System.Object)

Info App: Entry point completed: Emby.Server.Implementations.Networking.RemoteAddressEntryPoint. Duration: 0.0040789 seconds
Info App: Starting entry point Emby.Server.Connect.ConnectEntryPoint
Info App: Loading data from /config/data/connect.txt
Info App: Entry point completed: Emby.Server.Connect.ConnectEntryPoint. Duration: 0.0013219 seconds
Info App: Core startup complete
Info App: Post-init migrations complete
Info App: Starting entry point Emby.Notifications.NotificationManager
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.
mikabytes commented 9 months ago

I managed to fix it first by deleting all files in config directory. I then restored them from backup and started deleting things randomly until the system started working again.

I managed to isolate the problem to plugins/ folder. These files were anyway re-downloaded on startup. Doesn't make much sense to me why that would work, as I tried backups from days and even months ago with no results. Everything was working yesterday.

I guess it triggered something unlikely which solved my problem.

LukePulverenti commented 9 months ago

Thanks for following up!