Shokofin 4 Plugin throws unhandled exception

[exorsize]

Jellyfin version.

10.9.1

Shokofin version.

4.0.0.0

Shoko Server version, release channel, and commit hash.

4.2.2 stable

File structure of your Media Library Folder in Jellyfin/Import Folder in Shoko Server.

../anime/Anime A/Season A/Episode.mkv

Screenshot of the "library settings" section of the plugin settings.

Library Type(s).

Shows

Do the issue persists after creating a library with Shoko set as the only metadata provider? (Now is your time to check if you haven't already.)

• [X] Yes, I hereby confirm that the issue persists after creating a library with Shoko set as the only metadata provider.

Issue

In trying to use the new 4.0 release of shokofin but it throws an exception. I removed the old v3 plugin and the old library built with version 3, rebooted jellyfin and then tried to add back a new library.

Stack Trace

from shokoserver log: shokoserver | 16:34:09| SentryMiddleware --- Sending event 'Sentry.SentryEvent' to Sentry. shokoserver | 16:34:09| SentryMiddleware --- Event '00000000000000000000000000000000' queued. shokoserver | 16:34:09| Kestrel --- Connection id "0HN4T16NPFC0I", Request id "0HN4T16NPFC0I:00000004": An unhandled exception was thrown by the application.

from jellyfin.log: [2024-07-05 18:34:09.909 +02:00] [ERR] Threw unexpectedly; "InternalServerErrorException" Shokofin.API.Models.ApiException: InternalServerErrorException at Shokofin.API.ShokoAPIClient.Get[ReturnType](String url, HttpMethod method, String apiKey, Boolean skipCache) at Shokofin.API.ShokoAPIClient.GetFilesForImportFolder(Int32 importFolderId, String subPath, Int32 page) at Shokofin.Resolvers.VirtualFileSystemService.GetFilesForImportFolder(IReadOnlyList1 mediaConfigs, HashSet1 fileSet)+MoveNext() at System.Linq.Enumerable.SelectEnumerableIterator2.MoveNext() at System.Threading.Tasks.Task.WhenAll(IEnumerable1 tasks) at Shokofin.Resolvers.VirtualFileSystemService.GenerateStructure(Nullable1 collectionType, String vfsPath, IEnumerable1 allFiles) at Shokofin.Resolvers.VirtualFileSystemService.<>cDisplayClass16_0.<b2>d.MoveNext() --- End of stack trace from previous location --- at Shokofin.Utils.GuardedMemoryCache.GetOrCreateAsync[TItem](Object key, Func1 createFactory, MemoryCacheEntryOptions createOptions) at Shokofin.Resolvers.VirtualFileSystemService.GenerateStructureInVFS(Folder mediaFolder, String path) at Shokofin.Resolvers.ShokoResolver.ResolveMultiple(Folder parent, Nullable1 collectionType, List1 fileInfoList) [2024-07-05 18:34:09.911 +02:00] [ERR] Error retrieving children folder Shokofin.API.Models.ApiException: InternalServerErrorException at Shokofin.API.ShokoAPIClient.Get[ReturnType](String url, HttpMethod method, String apiKey, Boolean skipCache) at Shokofin.API.ShokoAPIClient.GetFilesForImportFolder(Int32 importFolderId, String subPath, Int32 page) at Shokofin.Resolvers.VirtualFileSystemService.GetFilesForImportFolder(IReadOnlyList1 mediaConfigs, HashSet1 fileSet)+MoveNext() at System.Linq.Enumerable.SelectEnumerableIterator2.MoveNext() at System.Threading.Tasks.Task.WhenAll(IEnumerable1 tasks) at Shokofin.Resolvers.VirtualFileSystemService.GenerateStructure(Nullable1 collectionType, String vfsPath, IEnumerable`1 allFiles) at Shokofin.Resolvers.VirtualFileSystemService.<>cDisplayClass16_0.<b2>d.MoveNext() --- End of stack trace from previous location --- at Shokofin.Utils.GuardedMemoryCache.GetOrCreateAsync[TItem](Object key, Func1 createFactory, MemoryCacheEntryOptions createOptions) at Shokofin.Resolvers.VirtualFileSystemService.GenerateStructureInVFS(Folder mediaFolder, String path) at Shokofin.Resolvers.ShokoResolver.ResolveMultiple(Folder parent, Nullable1 collectionType, List1 fileInfoList) at Shokofin.Resolvers.ShokoResolver.MediaBrowser.Controller.Resolvers.IMultiItemResolver.ResolveMultiple(Folder parent, List1 files, Nullable1 collectionType, IDirectoryService directoryService) at Emby.Server.Implementations.Library.LibraryManager.ResolvePaths(IEnumerable1 files, IDirectoryService directoryService, Folder parent, LibraryOptions libraryOptions, Nullable1 collectionType, IItemResolver[] resolvers) at MediaBrowser.Controller.Entities.Folder.GetNonCachedChildren(IDirectoryService directoryService) at MediaBrowser.Controller.Entities.Folder.ValidateChildrenInternal2(IProgress1 progress, Boolean recursive, Boolean refreshChildMetadata, Boolean allowRemoveRoot, MetadataRefreshOptions refreshOptions, IDirectoryService directoryService, CancellationToken cancellationToken)

[revam]

1. don't self assign reviewers
2. there should be more in your shoko server log, since it threw in shoko server and not in shokofin, so either find the error yourself, or send a larger snippet of the log after it throws in jellyfin during a scan.
3. update to Jellyfin 10.9.7 for some critical bug fixes

[revam]

2. there should be more in your shoko server log, since it threw in shoko server and not in shokofin, so either find the error yourself, or send a larger snippet of the log after it throws in jellyfin during a scan.

You might need to enable trace logging for it to show up in the log file, but it should be visible in the docker log if you use docker.

[exorsize]

sry i didnt actively assign reviewers, was probably something in the submission form that did that

tnx for suggestions, will update jellyfin and search deeper in docker logs

[exorsize]

updated jellyfin to 10.9.7, problem persists.

found stacktrace in shoko server log:

[2024-07-05 18:35:31:751] Info|AsyncMethodBuilderCore.Start => <InvokeAsync>d__10.MoveNext => <>c__DisplayClass10_0.<InvokeAsync>g__CaptureException|4 Event '00000000000000000000000000000000' queued.
[2024-07-05 18:35:31:764] Error|AsyncStateMachineBox`1.ExecutionContextCallback => <ProcessRequests>d__226`1.MoveNext => HttpProtocol.ReportApplicationError Connection id "0HN4T44PTCHVM", Request id "0HN4T44PTCHVM:0>
   at Shoko.Server.API.v3.Controllers.TreeController.<>c.<GetFilesInImportFolder>b__0_2(SVR_VideoLocal file) in /usr/src/app/source/Shoko.Server/API/v3/Controllers/TreeController.cs:line 61
   at System.Linq.EnumerableSorter`2.ComputeKeys(TElement[] elements, Int32 count)
   at System.Linq.EnumerableSorter`1.ComputeMap(TElement[] elements, Int32 count)
   at System.Linq.OrderedEnumerable`1.GetEnumerator(Int32 minIdx, Int32 maxIdx)+MoveNext()
   at System.Linq.Enumerable.SelectIPartitionIterator`2.ToList()
   at Shoko.Server.API.v3.Helpers.ModelHelper.ToListResult[T,U](IEnumerable`1 enumerable, Func`2 mapper, Int32 page, Int32 pageSize) in /usr/src/app/source/Shoko.Server/API/v3/Helpers/ModelHelper.cs:line 55
   at Shoko.Server.API.v3.Controllers.TreeController.GetFilesInImportFolder(Int32 folderID, Int32 pageSize, Int32 page, Boolean includeXRefs) in /usr/src/app/source/Shoko.Server/API/v3/Controllers/TreeController.cs:>
   at lambda_method980(Closure , Object , Object[] )
   at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
   at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
   at Shoko.Server.API.APIExtensions.<>c.<<UseAPI>b__2_0>d.MoveNext() in /usr/src/app/source/Shoko.Server/API/APIExtensions.cs:line 208
--- End of stack trace from previous location ---
   at Sentry.AspNetCore.SentryMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
   at Sentry.AspNetCore.SentryMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
   at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.<>c__DisplayClass6_1.<<UseMiddlewareInterface>b__1>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

[revam]

updated jellyfin to 10.9.7, problem persists.

It was meant to fix other problems, not this problem.

found stacktrace in shoko server log:

It seems you have some invalid file locations in your database. Run the action to clean up missing files in the web ui or shoko desktop.

[revam]

Closing this issue as the above action should fix it.

Also a screenshot for where to run the action in the stable web ui;

[revam]

Feel free to reopen this issue if running the action (and making sure it completes) followed by a new scan/refresh doesn't fix the issue.

[revam]

It seems that didn't fix it. Will need to investigate how to properly fix it for the current stable server, but the issue seems to be that the server have some file locations without a valid file entry, so it errs out when trying to read the creation date on a null reference file, since it was unable to look up the file entry for the file location.

[exorsize]

Maybe superfluous at this point but wanted to do some more experiments before getting back to this, can however confirm that running the task did not fix the problem. Is there a good way to search for, and rectify these possible entries manually via ShokoDesktop or some other way in the meantime?

[revam]

I'll see if i can write some SQL tomorrow to fix a) identify if the error is what i think it is, and b) fix it. (so two SQL statements)

@exorsize do you know how to run SQL on a SQLite 3 database file or will you need instructions for how to run the SQL statements?

[exorsize]

The statements will suffice

[revam]

@ElementalCrisis it's a bug in Shoko Server, not in the plugin itself.

[revam]

First the check to make sure this is the actual issue;

SELECT COUNT(*) FROM VideoLocal_Place vlp WHERE NOT EXISTS (SELECT 1 FROM VideoLocal vl WHERE vlp.VideoLocalID = vl.VideoLocalID);

If the count is above zero then there is an easy way to let Shoko Server fix the issue, instead of trying manually editing the database; Just press the "scan" icon on all the import folders with files in them and let it settle. Edit: If you know the import folder that's erring, then you only need to press "scan" on that particular folder.

@exorsize can you post the output of the above SQL when ran against your database?

[exorsize]

The count was above zero, returned 8. After running scans and seeing 8 new episodes added in shokoweb log and confirming via SQL that the count was 0 I added back my library in Jellyfin and it added without problems.

Thank you for the help.