Server fails - onnxruntime

[Ace018]

Hey,

been trying to set up the server on my machine. It starts running, but once it gets an image to censor, it returns a failure.

I'm using macOS 12.4 with Apple Silicone (arm) processor. Downloaded beta-censoring server version 0.0.7 and using Chrome v104.0.5112.79 with beta-protection extension v0.1.5

I run the BetaCensor.Server file in terminal after giving it execute permissions and it starts fine. But return the following error:

fail: BetaCensor.Workers.DispatchWorkerService[0]
      Error occurred executing task work item.
      System.InvalidOperationException: Error constructing handler for request of type MediatR.IRequestHandler`2[BetaCensor.Core.Messaging.CensorImageRequest,BetaCensor.Core.Messaging.CensorImageResponse]. Register your handlers with the container. See the samples in GitHub for examples.
       ---> System.TypeInitializationException: The type initializer for 'Microsoft.ML.OnnxRuntime.NativeMethods' threw an exception.
       ---> System.DllNotFoundException: Unable to load shared library 'onnxruntime' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(libonnxruntime, 0x0001): tried: 'libonnxruntime' (no such file), '/usr/local/lib/libonnxruntime' (no such file), '/usr/lib/libonnxruntime' (no such file), '/Users/username/Documents/username2022/Coding/Beta/betacensor-server-osx-x64/libonnxruntime' (no such file)
         at Microsoft.ML.OnnxRuntime.NativeMethods.OrtGetApiBase()
         at Microsoft.ML.OnnxRuntime.NativeMethods..cctor()
         --- End of inner exception stack trace ---
         at Microsoft.ML.OnnxRuntime.SessionOptions..ctor()
         at CensorCore.BodyAreaImageHandler..ctor(IImageHandler implHandler, OptimizationMode mode) in /home/runner/work/censor-core/censor-core/src/CensorCore/BodyAreaImageHandler.cs:line 22
         at BetaCensor.Server.ServerConfigurationExtensions.<>c__DisplayClass2_0.<BuildImageHandler>b__1(IServiceProvider p) in /home/runner/work/beta-censoring/beta-censoring/src/BetaCensor.Server/ConfigurationExtensions.cs:line 55
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactory(FactoryCallSite , RuntimeResolverContext )
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite , TArgument )
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite , RuntimeResolverContext )
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite , TArgument )
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
         at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type )
         at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey , Func`2 )
         at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type , ServiceProviderEngineScope )
         at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
         at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
         at BetaCensor.Workers.ServicesExtensions.<>c__DisplayClass4_0.<AddCensoring>b__1(IServiceProvider p) in /home/runner/work/beta-censoring/beta-censoring/src/BetaCensor.Workers/ServicesExtensions.cs:line 47
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactory(FactoryCallSite , RuntimeResolverContext )
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite , TArgument )
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite , RuntimeResolverContext )
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite , TArgument )
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite , RuntimeResolverContext )
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite , TArgument )
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitDisposeCache(ServiceCallSite , RuntimeResolverContext )
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite , TArgument )
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass2_0.<RealizeService>b__0(ServiceProviderEngineScope )
         at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type , ServiceProviderEngineScope )
         at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
         at MediatR.ServiceFactoryExtensions.GetInstance[T](ServiceFactory factory)
         at MediatR.Wrappers.HandlerBase.GetHandler[THandler](ServiceFactory factory)
         --- End of inner exception stack trace ---
         at MediatR.Wrappers.HandlerBase.GetHandler[THandler](ServiceFactory factory)
         at MediatR.Wrappers.RequestHandlerWrapperImpl`2.<>c__DisplayClass1_0.<Handle>g__Handler|0()
         at MediatR.Pipeline.RequestExceptionProcessorBehavior`2.Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate`1 next)
         at MediatR.Pipeline.RequestExceptionProcessorBehavior`2.Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate`1 next)
         at MediatR.Pipeline.RequestExceptionActionProcessorBehavior`2.Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate`1 next)
         at MediatR.Pipeline.RequestExceptionActionProcessorBehavior`2.Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate`1 next)
         at MediatR.Pipeline.RequestPostProcessorBehavior`2.Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate`1 next)
         at MediatR.Pipeline.RequestPreProcessorBehavior`2.Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate`1 next)
         at BetaCensor.Workers.DispatchWorkerService`2.ProcessTaskQueueAsync(CancellationToken stoppingToken)

That's the first 'fail'. The whole terminal output is in error_beta.txt. I only changed the configuration files by changing workerCount to 8. But no matter on that setting, the result is the same.

I did notice that at localhost:2382 settings view, it says CensoreCore Version: v0.0.0. Is that supposed to be 0?

Thank you for your help

[silveredgold]

Thanks for the super-detailed error report! Looks like there's issues with the runtime loading on Apple platforms, so I will need to look into that.

The issue is there's no way to test since I don't have a Mac. I'll see what I can do.

[Ace018]

Let me know, if you need help testing it on apple device

[silveredgold]

@Ace018 Can you try with the latest CI build? You can download it from this link, then extract it. Ignore the 10.14 in the file name, that's just the minimum version.

I've included both an x64 and an arm64 (i.e. M1-specific) builds but as I've never worked with M1, I don't know if that will work at all. If you can, can you try running both (just not at the same time) and let me know if they work?

[Ace018]

I tried both builds. The arm64 one doesn't execute. It only returns killed signal. I'm seeing that for the first time, so I'm more or less clueless what could be wrong. Screenshot is below.

The x64 build seems to execute well. Although, using rosetta it is kinda heavier for the computer, but at least works on arm mac now. Let me know if you need any further "help".

[silveredgold]

Awesome, thanks for the update. I'll look into the M1 thing since I haven't had to build for ARM much before so I might be missing something there. Thanks for confirming that x64 works at least!

[silveredgold]

Turns out that's another fun Apple-specific quirk: M1/M2 Macs will just kill anything that isn't signed and notarized 🤦

I'm going to close this and track that issue separately (just for clarity). Thanks so much for the assistance with this @Ace018 !