already added remote mappings are not detected and config fails to apply

scratchmex commented 9 months ago

the first time buildarr is ran it adds the mappings correctly but any consequent runs will fail with the following. the temporal fix is to comment the config entry or remove the mapping from sonarr itself everytime

# buildarr.yml
      remote_path_mappings:
        definitions:
          - host: qbittorrent
            remote_path: "/downloads"
            local_path: "/data/downloads"

buildarr  | 2024-02-05 22:36:09,567 buildarr:1 buildarr.cli.run [INFO] <sonarr> (default) Updating remote configuration
buildarr  | 2024-02-05 22:36:09,580 buildarr:1 buildarr_sonarr.config.download_clients.remote_path_mappings [INFO] <sonarr> (default) sonarr.settings.download_clients.remote_path_mappings.definitions[0]: RemotePathMapping(host='qbittorrent', remote_path='/downloads', local_path='/data/downloads', ensure=<Ensure.present: 'present'>) -> (created)
buildarr  | 2024-02-05 22:36:09,580 buildarr:1 buildarr.config.base [INFO] <sonarr> (default) sonarr.settings.download_clients.remote_path_mappings.definitions[0].host: 'qbittorrent' -> (created)
buildarr  | 2024-02-05 22:36:09,580 buildarr:1 buildarr.config.base [INFO] <sonarr> (default) sonarr.settings.download_clients.remote_path_mappings.definitions[0].remote_path: '/downloads' -> (created)
buildarr  | 2024-02-05 22:36:09,580 buildarr:1 buildarr.config.base [INFO] <sonarr> (default) sonarr.settings.download_clients.remote_path_mappings.definitions[0].local_path: '/data/downloads' -> (created)
buildarr  | 2024-02-05 22:36:09,587 buildarr:1 buildarr.cli.daemon [ERROR] <sonarr> (default) An error occurred while applying initial configuration: Unexpected response with status code 500 from 'POST http://sonarr:8989/api/v3/remotepathmapping': RemotePath already configured.
buildarr  | System.InvalidOperationException: RemotePath already configured.
buildarr  |    at NzbDrone.Core.RemotePathMappings.RemotePathMappingService.ValidateMapping(List`1 existing, RemotePathMapping mapping) in ./Sonarr.Core/RemotePathMappings/RemotePathMappingService.cs:line 119
buildarr  |    at NzbDrone.Core.RemotePathMappings.RemotePathMappingService.Add(RemotePathMapping mapping) in ./Sonarr.Core/RemotePathMappings/RemotePathMappingService.cs:line 58
buildarr  |    at Sonarr.Api.V3.RemotePathMappings.RemotePathMappingController.CreateMapping(RemotePathMappingResource resource) in ./Sonarr.Api.V3/RemotePathMappings/RemotePathMappingController.cs:line 48
buildarr  |    at lambda_method189(Closure , Object , Object[] )
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
buildarr  | --- End of stack trace from previous location ---
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
buildarr  | --- End of stack trace from previous location ---
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
buildarr  | --- End of stack trace from previous location ---
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
buildarr  |    at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
buildarr  |    at Sonarr.Http.Middleware.BufferingMiddleware.InvokeAsync(HttpContext context) in ./Sonarr.Http/Middleware/BufferingMiddleware.cs:line 28
buildarr  |    at Sonarr.Http.Middleware.IfModifiedMiddleware.InvokeAsync(HttpContext context) in ./Sonarr.Http/Middleware/IfModifiedMiddleware.cs:line 41
buildarr  |    at Sonarr.Http.Middleware.CacheHeaderMiddleware.InvokeAsync(HttpContext context) in ./Sonarr.Http/Middleware/CacheHeaderMiddleware.cs:line 33
buildarr  |    at Sonarr.Http.Middleware.StartingUpMiddleware.InvokeAsync(HttpContext context) in ./Sonarr.Http/Middleware/StartingUpMiddleware.cs:line 38
buildarr  |    at Sonarr.Http.Middleware.UrlBaseMiddleware.InvokeAsync(HttpContext context) in ./Sonarr.Http/Middleware/UrlBaseMiddleware.cs:line 27
buildarr  |    at Sonarr.Http.Middleware.VersionMiddleware.InvokeAsync(HttpContext context) in ./Sonarr.Http/Middleware/VersionMiddleware.cs:line 29
buildarr  |    at Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware.InvokeCore(HttpContext context)
buildarr  |    at Microsoft.AspNetCore.Authorization.Policy.AuthorizationMiddlewareResultHandler.HandleAsync(RequestDelegate next, HttpContext context, AuthorizationPolicy policy, PolicyAuthorizationResult authorizeResult)
buildarr  |    at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
buildarr  |    at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
buildarr  |    at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.<Invoke>g__Awaited|6_0(ExceptionHandlerMiddleware middleware, HttpContext context, Task task)
buildarr  | Traceback (most recent call last):
buildarr  |   File "/usr/local/lib/python3.11/site-packages/buildarr/cli/daemon.py", line 215, in _initial_run
buildarr  |     run_apply()
buildarr  |   File "/usr/local/lib/python3.11/site-packages/buildarr/cli/run.py", line 248, in _run
buildarr  |     if manager.update_remote(
buildarr  |        ^^^^^^^^^^^^^^^^^^^^^^
buildarr  |   File "/usr/local/lib/python3.11/site-packages/buildarr/manager/__init__.py", line 192, in update_remote
buildarr  |     return local_instance_config.update_remote(
buildarr  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
buildarr  |   File "/usr/local/lib/python3.11/site-packages/buildarr/config/base.py", line 272, in update_remote
buildarr  |     if isinstance(field, ConfigBase) and field.update_remote(
buildarr  |                                          ^^^^^^^^^^^^^^^^^^^^
buildarr  |   File "/usr/local/lib/python3.11/site-packages/buildarr_sonarr/config/__init__.py", line 98, in update_remote
buildarr  |     self.download_clients.update_remote(
buildarr  |   File "/usr/local/lib/python3.11/site-packages/buildarr_sonarr/config/download_clients/__init__.py", line 204, in update_remote
buildarr  |     rpms_updated = self.remote_path_mappings._update_remote(
buildarr  |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
buildarr  |   File "/usr/local/lib/python3.11/site-packages/buildarr_sonarr/config/download_clients/remote_path_mappings.py", line 216, in _update_remote
buildarr  |     rpm._create_remote(tree=rpm_tree, secrets=secrets)
buildarr  |   File "/usr/local/lib/python3.11/site-packages/buildarr_sonarr/config/download_clients/remote_path_mappings.py", line 96, in _create_remote
buildarr  |     api_post(
buildarr  |   File "/usr/local/lib/python3.11/site-packages/buildarr_sonarr/api.py", line 188, in api_post
buildarr  |     api_error(method="POST", url=url, response=res)
buildarr  |   File "/usr/local/lib/python3.11/site-packages/buildarr_sonarr/api.py", line 313, in api_error
buildarr  |     raise SonarrAPIError(error_message, status_code=response.status_code)
buildarr  | buildarr_sonarr.exceptions.SonarrAPIError: Unexpected response with status code 500 from 'POST http://sonarr:8989/api/v3/remotepathmapping': RemotePath already configured.
buildarr  | System.InvalidOperationException: RemotePath already configured.
buildarr  |    at NzbDrone.Core.RemotePathMappings.RemotePathMappingService.ValidateMapping(List`1 existing, RemotePathMapping mapping) in ./Sonarr.Core/RemotePathMappings/RemotePathMappingService.cs:line 119
buildarr  |    at NzbDrone.Core.RemotePathMappings.RemotePathMappingService.Add(RemotePathMapping mapping) in ./Sonarr.Core/RemotePathMappings/RemotePathMappingService.cs:line 58
buildarr  |    at Sonarr.Api.V3.RemotePathMappings.RemotePathMappingController.CreateMapping(RemotePathMappingResource resource) in ./Sonarr.Api.V3/RemotePathMappings/RemotePathMappingController.cs:line 48
buildarr  |    at lambda_method189(Closure , Object , Object[] )
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
buildarr  | --- End of stack trace from previous location ---
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
buildarr  | --- End of stack trace from previous location ---
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
buildarr  | --- End of stack trace from previous location ---
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
buildarr  |    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
buildarr  |    at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
buildarr  |    at Sonarr.Http.Middleware.BufferingMiddleware.InvokeAsync(HttpContext context) in ./Sonarr.Http/Middleware/BufferingMiddleware.cs:line 28
buildarr  |    at Sonarr.Http.Middleware.IfModifiedMiddleware.InvokeAsync(HttpContext context) in ./Sonarr.Http/Middleware/IfModifiedMiddleware.cs:line 41
buildarr  |    at Sonarr.Http.Middleware.CacheHeaderMiddleware.InvokeAsync(HttpContext context) in ./Sonarr.Http/Middleware/CacheHeaderMiddleware.cs:line 33
buildarr  |    at Sonarr.Http.Middleware.StartingUpMiddleware.InvokeAsync(HttpContext context) in ./Sonarr.Http/Middleware/StartingUpMiddleware.cs:line 38
buildarr  |    at Sonarr.Http.Middleware.UrlBaseMiddleware.InvokeAsync(HttpContext context) in ./Sonarr.Http/Middleware/UrlBaseMiddleware.cs:line 27
buildarr  |    at Sonarr.Http.Middleware.VersionMiddleware.InvokeAsync(HttpContext context) in ./Sonarr.Http/Middleware/VersionMiddleware.cs:line 29
buildarr  |    at Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware.InvokeCore(HttpContext context)
buildarr  |    at Microsoft.AspNetCore.Authorization.Policy.AuthorizationMiddlewareResultHandler.HandleAsync(RequestDelegate next, HttpContext context, AuthorizationPolicy policy, PolicyAuthorizationResult authorizeResult)
buildarr  |    at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
buildarr  |    at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
buildarr  |    at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.<Invoke>g__Awaited|6_0(ExceptionHandlerMiddleware middleware, HttpContext context, Task task)
b

Callum027 commented 8 months ago

Hi there, thank you for the bug report, and I apologise for the late reply (I was on break for a while).

Looking at the logs, it's clear that Buildarr thinks the remote path mapping does not exist on the remote instance. There might be an issue with how it fetches them from the remote instance, or how it parses the received list. I'll have a look, and prepare a patch for the next minor release.

Callum027 commented 8 months ago

The cause was that Sonarr always adds a trailing slash (/ for POSIX paths, \ for Windows paths) to the local and remote paths of remote path mappings. If the corresponding values in the Buildarr configuration did not have these trailing slashes, then Buildarr would not detect that the remote path mapping already exists, and will always try to create it.

I have just merged a PR resolving this issue, which will be included in the next minor release, hopefully within the next day or two.

Callum027 commented 8 months ago

Thanks for waiting. The fix for this issue has been released as:

Docker Container: callum027/buildarr:0.7.6 (or update your local container if using latest)
Windows Installer: buildarr-installer-0.7.6-x64.exe
Python Package: buildarr-sonarr==0.6.4

Please give the new version a try to check that it resolves your issue.

Callum027 commented 6 months ago

As some time has passed since this has been addressed, I will now close this issue.

Let me know if you have any more problems with either this issue, or using Buildarr in general.

buildarr / buildarr-sonarr

already added remote mappings are not detected and config fails to apply #49