search

crazy-max / docker-rtorrent-rutorrent

rTorrent and ruTorrent Docker image
MIT License
457 stars 103 forks source link

500 error when upload file greater than 1M via rpc.php #245

Open L3o-pold opened 1 year ago

L3o-pold commented 1 year ago

Support guidelines

I've found a bug and checked that ...

Description

When uploading a torrent greater than 1M on /plugins/rpc/rpc.php a 500 HTTP header occurs. I found no interesting error log to investigate. I already increased XMLRPC_SIZE_LIMIT to 16M and MEMORY_LIMIT to 2048M without success.

rutorrent-6db6c7bb8c-slkzs rutorrent 2023/05/22 15:37:56 [warn] 552#552: *681 a client request body is buffered to a temporary file /tmp/nginx/8/00/0000000008, client: 10.0.0.128, server: , request: "POST /plugins/rpc/rpc.php HTTP/1.1", host: "192.168.1.200:31705"
rutorrent-6db6c7bb8c-slkzs rutorrent 10.0.0.128 - - [22/May/2023:15:37:56 +0000] "POST /plugins/rpc/rpc.php HTTP/1.1" 500 48 "-" "Sonarr/4.0.0.500 (alpine 3.16.5)"
rutorrent-6db6c7bb8c-slkzs rutorrent 2023/05/22 15:37:59 [info] 552#552: *684 client closed connection while waiting for request, client: 10.0.0.128, server: 0.0.0.0:8080
rutorrent-6db6c7bb8c-slkzs rutorrent 2023/05/22 15:37:59 [info] 552#552: *683 client closed connection while waiting for request, client: 10.0.0.128, server: 0.0.0.0:8080

Expected behaviour

Should return an 200 http

Actual behaviour

500 error code

Steps to reproduce

  1. Upload a large torrent to /plugins/rpc/rpc.php
  2. see the 500 error

Docker info

Kubernetes

Version

Docker compose

Kubernetes

Container logs

rutorrent-6db6c7bb8c-slkzs rutorrent 2023/05/22 15:37:56 [warn] 552#552: *681 a client request body is buffered to a temporary file /tmp/nginx/8/00/0000000008, client: 10.0.0.128, server: , request: "POST /plugins/rpc/rpc.php HTTP/1.1", host: "192.168.1.200:31705"
rutorrent-6db6c7bb8c-slkzs rutorrent 10.0.0.128 - - [22/May/2023:15:37:56 +0000] "POST /plugins/rpc/rpc.php HTTP/1.1" 500 48 "-" "Sonarr/4.0.0.500 (alpine 3.16.5)"
rutorrent-6db6c7bb8c-slkzs rutorrent 2023/05/22 15:37:59 [info] 552#552: *684 client closed connection while waiting for request, client: 10.0.0.128, server: 0.0.0.0:8080
rutorrent-6db6c7bb8c-slkzs rutorrent 2023/05/22 15:37:59 [info] 552#552: *683 client closed connection while waiting for request, client: 10.0.0.128, server: 0.0.0.0:8080

Additional info

Using 4.1.5-0.9.8-0.13.8 docker image

f0oster commented 1 year ago

Seeing the same behavior with the latest image.

Attempting a 1,715KB .torrent file upload via RPC.php triggers HTTP 500 response.

Also manually uploading .torrent via rutorrent interface fails with response HTTP 413 Entity Too Large, which appears to be due to nginx configuration. Possibly a hint as to what's happening?

Even with XMLRPC logging enabled, seemingly nothing insightful is logged.

stickz commented 1 year ago

Could you try setting RU_REMOVE_CORE_PLUGINS to rpc? This will use httprpc instead.

L3o-pold commented 1 year ago

@stickz it's not working 10.0.0.128 - - [23/May/2023:07:19:37 +0000] "POST /plugins/rpc/rpc.php HTTP/1.1" 404 146 "-" "Sonarr/4.0.0.500 (alpine 3.16.5)"

L3o-pold commented 1 year ago

@f0oster for me it's working from the rutorrent interface. Do you have an nginx ingress in front?

f0oster commented 1 year ago

@f0oster for me it's working from the rutorrent interface. Do you have an nginx ingress in front?

Good point... I do have an nginx reverse proxy configured which I was likely using during testing, whoops.

I'll test and confirm shortly.

Edit:

Yep, confirming that it does work through the rutorrent interface when I bypass my reverse proxy. Sorry for the false alarm.

stickz commented 1 year ago

@stickz it's not working 10.0.0.128 - - [23/May/2023:07:19:37 +0000] "POST /plugins/rpc/rpc.php HTTP/1.1" 404 146 "-" "Sonarr/4.0.0.500 (alpine 3.16.5)"

You may need to update your configuration to try httprpc. It looks like you're running sonarr. It should POST to /plugins/httprpc/action.php instead. The rpc plugin must be disabled to use httprpc.

httprpc works differently and may fix the issue. The response will be encoded in JSON instead of XML. It will be processed internally using rutorrent. Nginx will be skipped in favour of PHP and an xmlrpc request will be sent directly to rtorrent.

It's actually faster to do this, if you have the CPU processing resources. There is limited situations (not yours) where it creates too much overhead. Sonarr has much more overhead than this task. If you can run sonarr, this will work great.

L3o-pold commented 1 year ago

@stickz I don't think Sonarr works with httprpc.

stickz commented 1 year ago

@stickz I don't think Sonarr works with httprpc.

It does. You just need to change the url base path. ruTorrent will accept the XML response from Sonarr through httprpc. https://forums.sonarr.tv/t/rtorrent-support/3644/22

L3o-pold commented 1 year ago

rutorrent returns a 200 HTTP code, but Sonarr fails

rutorrent-f68bf68f4-6xtxp rutorrent 10.0.0.128 - - [24/May/2023:13:32:14 +0000] "POST /plugins/httprpc/action.php HTTP/1.1" 200 14151 "-" "Sonarr/4.0.0.500 (alpine 3.16.5)"

{
  "message": "Data at the root level is invalid. Line 1, position 1.",
  "description": "System.Xml.XmlException: Data at the root level is invalid. Line 1, position 1.\n   at System.Xml.XmlTextReaderImpl.Throw(Exception e)\n   at System.Xml.XmlTextReaderImpl.Throw(String res, String arg)\n   at System.Xml.XmlTextReaderImpl.ParseRootLevelWhitespace()\n   at System.Xml.XmlTextReaderImpl.ParseDocumentContent()\n   at System.Xml.Linq.XDocument.Load(XmlReader reader, LoadOptions options)\n   at System.Xml.Linq.XDocument.Parse(String text, LoadOptions options)\n   at NzbDrone.Core.Download.Clients.RTorrent.RTorrentProxy.ExecuteRequest(RTorrentSettings settings, String methodName, Object[] args) in C:\\BuildAgent\\work\\13f3e374fa512e16\\src\\NzbDrone.Core\\Download\\Clients\\rTorrent\\RTorrentProxy.cs:line 212\n   at NzbDrone.Core.Download.Clients.RTorrent.RTorrentProxy.AddTorrentFromFile(String fileName, Byte[] fileContent, String label, RTorrentPriority priority, String directory, RTorrentSettings settings) in C:\\BuildAgent\\work\\13f3e374fa512e16\\src\\NzbDrone.Core\\Download\\Clients\\rTorrent\\RTorrentProxy.cs:line 105\n   at NzbDrone.Core.Download.Clients.RTorrent.RTorrent.AddFromTorrentFile(RemoteEpisode remoteEpisode, String hash, String filename, Byte[] fileContent) in C:\\BuildAgent\\work\\13f3e374fa512e16\\src\\NzbDrone.Core\\Download\\Clients\\rTorrent\\RTorrent.cs:line 102\n   at NzbDrone.Core.Download.TorrentClientBase\u00601.DownloadFromWebUrl(RemoteEpisode remoteEpisode, IIndexer indexer, String torrentUrl) in C:\\BuildAgent\\work\\13f3e374fa512e16\\src\\NzbDrone.Core\\Download\\TorrentClientBase.cs:line 192\n   at NzbDrone.Core.Download.TorrentClientBase\u00601.Download(RemoteEpisode remoteEpisode, IIndexer indexer) in C:\\BuildAgent\\work\\13f3e374fa512e16\\src\\NzbDrone.Core\\Download\\TorrentClientBase.cs:line 117\n   at NzbDrone.Core.Download.DownloadService.DownloadReport(RemoteEpisode remoteEpisode, IDownloadClient downloadClient) in C:\\BuildAgent\\work\\13f3e374fa512e16\\src\\NzbDrone.Core\\Download\\DownloadService.cs:line 92\n   at NzbDrone.Core.Download.DownloadService.DownloadReport(RemoteEpisode remoteEpisode, Nullable\u00601 downloadClientId) in C:\\BuildAgent\\work\\13f3e374fa512e16\\src\\NzbDrone.Core\\Download\\DownloadService.cs:line 57\n   at Sonarr.Api.V3.Indexers.ReleaseController.DownloadRelease(ReleaseResource release) in C:\\BuildAgent\\work\\13f3e374fa512e16\\src\\Sonarr.Api.V3\\Indexers\\ReleaseController.cs:line 159\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State\u0026 next, Scope\u0026 scope, Object\u0026 state, Boolean\u0026 isCompleted)\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()\n--- End of stack trace from previous location ---\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State\u0026 next, Scope\u0026 scope, Object\u0026 state, Boolean\u0026 isCompleted)\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()\n--- End of stack trace from previous location ---\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.\u003CInvokeNextResourceFilter\u003Eg__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State\u0026 next, Scope\u0026 scope, Object\u0026 state, Boolean\u0026 isCompleted)\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()\n--- End of stack trace from previous location ---\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.\u003CInvokeAsync\u003Eg__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)\n   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.\u003CInvokeAsync\u003Eg__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)\n   at Microsoft.AspNetCore.Routing.EndpointMiddleware.\u003CInvoke\u003Eg__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)\n   at Sonarr.Http.Middleware.BufferingMiddleware.InvokeAsync(HttpContext context) in C:\\BuildAgent\\work\\13f3e374fa512e16\\src\\Sonarr.Http\\Middleware\\BufferingMiddleware.cs:line 28\n   at Sonarr.Http.Middleware.IfModifiedMiddleware.InvokeAsync(HttpContext context) in C:\\BuildAgent\\work\\13f3e374fa512e16\\src\\Sonarr.Http\\Middleware\\IfModifiedMiddleware.cs:line 41\n   at Sonarr.Http.Middleware.CacheHeaderMiddleware.InvokeAsync(HttpContext context) in C:\\BuildAgent\\work\\13f3e374fa512e16\\src\\Sonarr.Http\\Middleware\\CacheHeaderMiddleware.cs:line 33\n   at Sonarr.Http.Middleware.StartingUpMiddleware.InvokeAsync(HttpContext context) in C:\\BuildAgent\\work\\13f3e374fa512e16\\src\\Sonarr.Http\\Middleware\\StartingUpMiddleware.cs:line 38\n   at Sonarr.Http.Middleware.UrlBaseMiddleware.InvokeAsync(HttpContext context) in C:\\BuildAgent\\work\\13f3e374fa512e16\\src\\Sonarr.Http\\Middleware\\UrlBaseMiddleware.cs:line 27\n   at Sonarr.Http.Middleware.VersionMiddleware.InvokeAsync(HttpContext context) in C:\\BuildAgent\\work\\13f3e374fa512e16\\src\\Sonarr.Http\\Middleware\\VersionMiddleware.cs:line 29\n   at Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware.InvokeCore(HttpContext context)\n   at Microsoft.AspNetCore.Authorization.Policy.AuthorizationMiddlewareResultHandler.HandleAsync(RequestDelegate next, HttpContext context, AuthorizationPolicy policy, PolicyAuthorizationResult authorizeResult)\n   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)\n   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)\n   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.\u003CInvoke\u003Eg__Awaited|6_0(ExceptionHandlerMiddleware middleware, HttpContext context, Task task)"
}
stickz commented 1 year ago

rutorrent returns a 200 HTTP code, but Sonarr fails

You can try to talk directly to rTorrent on Sonarr through the URL path RPC2 or /RPC2 on port 8000. Then use httprpc for ruTorrent. This is a better configuration because ruTorrent won't be required to function properly. This docker image exposes that.

L3o-pold commented 1 year ago

(port 31299 is forwarded to 8000)

Sonarr test works (as before with httprcp) 

rutorrent-6866c47d4-qvjgm rutorrent 10.0.0.128 - - [24/May/2023:14:14:45 +0000] "POST /RPC2 HTTP/1.1" 200 14151 "-" "Sonarr/4.0.0.500 (alpine 3.16.5)"

but not uploading a torrent

rutorrent-6866c47d4-qvjgm rutorrent 2023/05/24 14:12:48 [warn] 549#549: *52 a client request body is buffered to a temporary file /tmp/nginx/2/00/0000000002, client: 10.0.0.128, server: , request: "POST /RPC2 HTTP/1.1", host: "192.168.1.200:31299"
rutorrent-6866c47d4-qvjgm rutorrent 2023/05/24 14:12:48 [error] 549#549: *52 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.128, server: , request: "POST /RPC2 HTTP/1.1", upstream: "scgi://unix:/var/run/rtorrent/scgi.socket:", host: "192.168.1.200:31299"
rutorrent-6866c47d4-qvjgm rutorrent 10.0.0.128 - - [24/May/2023:14:12:48 +0000] "POST /RPC2 HTTP/1.1" 502 150 "-" "Sonarr/4.0.0.500 (alpine 3.16.5)"

8000 port seems to be the XMLRPC_PORT

stickz commented 1 year ago

When switching Sonarr to /RPC2 you received the same error, but a more detailed a 502 instead of 500 error code. I can't be of any further assistance. I would recommend to continue using /RPC2 because it takes ruTorrent out of the picture.