search

rommapp / romm

A beautiful, powerful, self-hosted rom manager
https://romm.app
GNU Affero General Public License v3.0
2.52k stars 103 forks source link

[Bug] Uploads not working #1193

Open Crimson7806 opened 2 months ago

Crimson7806 commented 2 months ago

RomM version

3.5.1

Describe the bug

I cannot get uploads to work on non local connections, whether I am at work or using the guest network at home

To Reproduce

I made sure I was not connected locally to the host server in any way, I used my domain name I have setup via reverse proxy, and also tried my WAN IP with my port (yes I am port forwarded) and both pages loaded, but uploading doesn't work, it starts the upload, and cuts off seemingly randomly and says " All files skipped, nothing to upload." with an X mark on the file upload status. occasionally it says succeeded with a check mark, but the files do not appear to actually be on the server, and it still seemingly fails. nothing in the logs indicate any sort of status on uploads or downloads. I am not sure if the fact I am running on unraid changes anything. File formats do not seem to matter or affect it either, they all fail.

Expected behavior

Hopefully logs on Unraid would allow us to see a more in depth debugging on uploads and downloads, as there is no way to figure out if it is a client issue, server issue, or RomM issue. Myself and another user in the discord uploaded files to each other's servers, and no luck for either of us. Clearer debugging for uploads would be a big plus

Screenshots image

{BF8E6455-5D6C-403C-BCBE-2943B296C4C2}

Desktop (please complete the following information):

gantoine commented 2 months ago

can you post the container logs? either from the unraid UI or using the terminal with docker logs romm.

MightyDjinn commented 2 months ago

This is likely an issue with the “proxy-body-size” on your reverse proxy. Check the developer console on your browser and see if any of those “x” contain a HTTP 413 response.

gantoine commented 2 months ago

we use an internal nginx proxy as well, and we just merged a fix for proxy limits https://github.com/rommapp/romm/pull/1201

MightyDjinn commented 2 months ago

I was speaking to the proxy that @Crimson7806 has sitting in front of the app. Sorry for the confusion!

gantoine commented 2 months ago

ah no confusion at all! i was just mentioning that we have an internal proxy too that deals with proxy limits on uploads

Crimson7806 commented 1 month ago

Nothing shows in the logs or the developer console relating to uploads failing or anything, I also tried a cloudflare tunnel and I got the same results

MessiasLima commented 1 month ago

Im running locally and Im having the same

MessiasLima commented 1 month ago 
10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.0.1 Safari/605.1.15" rt=0.000 uct="-" uht="-" urt="-"
big-bear-romm    | INFO:    [RomM][2024-10-28 08:27:48] Detected platform: gbc
big-bear-romm    | INFO:    [RomM][2024-10-28 08:27:48] · gbc
big-bear-romm    | WARNING:  [RomM][2024-10-28 08:27:48] Twitch token invalid: fetching a new one...
big-bear-romm    | CRITICAL: [RomM][2024-10-28 08:27:48] IGDB Error: Invalid IGDB_CLIENT_ID or IGDB_CLIENT_SECRET
big-bear-romm    | 192.168.1.116:0 - "POST /api/platforms HTTP/1.0" 500
big-bear-romm    | INFO:      [nginx][2024-10-28 08:27:49]  192.168.1.116 - - "POST /api/platforms HTTP/1.1" 500 21 "http://redvelvet:1080/scan" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.0.1 Safari/605.1.15" rt=1.067 uct="0.000" uht="1.067" urt="1.067"
big-bear-romm    | [2024-10-28 08:27:49 +0000] [21] [ERROR] Exception in ASGI application
big-bear-romm    | Traceback (most recent call last):
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 69, in map_httpcore_exceptions
big-bear-romm    |     yield
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 373, in handle_async_request
big-bear-romm    |     resp = await self._pool.handle_async_request(req)
big-bear-romm    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request
big-bear-romm    |     raise exc from None
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request
big-bear-romm    |     response = await connection.handle_async_request(
big-bear-romm    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpcore/_async/connection.py", line 101, in handle_async_request
big-bear-romm    |     return await self._connection.handle_async_request(request)
big-bear-romm    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpcore/_async/http11.py", line 143, in handle_async_request
big-bear-romm    |     raise exc
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpcore/_async/http11.py", line 93, in handle_async_request
big-bear-romm    |     await self._send_request_headers(**kwargs)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpcore/_async/http11.py", line 151, in _send_request_headers
big-bear-romm    |     with map_exceptions({h11.LocalProtocolError: LocalProtocolError}):
big-bear-romm    |   File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__
big-bear-romm    |     self.gen.throw(value)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
big-bear-romm    |     raise to_exc(exc) from exc
big-bear-romm    | httpcore.LocalProtocolError: Illegal header value b'Bearer '
big-bear-romm    | 
big-bear-romm    | The above exception was the direct cause of the following exception:
big-bear-romm    | 
big-bear-romm    | Traceback (most recent call last):
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 407, in run_asgi
big-bear-romm    |     result = await app(  # type: ignore[func-returns-value]
big-bear-romm    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
big-bear-romm    |     return await self.app(scope, receive, send)
big-bear-romm    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
big-bear-romm    |     await super().__call__(scope, receive, send)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/applications.py", line 123, in __call__
big-bear-romm    |     await self.middleware_stack(scope, receive, send)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 186, in __call__
big-bear-romm    |     raise exc
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in __call__
big-bear-romm    |     await self.app(scope, receive, _send)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/middleware/base.py", line 189, in __call__
big-bear-romm    |     with collapse_excgroups():
big-bear-romm    |   File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__
big-bear-romm    |     self.gen.throw(value)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
big-bear-romm    |     raise exc
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/middleware/base.py", line 191, in __call__
big-bear-romm    |     response = await self.dispatch_func(request, call_next)
big-bear-romm    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/backend/utils/context.py", line 41, in set_context_middleware
big-bear-romm    |     return await call_next(request)
big-bear-romm    |            ^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/middleware/base.py", line 165, in call_next
big-bear-romm    |     raise app_exc
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/middleware/base.py", line 151, in coro
big-bear-romm    |     await self.app(scope, receive_or_disconnect, send_no_error)
big-bear-romm    |   File "/backend/handler/auth/middleware.py", line 146, in __call__
big-bear-romm    |     await self.app(scope, receive, send_wrapper)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/middleware/authentication.py", line 48, in __call__
big-bear-romm    |     await self.app(scope, receive, send)
big-bear-romm    |   File "/backend/handler/auth/middleware.py", line 19, in __call__
big-bear-romm    |     await super().__call__(scope, receive, send)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette_csrf/middleware.py", line 72, in __call__
big-bear-romm    |     await self.app(scope, receive, send)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 91, in __call__
big-bear-romm    |     await self.simple_response(scope, receive, send, request_headers=headers)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 146, in simple_response
big-bear-romm    |     await self.app(scope, receive, send)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
big-bear-romm    |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
big-bear-romm    |     raise exc
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
big-bear-romm    |     await app(scope, receive, sender)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/routing.py", line 758, in __call__
big-bear-romm    |     await self.middleware_stack(scope, receive, send)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/routing.py", line 778, in app
big-bear-romm    |     await route.handle(scope, receive, send)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/routing.py", line 299, in handle
big-bear-romm    |     await self.app(scope, receive, send)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/routing.py", line 79, in app
big-bear-romm    |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
big-bear-romm    |     raise exc
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
big-bear-romm    |     await app(scope, receive, sender)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/routing.py", line 74, in app
big-bear-romm    |     response = await func(request)
big-bear-romm    |                ^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 278, in app
big-bear-romm    |     raw_response = await run_endpoint_function(
big-bear-romm    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
big-bear-romm    |     return await dependant.call(**values)
big-bear-romm    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/starlette/authentication.py", line 84, in async_wrapper
big-bear-romm    |     return await func(*args, **kwargs)
big-bear-romm    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/backend/endpoints/platform.py", line 37, in add_platforms
big-bear-romm    |     scanned_platform = await scan_platform(fs_slug, [fs_slug])
big-bear-romm    |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/backend/handler/scan_handler.py", line 92, in scan_platform
big-bear-romm    |     (await meta_igdb_handler.get_platform(platform_attrs["slug"]))
big-bear-romm    |      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/backend/handler/metadata/igdb_handler.py", line 201, in wrapper
big-bear-romm    |     return await func(*args)
big-bear-romm    |            ^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/backend/handler/metadata/igdb_handler.py", line 308, in get_platform
big-bear-romm    |     platforms = await self._request(
big-bear-romm    |                 ^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/backend/handler/metadata/igdb_handler.py", line 209, in _request
big-bear-romm    |     res = await httpx_client.post(
big-bear-romm    |           ^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1892, in post
big-bear-romm    |     return await self.request(
big-bear-romm    |            ^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1574, in request
big-bear-romm    |     return await self.send(request, auth=auth, follow_redirects=follow_redirects)
big-bear-romm    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1661, in send
big-bear-romm    |     response = await self._send_handling_auth(
big-bear-romm    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1689, in _send_handling_auth
big-bear-romm    |     response = await self._send_handling_redirects(
big-bear-romm    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1726, in _send_handling_redirects
big-bear-romm    |     response = await self._send_single_request(request)
big-bear-romm    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1763, in _send_single_request
big-bear-romm    |     response = await transport.handle_async_request(request)
big-bear-romm    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 372, in handle_async_request
big-bear-romm    |     with map_httpcore_exceptions():
big-bear-romm    |   File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__
big-bear-romm    |     self.gen.throw(value)
big-bear-romm    |   File "/src/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 86, in map_httpcore_exceptions
big-bear-romm    |     raise mapped_exc(message) from exc
big-bear-romm    | httpx.LocalProtocolError: Illegal header value b'Bearer '
big-bear-romm    | 192.168.1.116:0 - "POST /api/roms HTTP/1.0" 400
big-bear-romm    | INFO:      [nginx][2024-10-28 08:27:50]  192.168.1.116 - - "POST /api/roms HTTP/1.1" 400 31 "http://redvelvet:1080/scan" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.0.1 Safari/605.1.15" rt=1.091 uct="0.000" uht="0.029" urt="0.030"
gantoine commented 2 weeks ago

Check if ROMM_AUTH_SECRET_KEY is set correctly, then we can go from there.

MessiasLima commented 1 week ago

My issue was caused by the misconfiguration of the IGDB secret keys. Thats simply solvable... However, the application should give better error message, what put us in a better path to solve the problem.

Some generic error message like that make us to come here to ask for help instead of trying to fix what is wrong ourselves.

adamantike commented 6 days ago

@MessiasLima do you have more context about the underlying issue? What do you mean by "misconfiguration of the IGDB secret keys"? It's hard to improve the error messages in-app if we don't get more information about what was this specific error about.

MessiasLima commented 3 days ago

This error happens when the IGDB_CLIENT_ID and the IGDB_CLIENT_SECRET has invalid values. When we have valid values the application works as expected.