search

galaxyproject / planemo

Command-line utilities to assist in developing Galaxy and Common Workflow Language artifacts - including tools, workflows, and training materials.
https://planemo.readthedocs.io/
MIT License
86 stars 85 forks source link

Galaxy upload tool fails when using planemo server on galaxy release branch 24.0 #1455

Closed richard-burhans closed 3 weeks ago

richard-burhans commented 1 month ago

The upload tool in Galaxy fails when running via planemo serve for galaxy branch release_24.0. The upload tool works when using branch release_23.2: rm -rf /scratch/rico/galaxy; planemo serve --galaxy_branch release_23.2 --galaxy_root /scratch/rico/galaxy. The upload tool fails when using branch release_24.0: rm -rf /scratch/rico/galaxy; planemo serve --galaxy_branch release_24.0 --galaxy_root /scratch/rico/galaxy. I tried this with the following python versions: 3.8.19, 3.9.19, 3.10.14, 3.11.9, 3.12.3

I'm seeing the following:

[2024-06-18 17:44:54 -0400] [70072] [ERROR] Exception in ASGI application
  + Exception Group Traceback (most recent call last):
  |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/base.py", line 192, in __call__
  |     await response(scope, wrapped_receive, send)
  |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/responses.py", line 264, in __call__
  |     await wrap(partial(self.listen_for_disconnect, receive))
  |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
  |     raise BaseExceptionGroup(
  | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 412, in run_asgi
    |     result = await app(  # type: ignore[func-returns-value]
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    |     return await self.app(scope, receive, send)
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/fastapi/applications.py", line 1054, in __call__
    |     await super().__call__(scope, receive, send)
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/applications.py", line 123, in __call__
    |     await self.middleware_stack(scope, receive, send)
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/errors.py", line 186, in __call__
    |     raise exc
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/errors.py", line 164, in __call__
    |     await self.app(scope, receive, _send)
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette_context/middleware/raw_middleware.py", line 92, in __call__
    |     await self.app(scope, receive, send_wrapper)
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/base.py", line 193, in __call__
    |     response_sent.set()
    |   File "/usr/local/lib/python3.8/contextlib.py", line 131, in __exit__
    |     self.gen.throw(type, value, traceback)
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    |     raise exc
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/responses.py", line 260, in wrap
    |     await func()
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/base.py", line 217, in stream_response
    |     return await super().stream_response(send)
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/responses.py", line 252, in stream_response
    |     await send({"type": "http.response.body", "body": chunk, "more_body": True})
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette_context/middleware/raw_middleware.py", line 81, in send_wrapper
    |     await send(message)
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/errors.py", line 161, in _send
    |     await send(message)
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 509, in send
    |     output = self.conn.send(event=h11.Data(data=data))
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/h11/_connection.py", line 512, in send
    |     data_list = self.send_with_data_passthrough(event)
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/h11/_connection.py", line 545, in send_with_data_passthrough
    |     writer(event, data_list.append)
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/h11/_writers.py", line 65, in __call__
    |     self.send_data(event.data, write)
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/h11/_writers.py", line 91, in send_data
    |     raise LocalProtocolError("Too much data for declared Content-Length")
    | h11._util.LocalProtocolError: Too much data for declared Content-Length
    +------------------------------------

During handling of the above exception, another exception occurred:

  + Exception Group Traceback (most recent call last):
  |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/_utils.py", line 87, in collapse_excgroups
  |     yield
  |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/base.py", line 193, in __call__
  |     response_sent.set()
  |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
  |     raise BaseExceptionGroup(
  | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/responses.py", line 264, in __call__
    |     await wrap(partial(self.listen_for_disconnect, receive))
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/responses.py", line 260, in wrap
    |     await func()
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/responses.py", line 237, in listen_for_disconnect
    |     message = await receive()
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/base.py", line 52, in wrapped_receive
    |     msg = await self.receive()
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 542, in receive
    |     await self.message_event.wait()
    |   File "/usr/local/lib/python3.8/asyncio/locks.py", line 309, in wait
    |     await fut
    | asyncio.exceptions.CancelledError
    |
    | During handling of the above exception, another exception occurred:
    |
    | Exception Group Traceback (most recent call last):
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/base.py", line 192, in __call__
    |     await response(scope, wrapped_receive, send)
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/responses.py", line 264, in __call__
    |     await wrap(partial(self.listen_for_disconnect, receive))
    |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 678, in __aexit__
    |     raise BaseExceptionGroup(
    | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
    +-+---------------- 1 ----------------
      | Traceback (most recent call last):
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 412, in run_asgi
      |     result = await app(  # type: ignore[func-returns-value]
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
      |     return await self.app(scope, receive, send)
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/fastapi/applications.py", line 1054, in __call__
      |     await super().__call__(scope, receive, send)
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/applications.py", line 123, in __call__
      |     await self.middleware_stack(scope, receive, send)
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/errors.py", line 186, in __call__
      |     raise exc
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/errors.py", line 164, in __call__
      |     await self.app(scope, receive, _send)
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette_context/middleware/raw_middleware.py", line 92, in __call__
      |     await self.app(scope, receive, send_wrapper)
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/base.py", line 193, in __call__
      |     response_sent.set()
      |   File "/usr/local/lib/python3.8/contextlib.py", line 131, in __exit__
      |     self.gen.throw(type, value, traceback)
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
      |     raise exc
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/responses.py", line 260, in wrap
      |     await func()
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/base.py", line 217, in stream_response
      |     return await super().stream_response(send)
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/responses.py", line 252, in stream_response
      |     await send({"type": "http.response.body", "body": chunk, "more_body": True})
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette_context/middleware/raw_middleware.py", line 81, in send_wrapper
      |     await send(message)
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/errors.py", line 161, in _send
      |     await send(message)
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 509, in send
      |     output = self.conn.send(event=h11.Data(data=data))
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/h11/_connection.py", line 512, in send
      |     data_list = self.send_with_data_passthrough(event)
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/h11/_connection.py", line 545, in send_with_data_passthrough
      |     writer(event, data_list.append)
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/h11/_writers.py", line 65, in __call__
      |     self.send_data(event.data, write)
      |   File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/h11/_writers.py", line 91, in send_data
      |     raise LocalProtocolError("Too much data for declared Content-Length")
      | h11._util.LocalProtocolError: Too much data for declared Content-Length
      +------------------------------------

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 412, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette_context/middleware/raw_middleware.py", line 92, in __call__
    await self.app(scope, receive, send_wrapper)
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/base.py", line 193, in __call__
    response_sent.set()
  File "/usr/local/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/_utils.py", line 93, in collapse_excgroups
    raise exc
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/responses.py", line 260, in wrap
    await func()
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/base.py", line 217, in stream_response
    return await super().stream_response(send)
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/responses.py", line 252, in stream_response
    await send({"type": "http.response.body", "body": chunk, "more_body": True})
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette_context/middleware/raw_middleware.py", line 81, in send_wrapper
    await send(message)
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/starlette/middleware/errors.py", line 161, in _send
    await send(message)
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 509, in send
    output = self.conn.send(event=h11.Data(data=data))
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/h11/_connection.py", line 512, in send
    data_list = self.send_with_data_passthrough(event)
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/h11/_connection.py", line 545, in send_with_data_passthrough
    writer(event, data_list.append)
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/h11/_writers.py", line 65, in __call__
    self.send_data(event.data, write)
  File "/scratch/rico/planemo/gx_venv_3_release_24.0/lib/python3.8/site-packages/h11/_writers.py", line 91, in send_data
    raise LocalProtocolError("Too much data for declared Content-Length")
h11._util.LocalProtocolError: Too much data for declared Content-Length
mvdbeek commented 1 month ago

Can you tell us how and what you're uploading and which upload tool this is ? All the tests are passing fine, so this seems odd. As a sidenote

planemo serve --galaxy_branch release_24.0 --galaxy_root /scratch/rico/galaxy

these are mutually exclusive and I don't know which one wins.

richard-burhans commented 1 month ago

Can you tell us how and what you're uploading and which upload tool this is ? All the tests are passing fine, so this seems odd. As a sidenote

planemo serve --galaxy_branch release_24.0 --galaxy_root /scratch/rico/galaxy

these are mutually exclusive and I don't know which one wins.

I have these two files downloaded to a local directory on my machine:

To upload I'm clicking on the Upload icon and clicking on Choose local file button. This is on a system running Debian 10. I found this odd as well, so I wouldn't be surprised if perhaps I'm missing some requirement or similar.

richard-burhans commented 1 month ago

The upload tool works as expected when running planemo serve using python 3.9.2 and Galaxy release_24.0 on a system running Debian 11. I don't know if it's worth it to track down the difference(s) between Debian 10 and Debian 11 that cause this issue.

mvdbeek commented 3 weeks ago

Hmm, I wonder if perhaps you've run into sqlite not being bundled with python, or sqlite being too old. We probably need a more verbose check that certain requirements are met.