lucafaggianelli / plombery

Python task scheduler with a user-friendly web UI
http://lucafaggianelli.com/plombery/
MIT License
280 stars 40 forks source link

Duration timer never stops after 1st run #256

Open lzzzam opened 11 months ago

lzzzam commented 11 months ago

Tried a basic pipeline to check how return values are used in the GUI. The first run of the following simple code seems to generate unstable behaviour:

@task
def test():
    return 10

@task
def test1():
    return 20

@task
def test2():
    return 30

register_pipeline(
    id="test",
    description="simple pipeline fails",
    tasks=[test, test1, test2],
)

Here the output log:

INFO:     Will watch for changes in these directories: ['/Users/luca/Documents/JobScraper']
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [33667] using StatReload
INFO:     Started server process [33669]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     127.0.0.1:60912 - "GET / HTTP/1.1" 304 Not Modified
INFO:     127.0.0.1:60912 - "GET /assets/index-86ef8bb9.js HTTP/1.1" 304 Not Modified
INFO:     127.0.0.1:60912 - "GET /assets/index-4fe84deb.css HTTP/1.1" 304 Not Modified
INFO:     127.0.0.1:60913 - "GET /assets/vendor-3fad8192.js HTTP/1.1" 304 Not Modified
INFO:     127.0.0.1:60913 - "GET /api/auth/whoami HTTP/1.1" 200 OK
INFO:     127.0.0.1:60913 - "GET /api/pipelines/ HTTP/1.1" 200 OK
INFO:     ('127.0.0.1', 60914) - "WebSocket /api/ws/" [accepted]
INFO:     connection open
INFO:     127.0.0.1:60913 - "GET /api/runs/?pipeline_id=&trigger_id= HTTP/1.1" 200 OK
INFO:     127.0.0.1:60912 - "GET /api/pipelines/ HTTP/1.1" 200 OK
INFO:     127.0.0.1:60913 - "GET /api/runs/?pipeline_id=&trigger_id= HTTP/1.1" 200 OK
INFO:     127.0.0.1:60916 - "GET /api/pipelines/ HTTP/1.1" 200 OK
INFO:     127.0.0.1:60915 - "GET /api/runs/?pipeline_id=&trigger_id= HTTP/1.1" 200 OK
INFO:     127.0.0.1:60915 - "GET /api/pipelines/test/input-schema HTTP/1.1" 200 OK
INFO:     127.0.0.1:60915 - "POST /api/runs/ HTTP/1.1" 200 OK
Executing pipeline `test` #1 via trigger `_manual`
Executing task test
Executing task test1
Executing task test2
INFO:     127.0.0.1:60915 - "GET /api/pipelines/test HTTP/1.1" 200 OK
INFO:     ('127.0.0.1', 60917) - "WebSocket /api/ws/" [accepted]
Task exception was never retrieved
future: <Task finished name='Task-33' coro=<WebSocket.send_json() done, defined at /Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/starlette/websockets.py:168> exception=ConnectionClosedOK(Close(code=1005, reason=''), Close(code=1005, reason=''), True)>
Traceback (most recent call last):
  File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/starlette/websockets.py", line 173, in send_json
    await self.send({"type": "websocket.send", "text": text})
  File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/starlette/websockets.py", line 85, in send
    await self._send(message)
  File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in sender
    await send(message)
  File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 320, in asgi_send
    await self.send(data)  # type: ignore[arg-type]
  File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/Users/luca/Documents/JobScraper/venv/lib/python3.10/site-packages/websockets/legacy/protocol.py", line 935, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: received 1005 (no status code [internal]); then sent 1005 (no status code [internal])
INFO:     connection open
INFO:     connection closed
INFO:     127.0.0.1:60916 - "GET /api/runs/1 HTTP/1.1" 200 OK
INFO:     127.0.0.1:60916 - "GET /api/runs/1/logs HTTP/1.1" 200 OK
INFO:     127.0.0.1:60918 - "GET /api/pipelines/test HTTP/1.1" 200 OK
INFO:     127.0.0.1:60920 - "GET /api/runs/1/logs HTTP/1.1" 200 OK
INFO:     127.0.0.1:60919 - "GET /api/runs/1 HTTP/1.1" 200 OK
lzzzam commented 11 months ago

UPDATE: it seems that adding a little delay into each task seems to fix problem, something like

time.sleep(0.2)

lucafaggianelli commented 10 months ago

hi, thanks for reporting this, I'll check what's going on.

lucafaggianelli commented 10 months ago

After some investigations on different OS and browsers I wasn't able to reproduce this issue, though it seems bound to the websocket and not to the pipeline executor, I'll try to handle those websocket connection issues!