search

monodera / pfs_target_uploader

A web application to validate and upload target lists with pointing simulations for PFS observation
https://pfs-etc.naoj.hawaii.edu/uploader/
MIT License
1 stars 0 forks source link

The online PPP cannot finish by failing to show plots under some conditions #212

Open monodera opened 1 month ago

monodera commented 1 month ago

Describe the bug In some cases, the online PPP fails to finish. As long as I tried with various cases, it seems to fail to show plots. Success/fail cases are as follows:

Here is a CSV file to reproduce the issue. There are 4000 objects at random coordinates either L or M resolution.

random_example_n00004000_lm.csv

It seems that the websocket over reverse proxy is likely the cause, but I haven't yet found the exact cause.

To Reproduce Steps to reproduce the behavior:

  1. Open the web app.
  2. Select the CSV file attached above.
  3. Click "Simulate".
  4. Wait to see if the calculation finishes in 5-10min.

Expected behavior Show the results in the right panel.

Computer Environment:

monodera commented 1 month ago

In some cases, the app output error message:

2024-05-31 10:23:51.209 | INFO     | pfs_target_uploader.utils.ppp:netflow_iter:885 - PPP stopped since Nppc > 200 [netflow_iter s2]
2024-05-31 10:23:53.909 | INFO     | pfs_target_uploader.utils.ppp:PPPrunStart:991 - PPP run finished in 490.4 seconds
2024-05-31 10:23:53.910 | INFO     | pfs_target_uploader.utils.ppp:PPPrunStart:992 - PPP running status: 999
2024-05-31 10:23:53.913 | INFO     | pfs_target_uploader.utils.ppp:ppp_result:1372 - start creating PPP figures
2024-05-31 10:23:55.121 | INFO     | pfs_target_uploader.utils.ppp:ppp_result:1454 - creating PPP figures finished
2024-05-31 10:23:55.121 | INFO     | pfs_target_uploader.widgets.PppResultWidgets:show_results:92 - showing PPP results
ERROR:bokeh.server.protocol_handler:error handling message
message: Message 'PATCH-DOC' content: {'events': [{'kind': 'MessageSent', 'msg_type': 'bokeh_event', 'msg_data': {'type': 'event', 'name': 'button_click', 'values': {'type': 
error: WebSocketClosedError()
Traceback (most recent call last):
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/bokeh/server/protocol_handler.py", line 97, in handle
   work = await handler(message, connection)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/bokeh/server/session.py", line 94, in _needs_document_lock_wrapper
   result = func(self, *args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/bokeh/server/session.py", line 286, in _handle_patch
   message.apply_to_document(self.document, self)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/bokeh/protocol/messages/patch_doc.py", line 104, in apply_to_document
   invoke_with_curdoc(doc, lambda: doc.apply_json_patch(self.payload, setter=setter))
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/bokeh/document/callbacks.py", line 443, in invoke_with_curdoc
   return f()
          ^^^
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/bokeh/protocol/messages/patch_doc.py", line 104, in <lambda>
   invoke_with_curdoc(doc, lambda: doc.apply_json_patch(self.payload, setter=setter))
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/bokeh/document/document.py", line 391, in apply_json_patch
   DocumentPatchedEvent.handle_event(self, event, setter)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/bokeh/document/events.py", line 245, in handle_event
   event_cls._handle_event(doc, event)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/bokeh/document/events.py", line 280, in _handle_event
   cb(event.msg_data)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/bokeh/document/callbacks.py", line 390, in trigger_event
   model._trigger_event(event)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/bokeh/util/callback_manager.py", line 113, in _trigger_event
   self.document.callbacks.notify_event(cast(Model, self), event, invoke)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/bokeh/document/callbacks.py", line 260, in notify_event
   invoke_with_curdoc(doc, callback_invoker)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/bokeh/document/callbacks.py", line 443, in invoke_with_curdoc
   return f()
          ^^^
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/bokeh/util/callback_manager.py", line 109, in invoke
   cast(EventCallbackWithEvent, callback)(event)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/panel/reactive.py", line 494, in _server_event
   self._comm_event(doc, event)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/panel/reactive.py", line 481, in _comm_event
   state._handle_exception(e)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/panel/io/state.py", line 442, in _handle_exception
   raise exception
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/panel/reactive.py", line 479, in _comm_event
   self._process_bokeh_event(doc, event)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/panel/reactive.py", line 416, in _process_bokeh_event
   self._process_event(event)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/panel/widgets/button.py", line 247, in _process_event
   self.clicks += 1
   ^^^^^^^^^^^
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/param/parameterized.py", line 528, in _f
   instance_param.__set__(obj, val)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/param/parameterized.py", line 530, in _f
   return f(self, obj, val)
          ^^^^^^^^^^^^^^^^^
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/param/parameters.py", line 543, in __set__
   super().__set__(obj,val)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/param/parameterized.py", line 530, in _f
   return f(self, obj, val)
          ^^^^^^^^^^^^^^^^^
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/param/parameterized.py", line 1552, in __set__
   obj.param._call_watcher(watcher, event)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/param/parameterized.py", line 2525, in _call_watcher
   self_._execute_watcher(watcher, (event,))
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/param/parameterized.py", line 2505, in _execute_watcher
   watcher.fn(*args, **kwargs)
 File "/work/monodera/dev/pfs_target_uploader/src/pfs_target_uploader/pn_app.py", line 266, in cb_PPP
   panel_ppp.show_results()
 File "/work/monodera/dev/pfs_target_uploader/src/pfs_target_uploader/widgets/PppResultWidgets.py", line 212, in show_results
   self.ppp_figure.append(self.ppp_alert)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/panel/layout/base.py", line 450, in append
   self.objects = new_objects
   ^^^^^^^^^^^^
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/param/parameterized.py", line 528, in _f
   instance_param.__set__(obj, val)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/param/parameterized.py", line 530, in _f
   return f(self, obj, val)
          ^^^^^^^^^^^^^^^^^
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/param/parameterized.py", line 1552, in __set__
   obj.param._call_watcher(watcher, event)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/param/parameterized.py", line 2525, in _call_watcher
   self_._execute_watcher(watcher, (event,))
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/param/parameterized.py", line 2505, in _execute_watcher
   watcher.fn(*args, **kwargs)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/panel/reactive.py", line 374, in _param_change
   self._apply_update(named_events, properties, model, ref)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/panel/reactive.py", line 301, in _apply_update
   with unlocked():
 File "/home/monodera/.pyenv/versions/3.11.9/lib/python3.11/contextlib.py", line 144, in __exit__
   next(self.gen)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/panel/io/document.py", line 327, in unlocked
   futures += dispatch_tornado(conn, dispatch_events)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/panel/io/document.py", line 252, in dispatch_tornado
   WebSocketHandler.write_message(socket, msg.content_json)
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/tornado/websocket.py", line 332, in write_message
   raise WebSocketClosedError()
tornado.websocket.WebSocketClosedError
WARNING:bokeh.server.views.ws:Failed sending message as connection was closed
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-929' coro=<WebSocketProtocol13.write_message.<locals>.wrapper() done, defined at /work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/rror()>
Traceback (most recent call last):
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/tornado/websocket.py", line 1088, in wrapper
   await fut
tornado.iostream.StreamClosedError: Stream is closed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
 File "/work/monodera/dev/pfs_target_uploader/.venv/lib/python3.11/site-packages/tornado/websocket.py", line 1090, in wrapper
   raise WebSocketClosedError()
tornado.websocket.WebSocketClosedError