viewer.add_markers() has error on browser dev console

[Jdaviz-Triage-Bot]

More notes at https://github.com/spacetelescope/jdaviz/issues/1664#issuecomment-1274821528

Reporter: pllim

Run the Imviz example notebook up to the point of viewer.add_markers() call. See screenshot.

Update: The red screenshot went away with https://github.com/glue-viz/glue-jupyter/pull/324

Maybe the table needs to be sanitized before being passed into glue.

Sometimes I also see this but not sure if it is related or a different problem:

Update: IOPub problem might have been addressed by https://github.com/glue-viz/glue-jupyter/pull/325

IOPub message rate exceeded.
The notebook server will temporarily stop sending output
to the client in order to avoid crashing it.
To change this limit, set the config variable
`--NotebookApp.iopub_msg_rate_limit`.

Current values:
NotebookApp.iopub_msg_rate_limit=1000.0 (msgs/sec)
NotebookApp.rate_limit_window=3.0 (secs)

This is a more specific case than #1367 .

🐱

---

DISCLAIMER: This issue was autocreated by the Jdaviz Issue Creation Bot on behalf of the reporter. If any information is incorrect, please contact Duy Nguyen

[pllim]

I tried tweaking add_markers logic but nothing worked. Perhaps this is deeper upstream, maybe should ask SMEs.

[maartenbreddels]

This is bqplot-image-gl : https://github.com/glue-viz/bqplot-image-gl/blob/master/js/lib/imagegl.js

which means the image gets fed the wrong type of data.

How can this be reproduced?

[pllim]

How can this be reproduced?

Run the https://github.com/spacetelescope/jdaviz/blob/main/notebooks/ImvizExample.ipynb and you will see it pop up during add_markers step.

[pllim]

add_markers is basically linking table to image. I got that code from an old glue-jupyter example at https://github.com/glue-viz/glue-jupyter/blob/main/notebooks/Astronomy/W5/W5%20Tutorial.ipynb

https://github.com/spacetelescope/jdaviz/blob/21cbeae0834a8d150fbc502f7b466f67f8dd9d10/jdaviz/core/astrowidgets_api.py#L391

[maartenbreddels]

IOPub message rate exceeded. The notebook server will temporarily stop sending output to the client in order to avoid crashing it. To change this limit, set the config variable --NotebookApp.iopub_msg_rate_limit.

Current values: NotebookApp.iopub_msg_rate_limit=1000.0 (msgs/sec) NotebookApp.rate_limit_window=3.0 (secs)

I've also seen this, and what I see happening is.

• a lot of triggering of vue_relayout (I see a lot of setting to 99.9 % and 100%
• a lot of triggering of updates to https://github.com/spacetelescope/jdaviz/blob/ac58644e2f3027f54f88dc5e4c3ee25f49d046f3/jdaviz/app.py#L179 which I think is due to https://github.com/glue-viz/glue-jupyter/blob/0fc5428fce5f9cb50588774de5f5316281cb21b2/glue_jupyter/state_traitlets_helpers.py#L126 triggering on any state change.
• a lot of 'glue_state' updates, but I do not know which layer class is responsible for that since there are multiple traits with that name in glue_jupyter.

What I have seen, is that two updates had the same update data, meaning that add_global_callback was triggered even when no data was changed. This could be a feature or a bug of echo (question for @astrofrog)

[pllim]

• @astrofrog will investigate unnecessary glue_state updates by maybe comparing JSON before sending that to frontend, see https://github.com/glue-viz/glue-jupyter/pull/325
• Jdaviz team needs to see if vue_relayout trigger can be removed, see https://github.com/spacetelescope/jdaviz/issues/1720
• Jdaviz team should review https://github.com/glue-viz/glue-jupyter/pull/324
• @rosteen will investigate transient "magic ignore" errors and post the traceback somewhere. -- @astrofrog could reproduce this and said he would submit a patch (2022-10-11), see https://github.com/spacetelescope/jdaviz/issues/1728

[astrofrog]

I think this will be partially fixed by https://github.com/glue-viz/glue-jupyter/pull/325 - can someone check?

[astrofrog]

This was closed accidentally by https://github.com/glue-viz/glue-jupyter/pull/324 which only addressed one of the issues here

[pllim]

I'll check after lunch break. Thanks for the quick fixes!

[rosteen]

@astrofrog I looked at the number of messages between the front and back ends when running the cell to add markers in the ImvizExample notebook, and it looks like it went from ~900 messages before your glue-jupyter PR to ~250 after, so I'd call that a success.

[maartenbreddels]

Nice, can I ask how you counted that? I didn't see any message counter in chrome's dev console. 250 is still quite large, is that the 99.9/100.0% trick we're seeing?

[rosteen]

I just looked at the Response tab for the channel in Firefox. Number of messages is below the last red arrow in the screenshot (note that I accidentally moused over the image here and added 70 extra messages, the number for the marker cell was ~200).

Edit: This also seems to have cut the execution time a bit for that cell (adding markers). It's consistently ~900-1000 ms on glue-jupyter main, and ~50-75% of that (490-700 ms) with Tom's PR. Not rigorous benchmarking, just tried it a few times with each.