Epidemic and Transportation Network Viz Through Pluto.jl

[yvs314]

Prototype visualizers are tools/vl-post-viz.jl and tools/vl-pre-viz.jl.

The pre-viz interfaces with Oboe but does not need experiment results. The post-viz does not interface ith Oboe but runs off the experiment results. To recompute experiment results, run the m-core/sweep.m, keying in the correct instance name.

CAVEAT: sweep.m is in MATLAB, and requires a MATLAB environment. If it doesn't quite work with MATLAB, some output files are available in the exp1 branch. The overall idea was to never store output as part of the repository, but making a stub branch for that I view as a reasonable solution for archiving.

The output files are documented in sweep.m itself. Please raise an issue if this documentation feels insufficient.

Objective 1: Solution Explorer App

In the Pluto.jl notebook environment, implement the pltOpt, pltNull, and pltIVs plots so that it is possible to select the day for which pltOpt and pltNull are shown with a slider. Add an option for .pdf and .svg export for the plots, but do not run it by default. Any export must default to the /fig directory in the repository, which is in .gitignore by design.

The day is in fact controlled by the column index in the output files, which vl-post-viz.jl reads. Absolute numbers of infected on day no. NNN are stored in the column ZNNN. Note that the display is meant to be paired, one frame for null-control pltNull, and the other for optimal control pltOpt, side-by-side, with the scale's median set to the median no. infected in the null-control plot.

It is possible to do the same in Jupyter NB, but VegaLite.jl there is slower than, at the very least, the VS Code plot feature. Plus, Pluto.jl was made with a view towards JavaScript NB Observable so it ought to work well for a low-maintenance not-quite-an-app

Optional 1.1 Add visual selection of output files. These are mandated to appear in epi-net-m/out

Optional 1.2 Add visual selection of days/compartment columns to display.

Objective 2: Network Explorer App

Ditto for vl-pre-viz.jl. Consider combining them, or at least sharing some common code via external Julia modules.

The plots to be re-fashioned into a Pluto NB are pltM2 and pltP. The filename loading/saving is somewhat cludgy in vl-pre-viz.jl, look in vl-post-viz.jl for better options.

More Objectives to come, I'll add them here.

[karaign]

Quick update: I've implemented a bare-minimum Pluto notebook. So far the only thing it does is display the optimal solution side-by-side with the null solution, complete with the day slider. Would appreciate a second look. Some notes/observations/problems:

• Redrawing the graph takes a little over 20 seconds, not exactly snappy. Though, curiously, after it's done running, the timer within Pluto shows that it only took around 40 ms.
• By contrast, using the REPL in VS Code takes over 50 seconds to produce the first graph in a session, but subsequent graphs are produced near-instantly.
• The day variable is updated at the first movement of the slider, rather than waiting for the user to release the mouse button. Thus, dragging the slider from e.g. 30 to 100 actually results in redrawing the graph twice, first for day = 31 and then for day = 100. I could try tapping into the JavaScript underpinnings to change that behaviour (something Pluto encourages), although if we manage to get the render time down, this won't be as relevant.

[yvs314]

Sorry, won't be able to take any look until the next week. Feel free to iterate further without more input.

• Right, 20 seconds through Pluto. Consider breaking it down to what exactly takes these 20 seconds, preferably with explicit benchmarks of the parts.
• I've had a similar behavior in Jupyter vs. VS Code. Here's a difference: Jupyter took a while to draw a pic, however, it would instantly export to svg. VS Code would do (subsequent) pics quickly, just as you report. But exporting to pdf would take the same 20-something seconds. Can it be that notebooks do more of exporting before drawing? Also, I wonder if it's possible to cache most of the drawing. I mean, the county boundaries do not shift between the days; dunno about the overall structure, but it shouldn't take that long to switch 75 numbers.
• Slider triggering double redrawing noted, thanks for describing the situation. Sliders are a death sentence for 20 seconds-long redraws, let's replace it. However, retain the slider-related code, it'll be useful if re-drawing can be tuned to, say, half a second. If it's feasible, I'd like you to replace the slider with the following:
  • field to enter the day number, and arrows to switch to the day before and the day after. Re-drawing is to launch once (a) a number is typed into the field and Return is pressed or focus is withdrawn from the field (b) the previous/next day arrow is clicked. First and last days should be indicated somewhere nearby so they're clearly visible.

P.S. I would appreciate an educated guess into whether descending to the lower-layer library, D3.js, is worth the re-implementation effort. As alternative hypothesis, consider choropleths via Plotly.jl.

[karaign]

Implemented the new day selector. Also, I have some thoughts regarding the performance issue. VegaLite.jl relies on calling the Node.js implementation of Vega to produce the SVG output (relevant function in VegaLite.jl). It appears that this is the bottleneck, as a node process can be seen in the system monitor whenever the graph is rebuilt, and up until it's done. Plotting the graph in VSCode does not appear to call the linked function or spawn a node process, which would indicate that it uses a different renderer. I'm currently investigating the exact mechanism that VSCode uses.

[karaign]

Update, the method that VSCode's Julia extension uses to render the plots is in fact Vega-Embed. In theory I should be able to integrate that into Pluto and thus achieve a similar performance to VSCode (which, let's not forget, is a fancy browser window). It's proving trickier than I expected to actually get it to work, though, but I should be able to within the next couple days.

[karaign]

And another update: I got Vega-Embed to work, and it works pretty well too. It's a bit hard to say how long exactly it takes to redraw the plot, with the multiple programming languages involved, but Pluto reports somewhere around 30 ms to run the cell and then on the JS side, the rendering takes around 110 ms. This is not a formal benchmark or anything, and there's quite a bit of spread, but it seems to take 0.2 s at worst which doesn't seem too bad.

This does mean that we can go back to the slider, though I think it is useful to be able to enter the exact day with precision, or to immediately jump to the next/previous day, which is why I suggest merging the two input methods and having both a slider and an input field.

P.S. While I was typing up this comment, I stumbled at this interesting line in the VegaLite.jl source code, which calls this function. Apparently, the code to use Vega-Embed within Pluto is already a part of VegaLite.jl -- but it's not part of the latest release version (current latest is 2.6.0). This makes my implementation seem like a workaround, but I think it'll do until this functionality becomes official.

[yvs314]

Great find with the Vega-Embed, the present performance is quite satisfactory, a very big step forward from my naive implementation.

Merging the two input modes is a sound idea, however, I'd say that the existing text field and first/prev/$day/next/last design quite convenient already. It would also help to add “fast reverse/fast forward though”, say, +- 10 days, possibly with an option to set the jump size by hand.

Interesting observation on Vega-Embed within Pluto; I support the decision to retain the workaround, that's the payment for the bleeding-edge solutions.

I've got a big question though: where is the code that draws the figure? I mean, I can't see it from the Pluto's interface. How would the user (me) plug in another viz. routine? I'd think it reasonable to isolate the figure VegaLite specs in separate files.

P.S. The existing visualizer requires by-county solution, it'd make sense to filter them out from the picker; ideally, not remove but grey out the non-matching solutions just add a text warning on the lines of “visualization only for county-level solutions”.

There are more visualization options coming, which would be agg-agnostic (bubble maps). There's an option to adjust any solution to match to county-level shapefiles, but I don't feel like formalizing it right now.

[karaign]

I've got a big question though: where is the code that draws the figure? I mean, I can't see it from the Pluto's interface. How would the user (me) plug in another viz. routine? I'd think it reasonable to isolate the figure VegaLite specs in separate files.

It's in the cell directly above the plots and below us10m = dataset("us-10m"), I tacked a nothing at the end of the cell to prevent Pluto from trying to render it, which confusingly results in Pluto displaying a totally blank cell. And moving the specs to a separate file is a good idea -- I'm still working on improving the code structure, the UI elements (the day picker and the Vega-Embed thing) also seem like something I should isolate since those things can be reused in other visualizations and also just clutter up the notebook.

[karaign]

Update on the progress so far – I have moved the visualization code to a sort of a modular structure (with UI elements and VegaLite specs in separate files) and modified the day picker to support arbitrary steps. Intending to move on to exporting and the network explorer now. A couple of notes:

• There is a bug in my implementation of the day selector where it sometimes refuses to accept new values from the textbox. I have yet to determine what circumstances this happens in, but clicking one of the jump buttons fixes it and you can input new values into the textbox again.
• It took me quite a while to figure out how to use modules within Pluto – FromFile doesn't work correctly (I assume this is due to the way Pluto handles macros – see here), and using include requires restarting the entire notebook every time one of the included files is changed. Several workarounds are discussed in this issue, and the one I went with is wrapping include calls inside a module since it seemed like it involves the least overhead. So that is what the includes module at the top of viz.jl is for.

[yvs314]

Thanks for documenting the progress, and the why of plain include within Pluto. It is OK to swap Pluto for something more convenient, but that ought to be discussed before switching to another option.

I can confirm that I had encountered the bug mentioned, and stumbled upon the workaround (“I clicked around, and it started to work”); it's good to have a tip on where exactly to click.

[karaign]

Implemented the exporting. Bit of a caveat – since Pluto is designed entirely around the idea of reactivity, any file-exporting code will be automatically re-run whenever any relevant parameter is changed, and you would need to disable the cell to stop it from doing so.

I believe I have covered all the requirements. Let me know if I missed anything, otherwise I will put together a pull request as soon as I can.

[yvs314]

Implemented the exporting.

Right, so I tested it and I like that it all works, and works fast enough.

I believe I have covered all the requirements. Let me know if I missed anything, otherwise I will put together a pull request as soon as I can.

Everything is in, but please see the Requests List below. Up to you whether you do it with the first PR or as an update. Don't neglect adding this work to your part in the Acknowledgments section in the main Readme

Bit of a caveat – since Pluto is designed entirely around the idea of reactivity, any file-exporting code will be automatically re-run whenever any relevant parameter is changed, and you would need to disable the cell to stop it from doing so.

Disable cell option noted and tested. Is it feasible to circumvent this problem by only activating export on pressing a button? The button should read “Press to Export” and, ideally, reflect that this particular picture was exported. The timestamp in the file name may be replaced by something else (a hash of some parameters?)

Please provide a sort of “after-action report” in pull request, e.g. like in this PR.

I would also ask you to speculate (in a PR, issue, or elsewhere) on whether Pluto.jl is appropriate for this kind of not-quite-app, which is rapidly turning into a dashboard:

In your opinion, is it better or worse than other options, e.g. Electron, or some more traditional GUI framework? What about something closer to plain dynamic HTML? JavaScript notebooks, e.g. Observable? How hard is rebuilding the project and writing code in a manageable, modular fashion in these cases?

The options listed here are off the top of my head, and I lack expertise in GUI apps, so please do not feel obliged to cover the exact “frameworks” mentioned above.

Requests List

R1. Please add a cell that specifies the export path before the export occurs, something like “will export to ...” R1.1. Consider the export-on-button-press option. A nasty hack like commenting out the export line is also an option, and it is better than the present unstoppable export. R2. Document the entry points (viz.jl, network-viz.jl). Consider encapsulating the auxiliary files ([viz-ua,vega-specs,exporter].jl) into a subdirectory. R3. (optional) Add export to EPS (Encapsulated Postscript), at least, if it's relatively straightforward. R4. (optional) viz.jl: ensure the output file read by default is something with cty in its name.

[karaign]

Disable cell option noted and tested. Is it feasible to circumvent this problem by only activating export on pressing a button? The button should read “Press to Export” and, ideally, reflect that this particular picture was exported. The timestamp in the file name may be replaced by something else (a hash of some parameters?)

Adding a button to toggle the export is trivial – the hard part is making the cell respond just to the button and not to changes in other variables. It's probably possible to find a workaround, but that honestly seems like going against Pluto's design philosophy – its central selling point is that there is no hidden state, which makes it great for playing around with code interactively, but not so great for developing an app with side effects.

In your opinion, is it better or worse than other options, e.g. Electron, or some more traditional GUI framework? What about something closer to plain dynamic HTML? JavaScript notebooks, e.g. Observable? How hard is rebuilding the project and writing code in a manageable, modular fashion in these cases?

It does seem like we are really stretching the intended use case for Pluto here. IMO the best use scenario for Pluto is for relatively small and self-contained notebooks where it's valuable to be able to change the code and get instant feedback. Trying to build a more complex "app" on it results in extra complications like modularity and side effects that it just wasn't designed to handle.

Something I've been thinking about is using a reactive JavaScript framework to build the UI (I'm partial to Vue.js). The architecture would consist of a single webpage (dashboard, if you will) containing the front end, and a Julia backend responsible for serving up that dashboard and generating Vega specs for the frontend to display. Porting things over to this new setup shouldn't be that hard, as most of the backend stuff has already been written (either in the Pluto notebooks or their auxiliary modules) and I could adapt the existing HTML and JavaScript – most work would probably go into setting up the toolchain and writing the HTTP glue connecting the frontend with the backend. (apparently people have already done some interesting work in this direction: see Stipple and Blink.jl)

[yvs314]

Adding a button to toggle the export is trivial – the hard part is making the cell respond just to the button and not to changes in other variables. It's probably possible to find a workaround, but that honestly seems like going against Pluto's design philosophy – its central selling point is that there is no hidden state, which makes it great for playing around with code interactively, but not so great for developing an app with side effects.

I see, thanks for explaining. In this case, I view commenting out the call to the exporting function as the lesser of two evils. To be supplemented with “uncomment the line below to export” message above it, naturally. That was said assuming the cell's disabled status is not retained by Pluto. If it is actually retained, then please just disable the export by default.

Something I've been thinking about is using a reactive JavaScript framework to build the UI (I'm partial to Vue.js). The architecture would consist of a single webpage (dashboard, if you will) containing the front end, and a Julia backend responsible for serving up that dashboard and generating Vega specs for the frontend to display. Porting things over to this new setup shouldn't be that hard, as most of the backend stuff has already been written (either in the Pluto notebooks or their auxiliary modules) and I could adapt the existing HTML and JavaScript – most work would probably go into setting up the toolchain and writing the HTTP glue connecting the frontend with the backend. (apparently people have already done some interesting work in this direction: see Stipple and Blink.jl)

That would be worth pursuing after the Request List is integrated into the current Pluto-based version. As for the particular tool, I'd go with the more mature and (I expect) well-documented solution such as Vue.js, with or without Stipple. As much as I like the idea of Julia interface to Electron (Blink.jl), it appears too raw at this time, what with all the caveats.