Redesign the menu for adding views

[alex-r-bigelow]

We need a new interface for managing and adding views; we should schedule a whiteboard meeting in the coming weeks. In the mean time, here are some of the things we'll want to consider—please feel free to add to / edit this list!

• Adding / managing views
  • Let the user pick metrics for Sayef's charts
  • Let the user compare two runs (from the same dataset? across datasets? and just in the tree, or other views?)
  • Anticipate views we haven't implemented yet:
  • histograms from traveler-gantt
  • bar chart from expression-trees
  • Anticipate Voyager-style, auto-"hey look at this correlation" functionality?
  • Let the user choose from multiple newick trees from the same run (e.g. LRA vs read_x vs read_y)
  • We should probably do something sensible to explain why some views are unavailable (e.g. when there's no OTF2 trace)
• Consolidating datasets
  • Multiple runs per dataset (e.g. Kevin's buildbot data) should be combined into one entry
  • Should Jupyter-generated data be combined similarly (e.g. combine data from the same Jupyter cell?)
  • Do the nested GoldenLayout views, across datasets, make sense (e.g. where would we put cross-dataset comparison trees)?
• Other
  • How much of Katy's key functionality should be incorporated at the app-wide level (e.g. my choice to move the inclusive / exclusive / difference menu out of the tree, into the sidebar, still feels... weird)?
  • The menu should be adjustable, instead of the fixed pane
  • Anything we need to incorporate / consider from the Agave workflow?

[kawilliams]

A big question (as a person who knows the ins and outs of one of our 3?4+? views), what is the Venn diagram for the data across the views? How are we matching the data across views (I'd assume by the PhySL primitive name)?

My data:

• a Newick tree representation of the expression tree containing primitive instance names (the PhySL version -- "/phylanx/access-argument$10$num_factors/9$44$42")
• a csv of the performance information for that expression tree
  • "primitive_instance": the same PhySL name of the primitive as in the tree [how I match the data]
  • "display_name": a prettier version of the primitive name (e.g. "access-argument/num_factors(44, 42)")
  • "count": the number of instances of that primitive. There was some discussion across teams about if we wanted the average time or the total time -- I think the final decision was total time (I.e. NOT time/count) so I have the calculations for avg_time still lurking in the code but the attribute is unused
  • "time": total time of execution of that primitive, in nanoseconds (oo, Devil's advocate Data review!: do the times in my csv match Kate's Gantt times? Should they match exactly or close enough or not at all?)
  • "eval_direct": whether the primitive was executed asynchronously, directly, or it was decided at runtime (we call it "undecided"). This is encoded as "1" for direct, "0" for async, and "-1" for undecided

A related Venn diagram (and slightly less clear to elucidate): what's the VD for the tasks/intent across views? e.g. I click a node in the tree in order to see it in the Gantt chart so that I can understand its functional dependencies and communication dependencies

[alex-r-bigelow]

Yes, the PhySL primitive name is how we match stuff from the OTF2 trace as well (in database.py, I use it as the key to a per-dataset primitives dict, that in turn maps to a per-primitive dict containing keys like display_name, count, time, and eval_direct... this dict is the tooltip that you see when you hover over a node in the tree).

I think the --debug flag when running bundle.py should add some information to each primitive dict about which data sources contributed to a primitive's information (otf2, csv, newick, and/or dot), but it doesn't (yet) tell you which attribute came from where—we could definitely change that.

oo, ~Devil's advocate~ Data review!: do the times in my csv match Kate's Gantt times? Should they match exactly or close enough or not at all?

Umm, yeah, about that... I feel like I've run across instances where the total time from intervals in the OTF2 trace doesn't seem to match up with what's reported in the performance CSV. I should probably add to something like an otf2_time running total for each primitive while processing intervals, and then do a check at the end to make sure the totals match.