Update state diagram/automatic client-side test generator

[kdahlquist]

I am thinking that our state diagram might need to be updated due to the changes we made for importing/exporting SIF and GraphML.

[kdahlquist]

@mihirsamdarshi is going to consult with @NAnguiano to work on this.

[mihirsamdarshi]

Looked over current diagram, now trying to figure out where to put the new functionality, and how it would likely fit into the current diagram.

[kdahlquist]

This is something @NAnguiano could possibly help with because she is the main architect of the diagram.

[kdahlquist]

@mihirsamdarshi is redoing the diagram right now in Illustrator because the original program LucidChart @NAnguiano used maxed out the free version.

We need to look into other options like yED or Gephi for a diagramming tool that is scalable. @mihirsamdarshi should take 15-20 minutes to evaluate each of those to see if they are viable in terms of ease of use and how the diagram will look.

[kdahlquist]

@mihirsamdarshi will use yED and aims to have all the old stuff done by next week.

[kdahlquist]

Almost done... @mihirsamdarshi, can you get this wrapped up this week?

[mihirsamdarshi]

done.

[kdahlquist]

reassigned to @ebachoura and @eileenchoe to add feature to show/hide all weights. They need to talk to @mihirsamdarshi to get a walkthrough on the diagram and to @anuvarsh to get a walkthrough on the feature.

[dondi]

This conversation has taken place and the addition of the new states is now in progress on the yEd file.

[eileenchoe]

After analyzing the current state diagram, and getting a walkthrough on the new feature (edge weight settings) @ebachoura and I have realized that adding this feature (and any future new feature) will increase the complexity of the state diagram by a factor of two or more. Additionally we've noted that, this feature only affects weighted networks but in its current implementation, is also available for unweighted and unloaded networks (but doesn't induce a change). Waiting to discuss for further direction on how to most effectively implement this new feature in the state diagram.

[kdahlquist]

I think we will need @dondi to weigh in on dealing with the complexity of the state diagram.

As for the observation that the feature is available for unloaded or unweighted networks, I'm not 100% sure what you mean except that the radio buttons are visible no matter what the condition of GRNsight. I think this is what we want because you might have an unweighted graph loaded, but then change the radio button state because you intend to load a weighted graph next. I think the bother of "greying it out" for unweighted networks not worth the effort.

[dondi]

@dondi and @kdahlquist discussed this and affirmed that the state diagram remains useful and valuable but needs to be reorganized in order to stay feasible:

• There remains an aspect that is genuinely sequential; e.g., the state where "No diagram is open" transitioning to "There is a diagram open" via an Open or Import command. For these types of behavior, a manually maintained state diagram remains called for and should not get too unwieldy.

• An existing aspect of the state diagram involves autonomous/independent settings, i.e., values which toggle across various options, and can be combined with other settings arbitrarily. It is this aspect that makes the current state diagram unwieldy because with every new option (e.g., "Display weights," "Zoom level"), a combination with every other existing option produces new states and connections.

For this latter aspect, the proposal is to instead write a program that takes as input the available settings in GRNsight and their possible values (e.g., "Display Weights" ---> { always, never, mouse over }) then generate a state diagram programmatically. We can output this diagram in some file format and then leave it to an application that displays this format (e.g., yEd reading GraphML) to do the heavy lifting of layout and edge calculation.

There might be some exceptions where a property is inapplicable when another property has a certain value, and those can be coded in as well because they are just "skips" over certain combinations of properties and edges.

[eileenchoe]

Started some initial work/research on this. Here are some links to some articles I found I'll save here for later reference: http://ieeexplore.ieee.org/abstract/document/614080/ http://ieeexplore.ieee.org/abstract/document/989456/ http://barbie.uta.edu/~mehra/23_Generating%20Test%20Cases%20for%20GUI%20Responsibilities%20Using%20Complete%20Interaction.pdf

Basically, it seems that keeping track of the many states in GUI-based applications is a common problem, and one approach that seems similar to the one suggested by @dondi in the previous comment is the Finite State Machine approach. So, what I think we are ultimately should go after (and what is generally suggested by these articles) is to generate a finite state machine to be able to fully model the user interaction and the corresponding states. Not only do we end up getting a state model of GRNsight, but also a defined logical abstraction of GRNsight and its allowed behaviors.

I've started by defining what the parameters of the GRNsight "state" should be, and the list of interactions that we have at this point (load Weighted Graph, check "Restrict Graph to Viewport", etc.). I have a rough implementation so far which generates the states (JSON objects) programmatically given a sequence of interactions. There's a lot of thinking to be done on the specifics of how this is implemented, after some initial web searching I haven't found any libraries that can do the heavy lifting of the data structure which aren't full blown FSM frameworks like http://machina-js.org/, so I've started this homegrown version to test it out. @dondi what do you think of this approach? Any suggestions to the implementation?

[eileenchoe]

Another idea I had to implement this is based on how we are calculating how many "configurations" of GRNsight are possible. In this simplified example, I'm just considering Hide/Show Weights (3 options), Viewport Size (4 options), and Restrict Graph to Viewport (2 options -- which comes out to a total of 24 possible configurations. In order to generate these configurations (I guess now this becomes an algorithms/ data structures question) is the best way to construct a tree, and traverse to get the 24 unique configurations?

var grnsightOptions = {
    "Hide/Show Edge Weights": {
        "text": "The edge weight should ",
        "options": [
            {"Show With Mouse Over": "display when user mouses over an edge."},
            {"Always Show Edge Weights": "always be visible."},
            {"Never Show Edge Weights": "not be displayed."}
        ]
    },
    "Viewport Size": {
        "text": "The viewport size should ",
        "options": [
            {"Detected": "be automatically detected and set to an optimal size."},
            {"Small": "be set to small, unless this was the size automatically detected."},
            {"Medium": "be set to medium, unless this was the size automatically detected."},
            {"Large": "be set to large, unless this was the size automatically detected"}
        ]
    },
    "Restrict Graph to Viewport": {
        "text": "The bounding box should ",
        "options" : [
            {"Checked": "always be within the viewport."},
            {"Unchecked": "be allowed to be past the viewport"}
        ]
    }
}

[eileenchoe]

https://github.com/dondi/GRNsight/wiki/Client-Side-Testing-Document

[eileenchoe]

The test generator now exports a markdown file to be put in a wiki page (linked above). @kdahlquist and I have decided that since we have some UI updates coming up because of the new features, we can hold off on finalizing the testing plan for now. Still doing some thinking about the best way to implement the graphic part of the testing plan.

[kdahlquist]

Here is a scan of the handwritten notes that @kdahlquist and @eileenchoe made yesterday. GRNsight_Handwritten_Notes_20150518.pdf

[eileenchoe]

After including all the options from the notes above, my script can't actually run because my laptop runs out of memory space. There are 92,897,280 tests that should be generated. There's a lot of redundancy because some of the options can be selected from two locations (menu bar and side bar).

When I take out all the redundancies, demo file loading, zooming, and navigation, the script is able to run and generates 138240 tests, with each test about this large:

Test 1

Instructions:

• Load Graph - None
• Sliders - Keep at Default Values
• Lock Force Graph Parameters - Checked
• Reset - Click
• Undo Reset - Click
• Restrict Graph to Viewport - Checked
• Viewport Size - Detected
• Side Bar - Hide/Show Edge Weights - Show With Mouse Over
• File - Reload - Click
• File - Print - Click

Results:

• GRNsight should have no graph in the viewport
• The graph should configure according to its default values if there is a graph loaded.
• The sliders should be disabled.
• The sliders should revert to its original configuration, if Lock Force Graph Parameters is unchecked.
• The sliders should undo the effects of the reset, if the Undo Reset button is enabled.
• The bounding box should always be within the viewport.
• The viewport size should be automatically detected and set to small, medium, or large, based on the browser window.
• The edge weight should display when user mouses over an edge.
• The graph should center, zoom to 100%, and reapply the force graph parameters, if there is a graph loaded
• GRNsight should open the Print Dialogue Box, if there is a graph loaded

Are there further ways in which we can reduce the size of this document? Right now even my text editor is starting to crash when it tries to open the markdown file that I generate.

[kdahlquist]

My suggestion (which I think we talked about before) is to limit the actions to either three or four in a row. The example you gave above has 10 actions. Permuting 10 actions is going to result in a lot more permutations than just permuting three actions or four actions. It will still be a lot, but hopefully it won't crash your computer.

Obviously limiting testing to permuting 3 or 4 actions won't hit all sequences that could potentially be buggy, but we can be more thorough when testing these short sequences. If a series of 3 actions (or 4 actions) ends up generating a bug, then we can test further afield in terms of sequences that lead to that bug.

Unless a series of actions is something commonly done (in which case we will see it upon normal usage of GRNsight), bugs arising from a particular series of 10 actions are likely to be seen rarely in practice because a user will have to exactly hit those 10 in a row (if that makes sense).

[kdahlquist]

Also, as long as we have documented (written out the task and expected outcome) all the current options in GRNsight, I imagine we could tell the permuter to consider all pairs, threes, fours, etc. of actions without having to do all combinations.

[eileenchoe]

When considering a subset of the actions then, would we hold the values of the other actions constant? Or would the combination generator just be generating on those 3-4, and be blind to the options of the rest of the actions?

Would it be useful to also include a list of all the actions and their results ? Then even when a subset of actions are considered, we can still view a listing of all the options at the top of the document.

[kdahlquist]

Yes, when considering a subset, the others would need to be held constant. It might make sense to leave those other values at their "default" settings, unless that doesn't make sense with the current permutation in that it is only possible when some other setting is not "default".

It would definitely be useful to have a list of all the actions and their results! In some sense, this is user documentation because the goal for the user documentation is to describe all the actions that can be taken and their results. We can use this to make sure that our user documentation is complete.

[eileenchoe]

Ok, I'll figure out a way to select which subset of the features to include in the combination generator then as well as encode the default behavior for each action.

For the markdown testing document I'll include a header with some metadata which specifies exactly which sub-features are being tested in the tests generated as well as the list of all the actions and results.

[eileenchoe]

How about a table format for the header? The status corresponds to whether the option is enabled or not for the combinations which would be produced below.

GRNsight Option	Status	Action	Result
Sliders	ENABLED	Keep at Default Values	The graph should configure according to its default values if there is a graph loaded.
Sliders	ENABLED	Link Distance Increase	The graph edges should visibly decrease in length if Lock Force Graph Parameters is unchecked and if there is a graph loaded.
Sliders	ENABLED	Link Distance Decrease	The graph edges should visibly increase in length if Lock Force Graph Parameters is unchecked and if there is a graph loaded.
Sliders	ENABLED	Charge Increase	The graph nodes should visibly increase attraction with each other if Lock Force Graph Parameters is unchecked and if there is a graph loaded.
Sliders	ENABLED	Charge Decrease	The graph nodes should visibly decrease attraction with each other if Lock Force Graph Parameters is unchecked and if there is a graph loaded.
Sliders	ENABLED	Charge Distance Increase	The graph nodes should visibly increase charge distance if Lock Force Graph Parameters is unchecked and if there is a graph loaded.
Sliders	ENABLED	Charge Distance Decrease	The graph nodes should visibly decrease charge distance if Lock Force Graph Parameters is unchecked and if there is a graph loaded.
Sliders	ENABLED	Gravity Increase	The graph response to gravity should visibly increase if Lock Force Graph Parameters is unchecked and if there is a graph loaded.
Sliders	ENABLED	Gravity Decrease	The graph response to gravity should visibly decrease if Lock Force Graph Parameters is unchecked and if there is a graph loaded.
File - Print	DISABLED	Click	GRNsight should open the Print Dialogue Box, if there is a graph loaded
Viewport Size	DISABLED	Detected	The viewport size should be automatically detected and set to small, medium, or large, based on the browser window.
Viewport Size	DISABLED	Small	The viewport size should be set to small, unless this was the size automatically detected.
Viewport Size	DISABLED	Medium	The viewport size should be set to medium, unless this was the size automatically detected.
Viewport Size	DISABLED	Large	The viewport size should be set to large, unless this was the size automatically detected.
Lock Force Graph Parameters	ENABLED	Checked	The sliders should be disabled.
Lock Force Graph Parameters	ENABLED	Unchecked	The sliders should be enabled.
Load Graph	ENABLED	None	GRNsight should have no graph in the viewport
Load Graph	ENABLED	File Tab - Open Excel Workbook	GRNsight should generate a network graph from the Excel workbook given no warnings or errors are generated.
Load Graph	ENABLED	File Tab - Import SIF	GRNsight should generate a network graph from the SIF file given no warnings or errors are generated.
Load Graph	ENABLED	File Tab - Import GraphML	GRNsight should generate a network graph from the SIF file given no warnings or errors are generated.

[kdahlquist]

Generally yes on the format of the table. I'll try to carve out some time tomorrow so that we can edit the messages together.

[eileenchoe]

Sure, I've still been focusing on the combination/table generation and not the message data itself yet. I think now I have an option data representation standard mostly down we can start editing the messages

[eileenchoe]

@kdahlquist can we meet sometime today or tomorrow to edit the data for this one?

[kdahlquist]

@kdahlquist and @eileenchoe met this afternoon to review wording. Changes were made on the wiki version that @eileenchoe is now incorporating into her code.

What is left to do besides these changes:

• make the "enabled/disabled" the leftmost column of the table, calling the header "Include in Testing Protocol"
• change the header "Action" to "User Action"
• Include stuff in the format menu that is redundant with the sidebar
• Help menu
• Fit to window
• Normalization factor (set and reset)
• gray edge threshold
• make list of options that are available when no graph is loaded, when an unweighted graph is loaded and when a weighted graph is loaded. This would be a separate table since it is not part of the testing protocol.

Re-order table according to these notes:

20170531114058241.pdf

[eileenchoe]

Incorporated all of the wording changes, reordering, and additions. Using https://www.npmjs.com/package/markdown-pdf to convert md into pdf, but need to tweak a few things to get the table to generate properly. For the list of options, I think its also better to maintain this programmatically and regenerate the function availability chart with each new feature change, as its just one more piece of data to add to each option in the JSON file. For sake of time I've just outlined the start of this in the test generator to pick up at a later time. I've started to write out the list manually, and its in this wiki page. It's also linked in the wiki home page. Since I still have 8 hours on the Rains Research Grant, I can finish this one up this weekend, since I ran short on time today.

[kdahlquist]

@eileenchoe do you now need to add markdown-pdf to our list of library dependencies?

[eileenchoe]

Yes @kdahlquist I'm adding markdown-pdf as a dev dependency. Finished documentation for using this testing script in wiki.

[eileenchoe]

Updated the GRNsight Function Availability Wiki. @kdahlquist please review.

[eileenchoe]

@kdahlquist Please review the two wiki files:

• GRNsight Function Availability
• Client Side Testing Document

[kdahlquist]

I'm looking at this now; I'll try to be done by the meeting today.

One thing I will note right now, the filename for data.json is pretty vague. I know that it is in the testing-script-generator directory, but a more informative name might be nice, like feature-list.json or similar. Just my two cents.

[kdahlquist]

I'm reviewing GRNsight Function Availability and have the following observations:

• I think we need to be verbose and include functions multiple times if they appear in multiple places. For example, Undo Reset Force Graph Parameter Sliders appears as both a menu item and as a button.
• We also need to spell out each thing that needs to be clicked on, i.e., radio buttons for always show edge weights, always hide, and show upon mouseover.
• The D-pad control needs to be split into individual arrows and the center button.
• Sub-menus for Export data need to be added
• Import menu item and sub-items need to be added
• Demo menu items need to be added

[kdahlquist]

It would be a pain to reorganize GRNsight Function Availability, but it would make more sense to me if Unweighted graph loaded was the middle column.

Actually, it looks like the labels might be swapped.

[kdahlquist]

I'm wondering if the content of the GRNsight Function Availability should be added to the Client Side Testing Document page. It would be an even longer page, but it seems like having to maintain two separate wiki pages leaves open the possibility of forgetting and only updating one of them.

[eileenchoe]

Ok, so "GRNsight Function Availability" is now generated programatically and is included in the Client Side Testing Document . The script also date stamps when it is run, so we know when the contents of the wiki page was last regenerated.

[kdahlquist]

I deleted the GRNsight Function Availability wiki page because it is now superseded by the Client Side Testing Document. We don't want documentation that is out of sync. I'm closing this now.