Have an ability to run the previously run analysis

[joeseibel]

There should be a way for OSATE to remember the last analysis which was run and which model it was run on so that a user need only to click a button to rerun the previously executed analysis.

One possible approach is to turn the analyses into run configurations instead of having them be actions or commands in the menu/toolbar.

This request came up during the AADL standards committee meeting.

[philip-alldredge]

I missed that portion of the committee meeting, but I like the idea of being able to run them from a launch configuration. It would open up many new capabilities like using them when launch groups and saving launch configurations to file.

[AaronGreenhouse]

What analyses do we have?

• Analyze Bus Load
• Analyze Power Requirements
• Analyze Resource Allocations (Bound)
• Analyze Resource Budgets (Not Bound)
• Calculate Weight Totals
• Model Statistics
• Find Unused Classifiers
• Run Fault Tree Analysis
• Run Functional Hazard Assessment
• Analyze Fault Impact
• Check Unhandled Faults
• Bind and Schedule Threads
• Check Rate Monotonic Priority Assignment
• Schedule Bound Threads
• Check Flow Latency

What kind of inputs do we have? -- Influences how we run the analysis

• Package file (no dialog or other inputs) (NOT package object in the outline view)

  • Find Unused classifiers

• Instance model file (no dialog or other inputs) (NOT system instance object in the outline view)

  • Model Statistics

• Any instance object (Converts to System Instance object)

  • Analyze Fault Impact
  • Check Unhandled Faults

• Instance Model file or Any instance object (Converts to System Instance object) + workspace preferences

  • Run Functional Hazard Assessment

• Any instance object (Converts to System Instance object) + Dialog box + workspace preferences

  • Run Fault Tree Analysis — contents of the dialog box depend on information extracted from the instance model!

• Instance model file or System instance object + Dialog box + workspace preferences

  • Check Flow Latency

• Instance model file or System instance object + Dialog box

  • Bind and Schedule threads

• Instance model file or System instance object

  • Check Rate Monotonic Priority Assignment
  • Schedule Bound Threads
  • Analyze Bus Load
  • Analyze Power Requirements
  • Analyze Resource Allocations (Bound)
  • Analyze Resource Budgets (Not Bound)
  • Calculate Weight Totals

What kind of outputs do we have? -- Shouldn't affect how the analysis is run, but interesting to know

• Markers only
  • Find Unused classifiers
  • Check Rate Monotonic Priority Assignment
  • Schedule Bound Threads
• Markers + .CSV file
  • Analyze Bus Load
  • Analyze Power Requirements
  • Analyze Resource Allocations (Bound)
  • Analyze Resource Budgets (Not Bound)
  • Calculate Weight Totals
• Dialog box
  • Model Statistics
• .CSV file
  • Run Functional Hazard Assessment
  • Analyze Fault Impact
  • Check Unhandled Faults
  • Check Flow Latency
• .faulttree file
  • Run Fault Tree Analysis
• Not sure
  • Bind and Schedule threads

[AaronGreenhouse]

I don't think the built-in Eclipse launch configuration mechanism the appropriate because it is meant to actually execute an external program. It provides run, debug, and profile modes. These don't make sense for us.

See https://www.vogella.com/tutorials/EclipseLauncherFramework/article.html

We probably want something similar, a mechanism that

• provides a standard UI for configuring an analysis
• allows analysis configuration to be named and persisted across OSATE runs
• remembers that last analysis configuration performed

But I think it's pushing the semantics to directly piggyback on the launch configuration extension point.

[AaronGreenhouse]

Question: What role should workspace preferences play in analysis configuration?

Fault tree generation, Flow latency analysis, and Functional hazard assessment make use of workspace preferences. Fault tree analysis even uses project-specific settings.

• FHA uses the workspace preference as its only configuration. It is obtained by the command handler, and then used as an argument to the class that performs the FHA task.

• Fault tree generation gets other settings from a dialog box (this is another problem discussed later). The generation process is not parameterized by the value workspace preference/project property. Instead the generation process is hardwired to repeatedly call FaultTreeModel.getPrecision().

• Flow latency uses the workspace preferences to initialize the default values in its configuration dialog box only, and to determine if the dialog box is shown at all.

I would argue that the way workspace preferences/project properties are used in fault tree generation is horrible for a variety of reasons. But from the consideration of "rerunning an analysis", it does not provide for a consistent way to rerun the generation. Changing the preference value between runs would change the manner in which the fault tree is generated.

In FHA, the workspace preference could easily be replaced by a dialog box that is presented at the start of the analysis.

In flow latency, the preferences have the least impact. As mentioned above, they are used to set the defaults for the dialog box. This is definitely the model that should be emulated, if any.

In summary, I think the correct role for workspace preferences/project properties is for them to determine the default settings of any parameters to the analysis. Most likely these parameter would in general be configured using a standardized dialog or pane that is used by the analysis "service."

We should absolutely avoid doing what fault tree generation is doing.

[AaronGreenhouse]

Dialog boxes are used by

• Flow latency analysis
• Bind and schedule threads
• Fault tree generation

The dialog boxes are used to get additional parameters for the analysis. These can definitely replaced by some kind of standardized dialog/pane that can be be used remember the analysis settings.

One interesting thing is that fault tree generation requires the user to choose a "state name" as input, and these states are taken from the model. Being able to populate the dialog/pane using information from the model needs to be taken into account in any future mechanism. In this case the resulting input is a name as String. It is possible the name won't exist anymore the next time the analysis is run. Rerunning the analysis based on the old settings would have to detect this case. Basically the analysis would have to either validate it's settings before starting or be able to tolerate bad settings, e.g., by terminating gracefully with a controlled failure.

[AaronGreenhouse]

Any analysis of a model is going to run over the instance model. But we do want to be able to run over declarative model components: the Find Unused Classifiers analysis runs over the declarative model. This doesn't analysis properties over a model, but rather deals with properties of the model "source code." There are no extant examples of similar analyses right now, but it is easy to imagine that there could be more in the future.

[AaronGreenhouse]

This issue probably has implications for #2384.

[AaronGreenhouse]

Went looking for how plugins are allowed to save state, because this information doesn't seem like it belongs in workspace preferences or project properties, which both seem to be based only on key–value pairs.

See: https://wiki.eclipse.org/FAQ_Where_do_plug-ins_store_their_state%3F

Plugins have a state location that is unique to a workspace into which they can store arbitrary files. This location is obtained by calling the method getStateLocation on the Plugin instance.

Point here is that we don't have to try to squeeze the analysis configurations into preferences or properties. We can create our own data file to save the information.

[AaronGreenhouse]

Having a way to rerun the analysis doesn't mean we cannot keep launching analyses the way we do now, e.g., via commands in the outline and navigator menus. It doesn't mean special dedicated dialogs always have to be used.

We need to have a standard way for the analysis to

• record that it ran
• record what its settings were

Then we need ways to

• access the analysis history
• view the settings of the historical runs
• clone/edit the settings of a historical run
• name a historical run
• remove a historical run
• add/remove runs to a favorites list

Should an analysis be able to provide a "blank template" for an analysis run so that a analysis configuration can be created from scratch without running a prior analysis?

[AaronGreenhouse]

As an API detail, it may be worth exploring having the analysis be able to marshal/unmarshal or be able to pickle/unpickle its own settings. This implies the analysis would be responsible for building its own "Settings" pane as well. This avoids having to save the settings as yet another form of attribute–value pairs.

A–V pairs drive me nuts because they are hard to type check and they constrain what kind of inputs are available.

[AaronGreenhouse]

We need a way to identify analyses so that they can be found and executed.

[AaronGreenhouse]

Discussed this @lwrage and we want to keep this as simple as possible:

• Just keep a list of the last N run analyses (where N is to be determine later). This list does not need to persist between runs of OSATE.
• No need to edit the past analysis settings
• Do need to be able to see what the old settings are.
• Don't worry about settings that come from workspace prefs or project properties.
• Only focus on real model analyses that run over instance models. If it happens to be easy to handle declarative models as well, then might as well do so, but if it complicates things too much, then don't worry about it. (Sorry "Find Unused Classifiers").

[AaronGreenhouse]

From an API and implementation point of view, we have the problem that currently there is no entity that represents an analysis. We have abstract classes that are used to implement AADL-aware commands, but they don't have to be analyses, and as a command typically hand off actual processing to other classes.

Here I think we need to introduce a Java interface for analysis. It needs to have a method that takes a configuration/picked state object and executes the analysis using it. Invoked this way

• The analysis never opens a dialog box to get settings as all the settings come from the configuration object.
• If the instance model no longer exists it should be instantiated from the original declarative model (assuming that still exists).
• Question: Do we reinstantiate the instance model if it already exists? Seems like the answer should be "yes" as the point here is to save time for the user. The counter argument would be that it might be possible for the user to modify the instance model directly, and that the user has update the model in this way since the last analysis, and reinstantiating the model would overwrite these changes.

Actually, the instantiation and possible reinstantiation should not be handled by the new analysis instance, but by the components that manage the "last used analysis" menu.

The pickled state object needs to be able to provide a human-readable description of the contents so that the actual parameters of the analysis can be shown to the user. This might a string, or maybe a UI pane so that it can be displayed in a standard dialog.

When an analysis runs (no matter how it runs), it needs to report that it ran and provided a pickled configuration to some central bookkeeper.

[AaronGreenhouse]

Missed these analyses some how:

• Check A429 Connection Consistency
• Check Port Connection Consistency
• Check Binding constraints
• Check Connection Binding consistency
• Check codegen prerequisites for POK
• Check codegen prerequisites for DeOS
• Check codegen prerequisites for VxWorks

These just take an instance model as input and do not present a dialog box or have other configuration options.

[AaronGreenhouse]

How to make a dynamic menu (toolbar or otherwise) that would show the most recent list of recently executed analyses:

• https://www.vogella.com/tutorials/EclipseCommands/article.html
• https://stackoverflow.com/questions/29391336/eclipse-how-to-get-the-menumanager-for-a-specific-menu-id-defined-in-plugin-xml
• https://wiki.eclipse.org/Menu_Contributions/Populating_a_dynamic_submenu
• https://wiki.eclipse.org/FAQ_How_do_I_make_menus_with_dynamic_contents%3F

[AaronGreenhouse]

Summary of points made in previous comments:

We want to make as few changes as possible to existing analysis implementations.

• Should avoid requiring classes to implement new interfaces.
• Should provide new "execution" objects that run the analysis on a given input configuration. These objects can create whatever steps are actually require to run the analysis.
• Ignore details of configuration that come from workspace preferences or project properties.
• Need a place to register that an analysis was executed. As this is meant to be a UI shortcut, this belongs in the UI plugin somewhere.
• Want to use "pickled" state objects because attribute–value pairs are annoying and don't type check nicely. This allows arbitrary information to be stored by the analysis and doesn't constrain what future analyses might need.
• Do need to be able to have human-readable form of the pickled state so the UI can show the details of how the analysis is going to invoked.

[AaronGreenhouse]

Step 1: Add a dynamic menu in a toolbar to the UI.

[AaronGreenhouse]

Advanced: analysis executor provides an image descriptor.

[AaronGreenhouse]

See: https://github.com/osate/osate2/wiki/Creating-a-Dynamically-Populated-Toolbar-Menu

[AaronGreenhouse]

Going to start by "upgrading" the Flow Latency analysis because this analysis uses a dialog box to get inputs, and I know what it's supposed to do.

[AaronGreenhouse]

Now that I'm defining some interfaces for this, it makes more sense for the configuration state to be part of the "execution" object.

[AaronGreenhouse]

As always, I forgot about modes and SystemOperationMode. Problem is there doesn't seem to be standard way of handling this for analyses in general. Currently every analysis does it's own thing, e.g.,

• Bus Load Analysis always analyzes in every SOM

• Flow Latency Analysis asks the user what to do, and can analyze in a single specific SOM, or in all of the them.

So for now, SOM needs to be part of the configuration state saved by the analysis.

Addendum: This is actually quite awful because the driver for looking at the different system operation modes is in the AbstractInstanceOrDeclarativeModelModifyHandler class. I was hoping not to have to modify this class considering that I'm trying to get rid of it, but it cannot be helped.

[AaronGreenhouse]

The configuration of an analysis can include multiple instance models:

• A single invocation of Bus Load Analysis can analyze multiple models. The configuration needs to expose all of them.

The model URIs are distinguished in the runner configuration because I want the command that invokes the runner to take care of reinstantiating the models before rerunning the command. So I need to expose a list of URIs, not just a single URI.

[lwrage]

See also #2329 about SOM selection. This should be unified across all analyses.

[AaronGreenhouse]

SOMs definitely complicate this issue, as current behavior of analyses is all over the place. Need to "step back" and review the overall analysis situation.

Started internal wiki page to discuss this: https://wiki-int.sei.cmu.edu/confluence/display/AADL/OSATE+Analysis+Execution