UWB-Biocomputing / WorkBench

Software and data provenance management platform for simulations of dissociated cortical cultures.
https://uwb-biocomputing.github.io/WorkBench/
Apache License 2.0
1 stars 3 forks source link

Give output xml of simulation better name #40

Closed jConquest closed 3 years ago

jConquest commented 4 years ago

in BaseTemplateDefault.xml in the target folder

it sets the default output name to be " tR_1.9--fE_0.98_historyDump_1.xml"

Given that this does not break the provenance backend (ie parsing output names in the turtle file) we should provide more meaningful name for default output file

jConquest commented 4 years ago

try updating name to "SimOutput" and test proVis can recognize the output file as desired (ie detects output as with original name).

This will require running at least a couple simulations and uploading them using ProVis to build a small network. Than determine if the output is being detected after provenance collection.

jConquest commented 4 years ago

Eventually update the output name to reflect the simulation name being run

jConquest commented 4 years ago

Do not assume particular extension, it can be either xml or more commonly hdf5

jConquest commented 4 years ago

Currently BrainGrid has two options for result file name specification. This needs to be resolved before worth working on this.

jConquest commented 4 years ago

Update to this card:

The name of the output file has been updated to use the input name and append "_historyDump"

This issue still requires future attention as BrainGrid is to have an update to output result specifications

stevecl5 commented 3 years ago

OK, I finally have a grasp of how document names and XML files are handled. In addition to changing the name of the simulator output file, I would like to reduce some unnecessary complexity and make things easier to understand.

Both the simulation config file and the simulation output file are stored in the project XML's simConfigFile field. For example:

<simConfigFile simulationConfigurationFile="results/historyDump_test.xml">
C:\GitHub\gr-wb\WorkbenchProject\target\projects\test\configfiles\test.xml</simConfigFile>

Notice that the simulationConfigurationFile attribute actually contains the output file name. I think a better option would be to have two different fields for simConfigFile and simOutputFile. For example:

<simConfigFile>C:\GitHub\gr-wb\WorkbenchProject\target\projects\test\configfiles\test.xml</simConfigFile>
<simOutputFile>results/historyDump_test.xml</simOutputFile>

Another issue is that simConfigFile uses an absolute path. Using a relative path instead would allow users to share projects more easily (I know that this is an issue in other XML files as well).

stevecl5 commented 3 years ago

The current naming rule for the simulator output file is: historyDump_<project>.xml

I would like to instead use: <project>_stateOutput.xml

Note that BaseTemplateDefault.xml calls the output file name stateOutputFileName. However, I noticed the configuration files in the BrainGrid repo use a mix of stateOutputFileName and resultFileName. I'm not sure which of these is more recent, but I personally find stateOutput to be more descriptive of the file contents. The output is also referred to as "state output" throughout the Workbench code.

stiber commented 3 years ago

Let's talk through this at our next meeting. There are a bunch of issues here and it would be good to come up with a solution that doesn't have to be reworked later on (or to decide we don't yet know all of the ramifications and have to think things through a bit and so we should come up with something flexible).

stevecl5 commented 3 years ago

Adding some info for future reference.

I traced the process of running the Simulation Starter which includes:

  1. getting default values from XML template files
  2. updating values from user input
  3. generating XML config files

Note: This is not a code snippet. Instead, it represents the approximate stack trace and relevant variable assignments for a specific process.

01 SimStartWiz constructor
02   calls workbenchManager.newProject()
03     npd = new NewProjectDialog(true)
04     initProject(npd.getProjectName(), npd.isProvEnabled())
05       projectMgr = new ProjectMgr(name, false)
06         initXML(rootNodeName) // creates project xml up to scriptVersion
07   calls configureSimulation()
08     calls workbenchManager.configureSimulation()
09       projectName = getProjectName()
10         returns projectMgr.getName() // the project name
11       configFilename = projectMgr.getSimConfigFilename() // always null at this point
12         returns getFirstChildTextContent(<project root element>, "simConfigFile")
13       iccsd = new InputConfigClassSelectionDialog(projectName, true, configFilename)
14         icm = new DynamicInputConfigurationManager(configFilename)
15           inputConfig = new DynamicInputConfiguration() // constructs xml doc from BaseTemplateDefault.xml
16         xmlDoc = icm.getInputConfigDoc()
17           returns inputConfig.getDocument() // sim config xml
18         loadParamsClassCBoxes(...) // creates class select drop-down boxes
19       icd = new DynamicInputConfigurationDialog(projectName, configFilename, iccsd.getInputConfigMgr(), ...)
20         xmlDoc = icm.getInputConfigDoc()
21           returns inputConfig.getDocument() // sim config xml with selcted classes
22         buildTabsGUI(xmlDoc)
23           // if (nodeName.equals("stateOutputFileName")
24           field.setText(setInitialOutputFilename())
25             returns "results/historyDump_" + projectName + ".xml"
26           stateOutputFileNameTextField = field
27           lastStateOutputFileName = field.getText()
28         configFilename_textField.setText(projectName + ".xml")
29         buildButtonActionPerformed(evt)
30           lastStateOutputFileName = ensureValidOutputName(stateOutputFileNameTextField.getText())
31           String fileName = configFilename_textField.getText()
32           fileName = icm.buildAndPersist(projectName, fileName)
33           lastBuiltFile = fileName
34       simulationConfigurationFile = icd.getBuiltFile() // file name as a String
35         returns lastBuiltFile
36       stateOutputFilename = icd.getStateOutputFilename() // file name as a String
37         returns lastStateOutputFileName
38       projectMgr.addSimConfigFile(simulationConfigurationFile)
39         setChildDataContent(project.getProjectData("simConfigFile"), "simulationConfigurationFile", filename)
40       projectMgr.setSimStateOutputFile(stateOutputFilename)
41         simulationConfigurationFile.setAttribute("simulationConfigurationFile", stateOutputFilename)
42   calls specifyScript()
43   calls generateScript()
44   calls runScript()
stevecl5 commented 3 years ago

Linked to Issue #5 and Issue #44. Leaving all three issues open since they are related but different.

stiber commented 3 years ago

The simulation starter outline is very good. Some form of it should probably be added to the programmer docs (which probably means we should add programmer docs).

stevecl5 commented 3 years ago

Changed to <sim_name>-out.xml in PR #97. This matches the existing output files in the Graphitti repo.