search

metamorph-inc / run_mdao

2 stars 0 forks source link

OPENMETA-708 PET CSV Recorder Ignore Missing FileRef Objectives #6

Closed josephcoombe closed 4 years ago

josephcoombe commented 4 years ago

...What does the visualizer do if an artifact that it's expecting isn't present?

Currently, the Visualizer code dealing with artifacts doesn't use any prior knowledge (e.g. from mdao_config.json) about what artifacts to expect.

It just unzips every artifact folder (if present) and if it finds a file matching certain patterns - e.g. images.zip containing .png/.jpeg - then it display said file. https://github.com/metamorph-inc/openmeta-visualizer/blob/2944900921197e51b7408ff37bc85f798c4e8524/Dig/tabs/Explore.R#L686-L706

This PR worked with images.zip files. I will look at whatever other output/file-type-specific Visualizer code there may be lurking... Also test what happens if an artifact is present for some iterations but not for others.

ksmyth commented 4 years ago

Maybe it should crash if analysisErrorOccurred is False and the file doesn't exist? Thoughts?

josephcoombe commented 4 years ago

From discussion at stand-up:

Regarding whether a PET should crash if the Objective FileRef does not exist and analysisErrorOccurred is False (alternatively raise an AnalysisError if the Objective FileRef does not exist)

... I think analyses with Design Spaces are a reason to not do either of the above.

In the image below, the HasEngine Test Bench creates an output FileRef only if the Design Space has an internal combustion engine (an alternative / optional).

IMG_20200228_131942

In this case, analysisErrorOccurred can still be False.

And, I think it is unnecessarily burdensome on the user to require him to create a placeholder file for configurations w/o an engine. The Visualizer also gets cluttered because we either have to tag "placeholder files" to be ignored or we're displaying blank engine.txt files in the Point Details tab for configurations w/o an engine.

josephcoombe commented 4 years ago

@ksmyth suggested a warning/notice when the CsvRecorder does not find a FileRef