File / Folder UIs for Vulcan

[dtm2451]

Overall:

• Adds new Visualization 'from File' workflow, because optional primary inputs are not possible with our current setup
• Adds ui-input components for selecting a metis bucket + path to a file, folder, or either ("path").
• Adds validators for this input type which will complain if no bucket is chosen or if the user chooses a file/folder when the input requires folder/file (the opposite). The validator can also be given a regex to additionally require a path_regex match.
• Edits to underlying etna-js PickBucket and/or Pick(FileOr)Folder components:
  • To allow type-awareness of the vulcan ui-inputs, adds target type knowledge to the Pick(FileOr)Folder and optional send up of this knowledge.
  • To let the ui-inputs extend past input group boundaries which otherwise cut off dropdowns, often breaking usability, makes diablePortal={false} behavior optional for all involved Autocompletes.
  • Cleanup:
  • Fixes PickFileOrFolder allowFiles input to actually work
  • Use placeholder of disabled Autocomplete in Pick(FileOr)Folder when bucket not chosen, rather than just null / nothing
  • Fixes PickBucket to not give error before a bucket is chosen (via adding hidden '' option)
  • Have PickBucket only add paddingTop when given a label (Standard for all similar components I've made)
• Vulcan/Archimedes Orchestration:
  • Adds pass through of metis_host to archimedes orchestrator and runners.***
  • So can use as primary inputs in vulcan workflow CWLs, adds "MetisFile", "MetisFolder", "MetisPath", and "MetisCSVorTSV" nominal types to etna cwl definition
  • minor cleanup of a variable-name typo, commit_res*e*t_buttons

PR To Dos:

• [x] actually add folder & file or folder variant inputs
• [x] Manual Testing - Fix dev connections
• [x] CI Tests for input validators
• [x] Implement properly in the Visualization workflow
  • [x] allow 'File' / 'Folder' type as primary inputs
  • [x] Let the Primary input box grow to space needed
  • [x] Remove need to double-commit sometimes or re-commit after reload
  • [x] ensure placeholder text-box stays aligned

***ToDo via portainer:

• Ensure metis host exists in the production vulcan config

[dtm2451]

One BIG note (reminder to check, and do, if needed!): may need to add metis_host to the production vulcan config in portainer! Otherwise, deploy likely breaks all vulcan steps that run a script.

[dtm2451]

I think we discussed giving pointers to reviewers as focus/entry points... Primarily, I'd love if you ran the workflow:

1. Put some tsv or csv in dev-metis for a project
2. Follow code edits in this slack to let the dev systems communicate and use proper targets.
3. Try out the "Visualization from File" workflow!

Some minor questions:

• I think these are fine/good, but could be worth a double-check: some changes here make slight alterations to the look of the polyphemus metis loader UI (definitely a change when model added before bucket choice: placeholder change for PickFolder; unlikely a change, but would be always: PickBucket paddingTop now only if label is given, but is 'Bucket' by default)
• For metis_path_validators.tsx, should validator for 'MetisPath' input still check that type is either 'file' or 'folder' and not a null meaning undetermined by the component? My implementation skips the check, reasoning that the GUIs only allow for building of valid paths, rendering a check not required.

[maflaven]

I tried changing the metis file to see if it triggers a refresh on the "Review your data frame" step. It only refreshes if the review step is committed.

Review step won't refresh in this case.

[dtm2451]

(Edit: nolonger confused. You're seeing behavior that works entirely as expected!)

I'm confused by the error case you are seeing in https://github.com/mountetna/monoetna/pull/1249#issuecomment-1716105336 @maflaven.

Perhaps we can slack or zoom to figure out how you got to the state you are in as I can't understand how you got there. It's confusing to me because the way things should work, and do for me in the gif attached, is that as soon as you 'commit' to using that 'Pancan_samp...' file, the workflow should refresh and clear or fill in next steps based on matching hashes with the new workflow state. Admittedly, I could understand how this part of the state system works better, so I am very interested in understanding your workflow state as it seems like it does point to a bug.

For me, if I change the path / 'File or Folder' target & commit... Changing from the original ".tsv" to the ".csv" I had also previously used successfully, when I hit 'Commit' the "Review your data frame" remains because there is a past state matching the '.csv' file. (And it looks the same because the ".tsv" and "*.csv" files are equivalent.) If I instead change to the 'xneo1_diagtestnew.csv' file that I'd never used before, when I hit 'Commit' the "Review your data frame" step completely disappears as I expect it should.

[dtm2451]

Ohhhhhhh! not a bug case... the state did update successfully =). Notice in your image that it refers to 224 rows and 2 columns. I think that matches your file? Certainly doesn't match the test data I'd shared!