Application Package standardization -stage in/out to platform [DESIGN]

[mike-gangl]

Application Package standardization -stage in/out to platform

Design work required to remove stage-in/out from applicaiton packages and migrate that fucntionality into the ADES/platform. This will A) align with best practices, B) allow more portability with the app package format, C) align with other app package centric system (e.g. MAAP).

Align with OGC Application Package Best Practices. e.g. decouple Stage-in and Stage-out from OGC Application Package itself. It implies that any A&A handling will be delegated to ADESes. and that CI/CD builds would need updates.

Acceptance Criteria

Acceptance criteria required to implement the epic

• Diagrams (Activity, sequence) showcasing the "new" flow app package execution
• Documented new roles/responsibilties for the MMO developers (e.g. SPS/DS collab) teams
• Documented what the new app package should look like after app-pack-gen service call
• Document Portability expectations of the new format (e.g. app packages look directly at a directory, so this can be run "anywhere")

Work Tickets

Link to work tickets required to implement the epic

• [ ] TBC (to be created)

Dependencies

Other epics or outside tickets required for this to work

• [ ] TBC

Associated Risks

links to risk issues associated with this epic

• [ ] As more projects get onbaorded, the technical debt accrued using the existing system increases

[mike-gangl]

Attached the slides that delineate changes needed. Met with ADS, DS, SPS 9/5 to discuss.

Application Package + Rearchitecture (1).pdf

Some caveats to the design

• Any url or STAC that needs to be staged must use the inputs name on job submission.
• inputs might eventually be an array of inputs (multiple stacs, multiple urls) that should be staged and added to a single directory and stac catalog
• stage input directory with generated STAC catalog will be mapped as a 'DIRECTORY' type, as specificed in the CWL accompanying the app-package
• the output of an app package is TBD- if there are outputs, it should be a DIRECTORY type with a catalog.json describing the contents. Can we call the directory "outputs" ? this way the stage-out part of platform always knows where to look for any output files.
• If there are no outputs... then what?