Execute and Deploy untiy app package in unity-venue-test environment

[mike-gangl]

This workflow will follow the "happy path" listed on https://lucid.app/lucidchart/42e6a4fa-a0b6-48ae-99db-784eed519b4c/edit?page=gtwMI5NZgBEf&invitationId=inv_083b1130-a8dd-4926-abab-8d0f377b4b07#

Execute an SBG application package on the untiy-venue-test airflow system

• get endpoint from https://unity-sds.gitbook.io/docs/mdps-overview/endpoints (User Interface: Airflow UI)
  • Login (request username, password from Luca if you cannot find it). This is NOT your untiy username/password right now.
• Follow Documentation at https://unity-sds.gitbook.io/docs/mdps-overview/tutorials/executing-a-job/executing-jobs-via-ui
  • Find the app package to use (cwl files)
  • Determine the inputs to use (e.g. CWL input yml)
• Monitor Run
• View search results in Jupyterhub
• Download 'data' to jupyter hub environment

[mike-gangl]

We'll need to work with the various teams to do the monitoring, viewing search results and downloading the data. If any issues come up, we'll need to make sure we're working them in the unity-all channel so that we get insights and fixes made.

I have concerns over the right cwl_dag (generic) being deployed to the airflow environment. I have concerns over the ability for SPS to write to venue bucket (permissions) I have concerns over the ability of Jupyter to read from the venue bucket (permissions)

[rtapella]

https://docs.google.com/document/d/1WhYS764bWqjlpN1-5fWme703RmePYrg0/edit

[rtapella]

I have had issues testing this for a week and a half. Will try testing again this week

[rtapella]

I think this is about ready to close, and we'll keep updating it over time. Now that emit/dev is up, we should update the guide for that team and test it, then release it to them