Support for EMIT

[mike-gangl]

Support for EMIT

Following use cases

As an Algorithm developer, i want to make updates to an existing algorithm As an Algorithm developer, i want to test my algorithm in a unity jupyter envionrment As an Algorithm developer, i want to package my algorithm into a unity-application-pacakge As an Algorithm developer, i want to run my algorithm against a set 200 known data products as a regression

For the last one, we want to run the processes against a known set of data products that are used for validation. We can import these into the unity system as a collection. We will want to ensure it's simple to run that regression test against all 200 items. How would a project set that up (e.g. in airflow via a DAG? command line calls?)

Acceptance Criteria

Acceptance criteria required to implement the epic

• Emit Venue set up {project: emit, venue: dev}
• Emit Venue is set up including: Jupyter, Airflow, databucket and APIGateway integrations
• Cognito user groups for emit users
• Data Services - Ingest/catalog 200 data products for validation data. Ideally we have a mechansism to let suers do this on their own.
• Process assistance for running the 200 datasets from the processing system (e.g. how-to guide, code samples, etc).

Work Tickets

Link to work tickets required to implement the epic

• [x] https://github.com/unity-sds/unity-data-services/issues/352
• [ ] https://github.com/unity-sds/unity-data-services/issues/354

Dependencies

Other epics or outside tickets required for this to work

• [ ] TBC

Associated Risks

links to risk issues associated with this epic

• [ ] TBC

[galenatjpl]

@mike-gangl moved to 24.3. Almost there!