UC5.1 A Product Team (PT) is convened to define the required input data sets, processing algorithms, output definitions and QA.
UC5.2 The PT, Community and SM agree that the ARD products will be supported through community funding, to be free at the point of use, and accessible from the EODH.
UC5.3 A Deployment Team (DT) takes requirements from PT and uses RC, WAS and HW to develop a multi-step workflow that produces multiple ARD products and includes additional Quality Assurance and Tagging workflows.
UC5.4 DT defines the resource requirements to process 50TB of input data to generate the ARD products. SM allocates appropriate space in the ARD cache.
UC5.5 DT tests and deploys HW using WAS.
UC5.6 WR runs the entire HW and generates outputs.
UC5.7 Intermediate QA and Tagging workflows are triggered by the generation of data using the ENS.
UC5.8 WR and WAS provide suitable logging and error reporting to enable re-runs and de-bugging of failed runs.
UC5.9 ARD products are published to the RC, according to the agreed rules of visibility and access.
UC5.10 RC product records include a full provenance chain of data inputs as workflow steps. QA and Tagging records add extra information and metadata to improve search and discovery of appropriate data.
UC5.EX1a A third-party may decide to build a new web-application that exposes or exploits the new ARD products.
UC5.EX1b Optionally, near real-time running of the HW is enabled through the ENS. In support of this new HW, an authorisation access policy is created.
Requirements
R3.24 Annotation capability: The catalog should include the capability for annotations to be connected to records, at all levels of the hierarchy. The initial annotation capability will support linking with QA results.
R3.25 QA results catalog: The results of QA workflows should be recorded in a catalog, which may optionally be part of the main data catalog or implemented separately.
R3.26 QA interface with data catalog: The results of QA workflows should be searchable as part of the main data catalog. The visual interface should present QA results graphically (such as a "Passed EODH QA") in a form that can be clicked to investigate more detailed results and links to the documented QA processes that were run.
R3.27 Annotation visibility: The catalog should allow different types of annotations to be actively managed by the search system. Supported annotations, such as the QA results, should be switched on by default, whereas user-defined annotations might only become visible when a search query actively enables them.
Note that the ITT includes the following:
Use Case 5: ARD Pipeline with QA
Requirements