Notification of incomplete projects

[parthshahva]

As a DCP data wrangler, I would like to be notified when a project has been in an incomplete state after a certain period of time (and which component is responsible) in order to triage the issue. (stretch) I would like for tickets to be automatically created in the relevant repo and the wrangler tagged

[mweiden]

@parthshahva is there some overlap between this and the report? https://github.com/HumanCellAtlas/data-monitoring-dashboard/issues/6

[parthshahva]

Nope this ticket is referring to either slack notifications or github issue creation. Ticket #6 is a recurring data report that will be send to the dcp via email

[mweiden]

OK, I see. ty for the clarification.

On Thu, Aug 1, 2019 at 3:37 PM Parth Shah notifications@github.com wrote:

Nope this ticket is referring to either slack notifications or github issue creation. Ticket #6 https://github.com/HumanCellAtlas/data-monitoring-dashboard/issues/6 is a recurring data report that will be send to the dcp via email

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/HumanCellAtlas/data-monitoring-dashboard/issues/26?email_source=notifications&email_token=AAEDOWCX2HI4SMFFO2IXOELQCNQRRA5CNFSM4H5AEWNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3MCQGQ#issuecomment-517482522, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEDOWES6KMLA73B4PXTZHLQCNQRRANCNFSM4H5AEWNA .

--

Matt Weiden Engineering Manager

https://chanzuckerberg.com/?email

c: +1 650-8 <+16509477330>85-0025 mweiden@chanzuckerberg.com http://chanzuckerberg.com http://chanzuckerberg.com/?email

[parthshahva]

FOR NOW:

Email (similar to monthly report) sent at 7am PST, noon, 3pm PST. Weekdays only.

data-operations-team@data.humancellatlas.org

[parthshahva]

Copying from slack conversation with @jahilton

1.Data Wrangler creates Epic in hca-projects repo when a dataset is selected from the list & an email is sent to the potential submitter Info to include in Epic: species, broad assay category, assay, technology & PE/SE, organ, seed network grant ID, paper DOI, contributors, link to wrangling tracker (when available)

2.Data Wrangler updates the Epic Title to match the Project Title (machine readable) and submits metadata spreadsheet & raw data Tracker updates the Epic when the spreadsheet & raw data have been submitted

3.Tracker updates the Epic when all bundles have been successfully exported. The Tracker adds the project uuid to the Epic. (Would the Epic be a logical/desirable place for the Data Wrangler to attach the post-submission spreadsheet?)

4.Tracker updates the Epic when all workflows have started

5.Tracker updates the Epic when all workflows have successfully completed

6.Tracker updates the Epic when all analysis submissionEnvelopes have been successfully exported

7.DataOps notifies MatrixService to ‘go’ (via a matrix-service ticket linked to the Epic)

8.Tracker updates the Epic when bundles have been indexed at MatrixService

9.Tracker updates the Epic when the project matrix is generated

10. DataOps completes QA checklist & updates the Epic (tagging the primary wrangler in the comment)

11. Data Wrangler notifies the submitter & closes the Epic

(is this also the time for the Data Wrangler to send UX the email for post-submission survey?) DataOps will link any troubleshooting tickets to the Epic along the way The tracker will display a link to the Epic AUDR/re-ingest needs: DataOps or Data Wrangler re-opens Epic & links appropriate ticket(s) to Epic The Epic is updated with a comment that an update submission was done (and each step happening afterwards - 2ryA, matrix…) New analysis pipeline needs: DataOps re-opens Epic & links pipeline ticket to Epic

[parthshahva]

Github integration turned on in production. Epics and comments can be found at https://github.com/HumanCellAtlas/dcp/issues?q=is%3Aopen+is%3Aissue+label%3Ahca-project