matchings
commented
11 months ago The infrastructure to do QC on individual data streams has been implemented in mouse QC architecture but is not being used in practice. For end users to use the information on specific datastream pass/fails, it will need to be added as metadata in the schema and as a tag in Code Ocean on specific data assets.
Leaving this open as a reminder to consider during the migration of QC to Code Ocean. @seanmcculloch
Description In the ongoing efforts to overhaul mouse-seeks, and taking into account the outcomes of previous qc-athons and data releases we’ve been having some deeper discussions about our qc process. We would like to explore the possibility of having a more modular qc system, where we can pass or fail individual data streams independent of one another.
For example, if only the behavior video fails, we would have the ability to fail just that data stream, but pass all the others (physio, eye tracking etc. ). Obviously this has some fairly large implications and could require some big adjustments to the SDK for data releases depending upon how it is implemented.
This will require coordination between the scientific team, qc ops team, and the technology teams
Related issues Provide links/ issue numbers to related Issues or Bug Reports or external sites that might document this lack of information.
Describe the solution you'd like We can create the ability to pass, flag and fail individual datastreams in the mouse-seeks qc system, and then pass that information off to the technology team before the data release. This would obviously be a more complicated system than we currently have so we would need to develop a specific process to make sure all needed information was provided in a timely and documented manner.
This may also require changes to the SDK, not only to deal with partial datasets but also to communicate these changes and updates to our end users.
Screenshots or example plots Provide any screenshots, example plots or other visualizations that are helpful in describing the issue or the suggested solution.
Scope This impacts all phase 4 projects that use the mouse-seeks qc system to determine it's data release candidates. Stakeholders include @matchings @pgroblewski @everythingevolves @mabuice
Additional context & analysis Modular_QC_Proposal.pdf
Attached is a PowerPoint presentation that was presented at a QC stakeholders meeting that broadly discussed the issue of a more modular qc report system with the ability to pass/flag and fail individual data streams.