Controls table #93

Closed mwes closed 4 years ago

In GitLab by @jakebeal on Aug 21, 2019, 06:55

Measurements need a set of controls.

Typically, these will be positive and negative, but they may also include things like live vs. dead or multi-color controls.

Which strains should be marked as controls depends on the experiment plan, so we need an auxiliary controls table.

In GitLab by @jakebeal on Oct 14, 2019, 10:17

removed assignee

In GitLab by @jakebeal on Oct 25, 2019, 10:29

assigned to @mweston

In GitLab by @jakebeal on Oct 25, 2019, 10:30

First step: @mweston please make some example tables containing the information that you'd like us to be able to export.

In GitLab by @mweston on Oct 25, 2019, 10:55

At a minimum, this will need to include a set of conditions under which that control operates. These will need to be able to be specified, and vary, per measurement. Measurements can have multiple valid control definitions.

We currently support:

Timepoint(s)
Strain(s)
Inducer(s)
Control type
FCS channel, if applicable

This has covered all of the 4 NC NAND experiments from Ginkgo (Q2/Q3 2019) and the NC Titration experiment (Q2 2019)

Real example 1:

{
    "type": "EMPTY_VECTOR",
    "strains": ["MG1655", "MG1655_LPV3"],
    "timepoints": [{
        "value": 18.0,
        "unit": "hour"
    }]
}

Real Example 2:

{
    "type": "HIGH_FITC",
    "channel": "YFP-A",
    "strains": ["MG1655___with___RPU_Standard_Plasmid"],
    "timepoints": [{
        "value": 18.0,
        "unit": "hour"
    }]
}

Real Example 3:

{
    "type": "HIGH_FITC",
    "strains": ["MG1655_NAND_Circuit"],
    "timepoints": [{
        "value": 18.0,
        "unit": "hour"
    }],
    "contents": [{
            "name": {
                "label": "IPTG",
                "sbh_uri": "https://hub.sd2e.org/user/sd2e/design/IPTG/1"
            },
            "value": 0e-06,
            "unit": "M"
        },
        {
            "name": {
                "label": "L-arabinose",
                "sbh_uri": "https://hub.sd2e.org/user/sd2e/design/Larabinose/1"
            },
            "value": 0e-06,
            "unit": "M"
        }
    ]
}

These conditions are matched against the produced samples, and we automatically annotate the lab traces with the relevant control tags provided all of the conditions match. If specified in the structured request, neither the labs, nor TA1, has to do any kind of control annotation or inference.

In GitLab by @mweston on Oct 25, 2019, 11:00

For calibration, add three boolean columns (optional)

For Flow Measurements:

Fluorescence Calibration

Size Calibration

For Plate Reader

IGEM Calibration

Presence of True/true/Y/Yes, etc.in the column -> set the relevant property on the measurement as true, e.g.

"has_fluorescence_calibration":true

"has_size_calibration":true

"has_igem_calibration":true

Absence of column or presence of False/false/No/no, etc. in the column -> Do nothing.

In GitLab by @mweston on Jan 28, 2020, 17:52

@jakebeal @tramyn the target examples are still valid. This needs a UI representation. I can draft a prototype.

In GitLab by @mweston on Jan 28, 2020, 18:02

Drafted UI examples: https://docs.google.com/document/d/1hNwJiaFr2QVb8gpwIUHxh7ORRVDWxM9G_Tsm2LDVwI4/edit

Target SRs that have control structures (manually added until IP supports)

https://gitlab.sd2e.org/sd2program/cp-request/blob/master/input/structured_requests/y4d_cen_pk_inducible_crispr_characterization.json

https://gitlab.sd2e.org/sd2program/cp-request/blob/master/input/structured_requests/y4d_crispr_dose_response.json

https://gitlab.sd2e.org/sd2program/cp-request/blob/master/input/structured_requests/YeastSTATES-CRISPR-Short-Duration-Time-Series-20191208.json

https://gitlab.sd2e.org/sd2program/cp-request/blob/master/input/structured_requests/YeastSTATES-CRISPR-Long-Duration-Time-Series-20191208.json

In GitLab by @jakebeal on Jan 29, 2020, 13:41

assigned to @tramyn and unassigned @mweston

In GitLab by @tramyn on Apr 26, 2020, 22:36

mentioned in merge request !107

In GitLab by @tramyn on Apr 26, 2020, 22:36

created merge request !107 to address this issue

In GitLab by @tramyn on May 6, 2020, 20:57

@mweston This is what I am understanding from this issue request. Could you go over each bullet point and confirm my understanding of them?

I have a measurement table in a ER document that will introduce a new column called control. Each row under control will refer to a row in a Control Table. A user will have to understand this referencing of information between a Measurement Table and a Control Table. Is this correct?
Given a control table with beta_estradiol 0.05 micromole as the Content, can intent parser assume that the unit for the Content always refer to a fluid unit?
A control table template should provide a user the ability to insert optional columns. These optional fields are:
- Fluorescence Calibration
- Size Calibration
- IGEM Calibration

Control Type, Channel, Strains, Contents, and Timepoints are considered as required fields. Is this correct?

In GitLab by @mweston on May 6, 2020, 21:27

@tramyn

Bullet 1) Yes, each measurement row can have one or more controls associated with it. I would not recommend packing this all into a single column with multiple rows. I think that will be very difficult to parse/read. If you look at the example I posted, one, more, or all of the rows of that table can apply to a single measurement row. So we should think very carefully about this before making something that is potentially difficult to use.

A better solution would be creating separate control tables that can at a minimum cover what I posted in the example. Then, we would need to be to link those tables to measurement rows - e.g. some sort of reference or anchor. You could even re-use a common control definition across multiple rows.

Bullet 2) Yes, fluid unit is fine

Bullet 3) I would ignore the calibration requirement for now. We can address this in a future ticket. Calibration is not planned/managed by the ERs/XPlan currently; the decision to run calibration is handled by the lab (we originally thought when I wrote this ticket that we would handle this directly and declaratively in the ER). Having a few quarters of experiments under our belt now, we know this is not the case.

RE: your last point on required fields, good question. Only the control type and strains are required (a control type has to apply to one or more strains).

Schema:

https://gitlab.sd2e.org/sd2program/cp-request/blob/master/schemas/measurement-request-schema.json#L288

Those controls can optionally, additionally, be limited to combinations of time values and contents. The channel is a hint for Flow processing and is not strictly required.

In GitLab by @tramyn on May 8, 2020, 17:44

@mweston How about we make use of table captions to specify Table numbers? I've modified your Control Table example here. Please take a look.

If you notice, there is a column in a Measurement table called Control with values referring to different control tables base off of its table captions. Intent Parser can then parse for table captions and form the necessary relationship between a Control Table to a Measurement Table.

In GitLab by @mweston on May 8, 2020, 18:44

Nice, that looks pretty straightforward to me. It would require some enforcement/convention so that folks get the captions lined up correctly and don't make any mistakes/typos.

Maybe it would help if we use bookmarks/anchors to explicitly link between the sections in the document: https://developers.google.com/apps-script/reference/document/bookmark

I've updated your document showing this - creating bookmarks on table 1 and table 2, and linking them down in the measurement stable.

In GitLab by @tramyn on Jun 1, 2020, 21:39

@mweston, @jakebeal : I did some testing with this and it turns out that bookmarks are only accessible from Google App Script and not from Google Doc API. This will limit intent parser from generating a structured request with controls if @mweston wants to use intent parser REST API. A work around is to rely on table caption text instead of bookmarks to generate a structured request with information about controls. Bookmarks can still be used but this feature is accessible if a user wants to generate a structured request from the GUI.

In GitLab by @mweston on Jun 2, 2020, 06:47

@tramyn thanks. The main path for the intent-parser is through the REST API. We'll have to rely on the captions then - so no need to use the bookmark feature.

In GitLab by @tramyn on Jun 8, 2020, 17:58

closed via merge request !107

In GitLab by @tramyn on Jun 8, 2020, 17:58

mentioned in commit 3a1d9e3e17eaeb00268ddae2b350f853cbcbcbfb

In GitLab by @tramyn on Jun 12, 2020, 18:58

mentioned in commit 1688f6735562b8aa50071608f42bc449a8d8513c

SD2E / experimental-intent-parser

Controls table #93