Provisional Classification Summary - Case Level Evidence

[jimmyzhen]

Create a table view for all Case Level evidence (Individual evidence primarily since other types of evidence can't be scored) associated with the provisional classification.

Table header title "Genetic Evidence: Case Level (variants, segregation)

Table columns include:

1. Variant type
2. Variant
3. Reference
4. Sex
5. Age
6. Ethnicity
7. Phenotypes
8. Associations
9. Segregations (sub-column - Number of affected, Number of unaffected, LOD score)
10. Previous testing
11. Methods of detection
12. Score (sub-column - Proband (default), Seg)

Sortable columns include:

1. Variant type
2. Variant
3. Reference

Could they also include:

4. Sex
5. Age
6. Ethnicity
7. Methods of detection
8. Score (depending on east of implementation)

[selinad]

@jimmyzhen @gto389 -- would be great to look into OLS API for returning term names for HPO IDs (one of the comments on mockup) -- this would be very valuable for expert reviewers and curators.

[SubmarineSammitch]

Current progress with the summary table.

[jimmyzhen]

@selinad, @wrightmw,

As we discussed today, the *Associations** column won't be included in the Case Level table for the time being.

As for the Variant column, bringing in the population data for the given variant won't be trivial due to the following:

1. the variant may not have any associated interpretations.
2. the variant may have multiple associated interpretations, but none of them belong to the currently logged-in user.
3. the variant may have an associated interpretation belonging to the currently logged-in user, but it does not yet have any evaluations (FYI: population data won't be saved until an evaluation is saved).

As for the Segregation Score column (last column), the segregation points are assigned given the total LOD scores of one or more families. But since each scored proband has only one associated family, and hence only one LOD score is represented. Would it makes sense to assign segregation points to just one LOD score? Or am I misinterpreting the specs?

[selinad]

Thanks, @jimmyzhen -- makes sense.

The spec is for the LOD scores to be summed across the record and then the points assigned, so it would be confusing to assign points to an individual LOD score for a family.

[wrightmw]

@jimmyzhen Comments on current instance:

1. Add 'Close' button to the Evidence Summary page
2. Case control is currently adding up incorrectly in the Evidence Summary table. I had total points 1.5 and it showed as 6 points, therefore multiplying the score by 4.
3. Add "Modification table data" to Evidence Summary

[selinad]

HI @jimmyzhen looking fantastic! So much fun to see this take shape. Thanks for all your hard work.

Here are a few things:

1. For "Date Provisional classification saved" -- if the user does not check to save as provisional, we would not want this text.

How about having "Status" say "Classification status:" (this will be the status of the user's summary, not the record status), and underneath it will say "Date Classification Saved:" e.g.

Classification status: in progress (or Provisional) Date Classification Saved: [timestamp]

Maybe we want to change "Summary" to Classification for the owner, too -- e.g. "Classification Owner"; @wrightmw we should look again at our use of "Summary" and "Classification"

2. It would be great to provide some hyperlinks -- e.g.

• variant hyperlink to ClinVar or CA ID
• Disease ID to OLS entry

3. Re scoring -- it definitely seems to be doubling everything -- e.g. doubles the count and the score for all the fields I entered.

4. For the Experimental -- I put in one A and one B Biochem function evidence, but both say "(Gene(s) with same function implicated in same disease); these 2 should have different descriptions.

5. Small detail: would be great to have the top table stand out more somehow.

The data I entered is displaying well -- will go back and add some more -- exciting!

---

@selinad I'm commenting below your comment to keep my response close to your question. Let me know if you find this confusing and I will stop this form of communication. I thought we were going with "Classification Summary" and "Evidence Summary", however I agree that these are fairly similar terms. I think we also need to think about the terms we currently use in the VCI, we don't want to diverge too far from these. However... how about "Classification Matrix" and "Evidence Summary" as an alternative?

[selinad]

Hi @jimmyzhen -- more feedback based on testing

For variant:

• "de novo" should be italicized
• for phenotypes, would it be possible to put a title in front -- e.g. "HPO term(s): HP:nnnnnnn, HP:nnnnnn; free text: blah blah"
• as a double-check, the only columns coming from Family are under Segregation, correct? (all other info from the proband)
• if there is no segregation info, should we put "-" in the space?
• for LOD score, would it be reasonable to show published or calculated in front of value? e.g. "Calculated: 1.45" or "Published: 1.25" -- could implement the same way for segregation without a proband (i.e. combine the 2 columns and display in one as above, for consistency)
• It would be great to add Proband in the column headers for the following: Sex (e.g. "Proband Sex" or "Sex (proband)" ), Age, Ethnicity, Phenotypes, Previous testing, Methods of detection.
• for Age, can we put "Type: age" (e.g. "Onset: 66 years")

For Case-Control:

• See new mockup on the Google doc for the latest columns. One note: the detection method will be coming from a free text field and there are changes to the columns. The good news is the display currently under Power be the same but under a different column
• can you stylize the relative risk as "Relative risk: 33"
• I think confidence interval and p-value are displaying in same column. Confidence interval should display as "1-5 (%)" in above fake example

Still adding more data -- looking good!

[selinad]

@jimmyzhen I think you reminded yourself of this -- need Score status column -- the second one is a "Review" in this table, but not clear:

[selinad]

@jimmyzhen For family segregation info with no proband:

Is it reasonable to add the following columns -- a reference is definitely needed

• Reference
• Family Ethnicity (or "Ethnicity (Family)", if that is what you do for probands)
• Family Phenotypes
• Segregations as you have it in case level (e.g. # aff, # unaff, LOD score, counted)

Maybe change "Case Level (segregation without proband)" to "Case Level (family segregation information without proband data)"

Also, when there is no evidence for this section, it says something about no Experimental evidence, it should say "No segregation evidence for a Family without a proband was found" -- along those lines, we should remove the word "scored" as we will be showing evidence for review and contradicts.

[selinad]

@jimmyzhen for contradicts, "Contradicts" will go in status column and I think we should do n/a in score column? Maybe in red? Or contradicts in red?

---

@selinad I very much like the idea of putting contradicting data in red.

[selinad]

@jimmyzhen Here is what it looks like for experimental:

• I added expression B but did not do a score choice -- I think it's OK it doesn't show ( @wrightmw let me know your thoughts)
• When I do get expression B to show, it says the same thing as A -- "Gene normally expressed in tissue relevant to the disease" It should say "The gene is altered in expression in patients who have the disease" (same problem observed earlier for Bioch function)
• for Biochem A and B and Expression A and B can the category value be "Biochemical Function A", etc. with appropriate description following. Also, would prefer this styling: "Biochemical Function A: Gene(s) with same function implicated in same disease"
• when I have a long description, it shifts the column widths -- can these be preset?
• Descriptions of evidence types missing for Protein Interactions, FA, MS, and Rescue
• we will need to retest this with new Rescue section
• would it be better to bold score and keep default in parentheses? e.g. "0.5 (1.0)"

---

@selinad I wonder if we should ask the curators about this? I could see a similar argument for including evidence that hasn't been scored as we do in the VCI, that is 'it's good to see evidence that hasn't been evaluated/scored because you might have missed it and then that gives you the prompt to go back and evaluate/score it'?

I'm also very much in favour of bolding the scores to differentiate them from the default. Or instead, perhaps we could bold both of them when they are different to draw attention to those scores that have been changed?

[selinad]

@jimmyzhen based on discussion we just had, please ad "Age of" before the field (e.g. Report, Onset, etc.) in the table (also in bold). e.g.

Age of onset: 54 years

[selinad]

@jimmyzhen this is looking incredible so far! WOW!

I'm still testing a lot of things -- here's one small thing and we will see if @wrightmw agrees: for the Edit Classification button -- would prefer if it had the word "Edit" in it, even though I like the pencil icon. It took me a while to focus in on the icon and then one wonders what the "Classification" button does. Love the way it anchors to the bottom -- so clean!

---

A ✅

Looks great @jimmyzhen !

[selinad]

@jimmyzhen when user hits the edit Classification button, can it also anchor them to bottom? (wait to see if @wrightmw agrees).

Love the Curation Central button with the icon in it! Let's say "Record Curation page" like other places

---

B ✅

@jimmyzhen great! Anchor works and button looks great.

[selinad]

@jimmyzhen I'm worried the Previous testing column in case level is coming from the Y/N pull down -- I think it should come from free text below that. @wrightmw do you know?

@jimmyzhen for the proband method of detection, would it be reasonable to put "Method 1:" and "Method 2:" in front of answers? (in bold, like "Onset").

✅ @jimmyzhen looks great!

[selinad]

@jimmyzhen I just confirmed previous testing should come from description not the Y/N --- so sorry for not catching before (or specifying in google doc)

✅ Great @jimmyzhen !!!

[selinad]

Hi @jimmyzhen Here's one with no variant info type, but is marked review:

@wrightmw should we show these? It might be ok, but if we do the "()" looks funny. Not a big deal, but let me know if there is a quick fix @jimmyzhen (i.e. would it be simple to not show if curator does not pick a variant type?)

---

C ✅

Looks great @jimmyzhen !

[selinad]

@jimmyzhen per earlier point about previous testing fields: when I select Yes and then add a description, it looks like it might be combining the fields? Kind of funny as there is period at end of "Yes"

✅ The rendering of the "Yes/No" field has been removed.

Great @jimmyzhen ! I only see the description now, not the Y/N field

[selinad]

@jimmyzhen how are you sorting the tables? Can we sort by variant type?

✨ ✅

@jimmyzhen Thanks for the sorting -- fantastic! If quick, one ask would be to have the evidence with no variant type be at the bottom instead of the top (rest of order looks good). Not urgent.

[selinad]

@jimmyzhen -- it used to be the explanation field was grayed out unless they changed from No Selection. Now it's not, but it is still required if there is a modification. I think this is OK (and perhaps that's where we ended up), but can we add an "*" to that field when something other than No Selection picked? thx!

✅

This works now -- thanks, @jimmyzhen !

[selinad]

@jimmyzhen is there a way to make the Evidence Summary window pop open as a new window (not tab) and in a smaller tab (like a modal almost?). Probably not, but thought I would check : )

:man_shrugging: ✅

lol -- I think this means not possible @jimmyzhen !

[selinad]

@jimmyzhen first bug. If I first select Yes to count a LOD score, then Edit to say No, the Matrix seems to update (based on score not counting), but when I go to the Evidence Summary, it still says yes:

I tried doing a shift-refresh and still said Yes

p.s. changed score for a proband and there was no problem

🐛 ✅

@jimmyzhen this bug is fixed -- thank you!

[selinad]

@jimmyzhen just realizing there is no TOTAL score for Case-level evidence...

✨ ✅

[selinad]

@jimmyzhen this is a late ask, but I realize we didn't have the comments field for the whole GDM record when we created the mockup for Summary. Would it be possible to have another "table" below the top one summarizing record that shows the "Evidence Summary" field? It could also have a dark blue header. I think this would be of great value to the reviewers....

✅

This looks fantastic @jimmyzhen !!! Huge plus. thank you!

[selinad]

I am not sure what is wrong wuth this: :P (for both titles).

🐛 ✅

👍 @jimmyzhen !

[selinad]

@jimmyzhen for below:

Could you please change "(Cases)" to "(Case)" under Disease and add "(Case)" under "Detection Method"? thx!

💅 ✅

Looks polished! Thx @jimmyzhen !

[wrightmw]

@selinad @jimmyzhen If I click "Save" the on the Classification Matrix and then linkout to "Curation Central". Then link back to the Classification Matrix (without making any edits), I now have the option of "Cancel" or "Save" again. I understand it's because the modification and some other fields are editable but the options don't make sense to me... if I've made no changes then shouldn't I have the option of linking straight to the Evidence Summary? Maybe the option to "Save" should only occur if I start to edit any fields? Or maybe when I go back to this page it should all be greyed out and I have to specifically select "Edit" to change anything on the page? Maybe it's just me but it seems strange to have to "Save" again when I haven't made any changes?

[selinad]

Hi @wrightmw: my opinion this is an unusual use case and we shouldn't worry about the logic. I'm not sure I've followed it all though.

[selinad]

@jimmyzhen what field are you pulling from in Bioch fxn B for this empty field (Explanation) -- I filled in all required fields, so it should be one of those, I think:

🐛 ✅

[wrightmw]

@selinad Responses from above: A - I'm ambivalent on this one. I like the pencil but can also see that "Edit" is more immediately obvious. B - Yes, I think anchoring it would look better C - I don't have a problem with showing them but I agree that maybe a dash or some other identifier to designate a null field would be better than the empty brackets. Although not essential.

[wrightmw]

@jimmyzhen I forgot to say how wonderful the work you've done is. You've done an incredible amount in a short time. Thanks so much! Much like Father Christmas, I don't believe in elves... this magic is all yours ;-)

🎅 was here

[selinad]

@jimmyzhen if takes 5 minutes, one suggestion I should have thought of earlier: Can you add (A) or (B) as appropriate for Biochem function and Expression to the text below? Not a big deal, only if fast and you are in that section of the code:

e.g. "...disease of interest (A), ...."

💅 ✅

Nice @jimmyzhen !!!

[selinad]

@jimmyzhen another bug I'm afraid. I changed a score to 0 from the default -- it saved correctly in the matrix but shows as the default (0.5) in the summary table:

🐛 ✅

This bug has been squashed, @jimmyzhen ! thank you!

[selinad]

@jimmyzhen I've tried every evidence type and made certain all fields there and everything is great except fields I've noted. Really superb job. Only thing I didn't see was the TOTALS, but maybe OK for now.

@jimmyzhen Amazing -- even the totals are in now! Wow, this is spectacular!

@wrightmw we should probably make sure it works well when we score each other's evidence too. Here's my record if you want to try a few things: https://1411-jz-evidence-summary.demo.clinicalgenome.org/curation-central/?gdm=dad11445-1394-4ad4-a341-5f7e1136160a&pmid=22134555

[wrightmw]

@selinad I think my explanation is the problem.. here again with screenshots. Imagine I'm a curator who is happy with the all the evidence/scores etc., I don't plan to add/modify anything. I've marked it as Provisional and I'm asking a Reviewer to look at it. Here is my Matrix that I've Saved and marked as Provisional:

Some time later I just want to go back and view everything I did. My only way to access what I Saved is via the View Classification Matrix and when I do that I get this screen:

My only options are "Cancel" and "Save". I have already Saved everything on my previous visit (when I marked it as Provisional) and now I have to "Save" it again, when all I want to do is View the evidence Summary. I feel like I should have the option to view the Evidence Summary without having to re-Save what I've already previously Saved.

[selinad]

Thanks @wrightmw that is easier to follow. It sounds like @jimmyzhen would need to add logic -- he can let you know how straightforward. For now, I think it's OK to ask them to click again as it's (I think) a bit of an outlier case. Maybe this goes along with having the ability to view the evidence summary from the dashboard if no evidence has changed....

thx for the explanation and screenshots. I'll leave it to @jimmyzhen to weigh in!

[wrightmw]

@selinad Do you think that the rationale as to why a score was increased or decreased compared to the default score should also be included as a column in the table? e.g. a "Reason for change" column.

💅 ✅

Love this! Nice work @jimmyzhen !!!!

[selinad]

Hi @wrightmw I do like that idea! I'm just not sure whether it's in scope for this release, but @jimmyzhen will need to weight in :-)

💅 ✅

[jimmyzhen]

is there a way to make the Evidence Summary window pop open as a new window (not tab) and in a smaller tab (like a modal almost?). Probably not, but thought I would check : )

@selinad, we don't have control over the user's browser in this matter.

[selinad]

Hi @jimmyzhen I am adding this based on my discussion with you. Please feel free to ask me to move it to R14 as it is absolutely not necessary for this release. Would be great to display the type of Functional Alteration or Rescue or Model Systems (e.g. patient cells, non-patient cells) either in same column as the evidence type or in a different column (whichever makes most sense).

@jimmyzhen Wow -- you even added this, too -- amazing! Thank you.

[jimmyzhen]

@wrightmw,

Are the following comments related? https://github.com/ClinGen/clincoded/issues/1411#issuecomment-334633525 https://github.com/ClinGen/clincoded/issues/1411#issuecomment-334627640

[selinad]

@wrightmw @jimmyzhen Matt, I'm reading your workflow and screenshots with a clearer head. It seems like you are looking for different logic when something is marked Provisional? Does that take us back to the things we were working through the other day? i.e. you would get non-editable classification if it is Provisional? But, we would need to know no evidence added in between, right? It seems like it gets into that other workflow in the Google doc, but not certain if I am understanding.

It's complicated because matrix scoring can be new, Provisional or not. Maybe that's why I was trying to separate out the last saved. Still in drawing mode...

[wrightmw]

@jimmyzhen Yes, those two comments are related.

[wrightmw]

@selinad Yes. That's what didn't seem right to me. If you are still "in progress" then I don't think it matters that you are asked to "Save" again. But if you have marked it as "provisional" and then you go back to your Matrix and Summary, then it seems kind of odd that you are being asked to Save it again. I think you should have the option of going directly to the Classification/Summary that you marked as Provisional?

[selinad]

@jimmyzhen I've looked at everything -- only need to check the bug fixes. Absolutely incredible. You added some amazing enhancements, including the HP terms. Thank you so much for your hard work.

Quick questions

• how is experimental sorting? I was thinking evidence type, but there are a few mixed in with one another
• one total we forgot -- total for LOD scores in proband table. Not certain whether in scope for this release. If not, I will make R14 ticket (let me know)

Again, fantastic work!!! p.s. comments interspersed in above edits -- not certain whether you receive notifications for these

[selinad]

@jimmyzhen both 🐛 s have been squashed and all your checkboxes work.

I can also confirm the experimental is sorting correctly -- looking great!

[selinad]

Re new workflow additions (thank you for your quick work on this @jimmyzhen !):

• If this does not take much time: Could the border be in blue when you come back in and the modified score is still different from calculated? (e.g. it's not been switched to "No Modification"). Maybe to a blue border? Right now, it almost looks like it needs to be changed or there is something wrong (due to the red color)

• "Note: The calculated values below are based on the set of saved evidence and accompanying scores that existed when the "View Classification Matrix" button was clicked. To save a Classification for this Gene Disease Record based on this evidence, please see the section below. Otherwise, click "Cancel" below."

I'm wondering if we should move this to right below Total Points and say: "The Total Points shown above is based on the the set of saved evidence and accompanying scores that existed when the "View Classification Matrix" button was clicked. To save a Classification for this Gene Disease Record based on this evidence, please see the section below. Otherwise, click "Cancel" to return to your previous page."

• Then the red note would be at the bottom, just above Save button: "Important! The above matrix represents the points based on the current evidence and its scores and could be different from the scored evidence used in the Last Saved Classification, below. When saving a new Classification, please be certain to first make desired changes to any of the fields below based on the new "Current Calculated Classification."

Change to: If you would like to 1) save the new "Calculated Classification" highlighted in blue or 2) change your Last Saved Classification value based on this new "Calculated Classification" value, make sure to edit the highlighted fields and click Save. You may also choose to mark your Classification as Provisional."

[selinad]

Continuing:

• For the first time coming to the Matrix, looks great -- one change to text: "Heads up! The Classification Matrix above was re-calculated based on the current saved evidence and accompanying scores when you navigated to this page by clicking the "View Classification Matrix" button on the record page. To save a new Classification based on this current evidence, please fill in the fields below and click "Save". Otherwise, click "Cancel"."

"The above Classification Matrix was calculated based on the current evidence and accompanying scores saved in the database when when you clicked the "View Classification Matrix" button to navigate to this page. To save a new Classification based on this current evidence, please fill in the fields below and click "Save". Otherwise, click "Cancel".

[selinad]

@jimmyzhen would it be quick to add word "Calculated" here so it says "Modify Calculated..." (keep linked item same, i.e. don't link "Calculated" in with it: That's what they are really doing each time, I believe.

[selinad]

@jimmyzhen This is working well -- a couple of suggested changes that I hope are quick:

• move red box to below the explanation
• change in wording from "Important! Modified classification has been reset because it is the same as the current calculated classification." to "Important! This value has been reset to "No Modification" as the Calculated Classification based on the new Total Points is now equivalent to your last saved Classification value. Click "Save" to save the Calculated Classification value, or modify to a new value and click "Save.""

@wrightmw that likely needs work.