VCI Expert Review and Approval - Provisional Interpretation

[jimmyzhen]

Implement the Provisional Interpretation section, which consists of the following requirements:

1. Section panel title: Provisional Interpretation
2. A dropdown of by Individual or on behalf of Group options for the user to select.
3. Dropdown label: Interpretation being saved 'by Individual' or 'on behalf of Group'
4. Display of form fields for labels ClinGen Group Name OR Non-ClinGen Group Name. Either one of these fields is required if the on behalf of Group option is selected.
5. A Save as Provisional Interpretation button for the user to save the selected option from the dropdown.
6. The dropdown selection is required if the user wants to save the provisional interpretation.
7. Display of a note with message: Note: Saving as a Provisional Interpretation does not submit it to ClinVar.
8. Display of the Current Saved Provisional Interpretation, including the timestamp.

[wrightmw]

@jimmyzhen ClinVar VariationID: 10000 does not load data in this instance. It doesn't load in this test instance but it does work in the normal demo and prod interfaces.

@jimmyzhen ClinVar VariationID: 17000 did work in the test instance when I created it as part of 10001 Affiliation but when I went back in and tried to enter VariationID: 17000 as part of Affiliation 10002 it would no longer load any data. In other words, it seems like this loading issue arises when a curator from an Affiliation tries to enter a ClinVar VariationID that has already been entered by a curator from a different Affiliation.

✔️

[wrightmw]

• [x] @jimmyzhen The "date reviewed" only allows to add Mon-Fri dates... perhaps open up to 7 days for our earnest weekend warriors :-)

oooh -- good catch @wrightmw ! Same in GCI (probably obvious)

✔️

[selinad]

@wrightmw @jimmyzhen I'm thinking we don't need this text any more (on Evaluation Summary)

To clarify -- this was confusing the 2nd time through when I'd already saved as Provisional

@selinad Do you mean get rid or it altogether or just the second time around? I think H & S requested this text.

[selinad]

@jimmyzhen @wrightmw I may be confused -- I thought it was possible to save an interpretation as LP or P, just not as Provisional. I got this trying to save going around the 2nd time after an Approved

[selinad]

@jimmyzhen this note works great the first time around when there has not bee a previously saved Provisional. One change to text if possible:

"The option to save an Interpretation as Provisional will not appear when the saved calculated or modified value is "Likely Pathogenic" or "Pathogenic" and there is no associated disease."

✔️

[jimmyzhen]

I thought it was possible to save an Interoperation as LP or P, just not as Provisional. I got this trying to save going around the 2nd time after an Approved

@selinad, in regards to your comments above, you need to provide an explanation to the modified pathogenicity before you can save the interpretation.

@jimmyzhen sorry, I think my point is why is the warning bar there -- I can save it if it's not provisional (after providing a reason)

@jimmyzhen it does not happen if I haven't saved as Provisional before

[selinad]

@jimmyzhen If I get a warning I can't save as Provisional then go back and add a disease and View Summary again, it gives me the Provisional box. The provisional box should only appear after a Save of the Classification

[selinad]

Need a space between sentences in this text:

✔️

[selinad]

@jimmyzhen If I had Modified to LP, then go back around and add more and get Calculated of P, then it still says "LP" for Modification, but if I change to P it makes me give a reason...is the same logic being used in GCI and VCI? thx!

[jimmyzhen]

@selinad, I don't know whether they are using the very same logic, although they kinda behave similarly.

[selinad]

@jimmyzhen here's what it does in GCI

[wrightmw]

@jimmyzhen An empty "Provisional Classification entered by" field in editable box is filled upon saving. Perhaps get rid of this from editable box and only display once saved?

[wrightmw]

• [x] @jimmyzhen I got confirmation back from the curators... they want "date reviewed" for the editable date field for Provisional Interpretation and "date approved" for the editable date field for Approved Interpretation.

[wrightmw]

• [x] @jimmyzhen The yellow text needs to be updated from "...thisInterpretation..." to "...this Interpretation..."...i.e. need gap...

[wrightmw]

• [x] @jimmyzhen The "Provisional Classification entered by:" should only appear after submitting:

[wrightmw]

• [x] @jimmyzhen This yellow text also needs a gap:

[wrightmw]

It's looking pretty fantastic so far @jimmyzhen ... you've done an amazing job with this!

[selinad]

• [ ] @jimmyzhen I Approved an Interpretation (looks fantastic!) and then added more evidence and came back to test as it was now Pathogenic. Saved and it did the right thing to ask me to add Disease. Did that came back and changed to In Progress -- maybe it happened when I saved w/o a disease (will check) -- went back in and repeated. It happens when I save another Interpretation (i.e. status went Approved --> In progress). I was saving a Patho without a disease.

@jimmyzhen it happens even when I don't have to add a disease -- in this case, had just made a change to another criteria and saved -- you can see I was presented with Provisional option:

---

@selinad According to the screen shot(s), there is a disease associated with the interpretation...

[selinad]

@jimmyzhen adding a 2nd Prov and 2nd Approved Interpretation works GREAT!

[selinad]

• [x] @jimmyzhen I can't load the same variant as a different curator than myself still -- not sure whether this bug was addressed?

---

@selinad, @wrightmw, I can't seem to be able to reproduce the problem anymore.

[wrightmw]

@selinad I don't think that bug is fixed because I am getting the same issue

[selinad]

• [x] Reasonable to put a note about Approving above Approval entry box?
  

Here is the one for Provisional:

Suggested text: "When ready, you may save this Interpretation as Approved. Once you have saved it as Approved, you will not be able to undo it, but you will be able to save a new current Approved Interpretation, archiving the current one and retaining access to its Evaluation Summary."

[wrightmw]

@selinad re. text above...

Suggested text: "When ready, you may save this Provisional Interpretation as Approved. Once you have saved it as Approved it will become uneditable, but you will be able to save a new current Approved Interpretation, thus archiving this current one and retaining access to its Evaluation Summary."

[selinad]

Looks good @wrightmw -- thank you!

[selinad]

• [x] @jimmyzhen for saved Provisional and Approved snapshot boxes, change "Classification" to "interpretation" (VCI Summary page)

• [x] @jimmyzhen when Provisional exists and not Approved and someone comes back to Summary, the Evaluation Summary box stays open like it is, but Approval box needs to disappear and button goes on snapshot for most current Saved Provisional (per GCI workflow)

• [ ] documents show new Status buttons for header

[selinad]

• [ ] @jimmyzhen everything is looking great! All the things we talked about look to be working well on first run-through. Only one potential problem is after delete disease -- then something that was approved goes back to in progress:

[selinad]

@jimmyzhen I think the above also occurs if I add a disease back....

[selinad]

p.s. love the "Approve this Saved Provisional" button!!!!

[wrightmw]

@jimmyzhen Just been looking at your new instance. It looks phenomenal! Love the orange "Approve this Saved Provisional" button!

[wrightmw]

And I am now able to work on the same variant as part of different Affiliations. Great!

[wrightmw]

@jimmyzhen @selinad

When you first visit an Interpretation, you get to see "All Existing Interpretations", which is great, but this is not behaving quite right.

Here are the headers for the existing Interpretations I randomly created:

1) An APPROVED "likely pathogenic" interpretation from the Hereditary Cancer WG Affiliation

2) An APPROVED/NEW PROVISIONAL "benign" interpretation by the Rett/Angelman

3) A PROVISIONAL VUS interpretation by the RASopathy EP

4) An IN PROGRESS interpretaion by the KCNQ1 EP.. for this one I have only Saved an Evaluation Summary... nothing else

5) I started an Interpretation as BCM-Education...added evidence ...but I have not saved an Evaluation Summary yet... so no status

Now when I go in as a different Affiliation, this time as "No Affiliation" and I go to the Evidence View for the above 5 examples, here is my header:

• [x] Rett/Angelman says "PROVISIONAL" but it is actually "APPROVED/NEW PROVISIONAL"

• [ ] For BCM-Education because I have not saved an Evaluation yet, it is not marked as IN PROGRESS in the headers but when I go back to my dashboard (acting as BCM-Education) it says "In progress":

Therefore, we are inconsistently showing the "In Progress" status.

[wrightmw]

Btw, as you can see I can definitely work on the same variant as different Affiliations. Whatever you did with the elastic search is working! Great!

[selinad]

@jimmyzhen @wrightmw this is what I saw before. I started an Interpretation, saved as BA1 Met -- you can see it responds I saved, but I don't get update button. Also criteria bar does not change... @jimmyzhen let me know if I should make a ticket....I believe a curator was writing in about this and I asked for more info.

Note criteria bar -- also, Summary shows no criteria saved

I then try to do PM2 and have the same problem:

First time in for a new variant and BS1 seems to work (and after this I was able to modify it to PM2 and it saved....almost as if saving as BS1 first works, but saving as BA1 or PM2 first doesn't work. Summary looks as expected)

[selinad]

@jimmyzhen when I clicked on the orange Approved button, it went back to In progress and I was not able to get the chance to Approve:

[selinad]

@jimmyzhen @wrightmw If I go go back to Summary to Approve a saved Provisional that was benign after adding more evidence and it's now pathogenic, I get this, which is very nice.

However, if I click to save it as Approved (orange button), I get this, which seems correct, but we may need to make certain curators aren't thinking they are saving the pathogenic....I'm not sure what to do for this, or whether we should worry right now?

Could we have some "simple feedback" on the save evaluation part saying not yet saved? When they click button, it would say saved?

[selinad]

@jimmyzhen steps for buggy thing

• saved a Provisional as benign
• modified Interpretation so it was Pathogenic and needed a disease
• added a diesease
• returned to summary
• clicked orange button to Approve previous Provisional
• got this: (looks like it is not bringing up Approval box, but a Provisional one....

[selinad]

@jimmyzhen thoughts:

• we have this text in GCI -- analogous text could be helpful in eliminating confusion in VCI (or maybe it is there and not displaying....sorry):

• the white is hard to see in the orange bar...should we make this text black so we can see Provisional vs. Approved?

[jimmyzhen]

@selinad,

the white is hard to see in the orange bar...should we make this text black so we can see Provisional vs. Approved?

I made the text bold and it seems to read better now:

we have this text in GCI -- analogous text could be helpful in eliminating confusion in VCI (or maybe it is there and not displaying...

There is currently a similar paragraph of note text in the VCI (see the screen shot above). Although it may not be exactly the same text you see in GCI, it is there when the approval form entry panel shows.

[jimmyzhen]

@selinad,

However, if I click to save it as Approved (orange button), I get this, which seems correct, but we may need to make certain curators aren't thinking they are saving the pathogenic....I'm not sure what to do for this, or whether we should worry right now?...Could we have some "simple feedback" on the save evaluation part saying not yet saved? When they click button, it would say saved?

There is also a paragraph of note text in the VCI being shown when the provisional form entry panel is displayed (upon saving the evaluation). You may choose to update it if it suits the needs. For anything more complicated or needing a more well thought-out user experience, I would suggest we look into that in subsequent releases if this is not a showstopper.

[selinad]

Testing steps

• add variant 21135

• add some evaluations for benign

• go to Summary

• Save summary. @jimmyzhen this looks great -- were we going to remove "In Progress" though? I thought it disappeared upon first save.

• go back and add another evaluation

• return to Summary and save a new one -- all looks good

• save this as Provisional -- all looks good

• return to interpretation and make another evaluation

• return to Summary -- all looks good

• modify to LP and give reason, save again; expect to see old provisional and ability to approve it, but see this (should still let me Approve original one, right? But, saved behavior good since there is no disease):

• go back out and add disease

• go back to Summary -- looks good except that still can't approve 1st provisional even though I haven't saved a second Provisional

• save a new Provisional --- looks great

• save a first approved -- looks great!

• went back to delete Disease -- great note on modal

• changed to VUS in summary and saved. All looked good

• saved Provisional and got correct status!

• saved as Approved

All in all looks great!!! Just the one bug when I saved as LP (see above) and there was missing ability to save the existing Prov as Approved....

Dashboard -- was I supposed to see this as Approved? It shows none

here is is when I go into it:

[wrightmw]

Previous error I reported whereby "Approved/New Provisional" was being incorrectly reported as just "Provisional" in the "All Existing Interpretations" section has now been fixed. Thanks Jimmy! @selinad I have added some Affiliations for you in this new testing instance... you should see them next time you logout and back in again.

[wrightmw]

• [x] Jimmy and @selinad ... there is an issue with the dashboard. If I save an "Approved, New Provisional":

@wrightmw @jimmyzhen -- I was not able to see any status in the Dashboard for the VCI, but was able to see the Approved + New Provisional for the GCI (one the Dashboard.
@jimmyzhen sorry -- I see your checkmark and I did a shift refresh and can see buttons on Dashboard! Now works great!!!!

..when I go back to the dashboard it says just "Provisional":

[wrightmw]

Whereas, as I said earlier... in the "All Existing Interpretations" section this is now reporting correctly. Which is fantastic! Thanks Jimmy!

[jimmyzhen]

@jimmyzhen please see comments to your comments :-) @selinad,

Please see my responses to your feedback from your testing last night:

were we going to remove "In Progress" though? I thought it disappeared upon first save.

"In progress" remains until the 1st Provisional is saved.

• ok, great -- I just looked back and see that now. Good and sorry about that

modify to LP and give reason, save again; expect to see old provisional and ability to approve it, but see this (should still let me Approve original one, right? go back to Summary -- looks good except that still can't approve 1st provisional even though I haven't saved a second Provisional

As the way it is implemented now, I am disabling the ability to approve a saved Provisional as soon as the user saves changes in the Summary. The thinking behind it is that the user should provision the newly saved changes in the Summary again prior to approving it.

• see my comment in GCI ticket -- @wrightmw do you agree with this behavior? I expected to be able to approve a saved provisional until a new one occurs.

[selinad]

Hi @jimmyzhen , @wrightmw discovered a bug when we were demo'ing for the curators. The current approved moves to archived -- it should have a green flag instead of a file box

@wrightmw confirmed this is independent of whether disease added or not

[selinad]

@jimmyzhen feedback from curators: 1) loved the workflow and no confusion around it! 2) adding "current" before green flag and "archived" before file would be good -- if this is not possible, could we make the icons a tad larger -- feedback was the file box looked like a trash can =:-0

[selinad]

@jimmyzhen would it be "quick" to have the gray "View Approved Summary" (or Provisional) have a blue border? NOT NECESSARY at all (they were fine with it being gray) but if quick it may make it more obvious it's clickable.

[selinad]

@jimmyzhen one more bug, which seems to be related to a long variant name -- problems on Dashboard display

[jimmyzhen]

@selinad, @wrightmw,

Please use the following instance to review the changes and fixes in regards to your comments and reports above: https://jz-approval-workflow.demo.clinicalgenome.org

Thanks!

[selinad]

@jimmyzhen looks GREAT!!!! I couldn't break it! Nicely done. Only 2 small items:

• it doesn't seem like the tiestamp for Saved Summary updates all the time. I saved one at 8:27 and one at 8:29 and it remained at 8:27.

• still a wrap problem -- I don't think there is a need to stress about it now, though (24229)