Closed mrmin123 closed 8 years ago
Each DB-changing action creates a user history object — a new object requiring a new schema.
The User History object contains:
Each action requires writing a new user history object to the DB. Assuming all actions involve either writing new objects to the DB or updating old ones, we can tie this mechanism to the routines that all of ClinGen uses to write or update objects:
rest.putRestData
: Write a single new object to the databaserest.postRestData
: Update a single object in the databaserest.postRestDatas
: Update an array of objects in the databaseAll these eventually call rest.writeRestData
. I propose having this function write a user history object to the database, as code doesn’t have to be duplicated, and rest.writeRestData
has error handling built in so it can easily make writing the user history object conditional on a successful database write.
Object deletion also needs to wind up in the user history. As we don’t yet have this ability, I’ll pretend a rest.deleteRestData
method exists that takes the URI of the object to be deleted, and returns a promise that would probably get ignored in most cases. This method writes an entry to the user history object :triangular_flag_on_post:
In the list below, “-1” after the name shows a primary object, a “-2” shows a secondary object, and “-3” shows the associated object.
AssessmentMixin.saveAssessment
: Used when saving any kind of assessment. The assessment object would be primary, but it has no human-readable identifying information, so we don’t display anything for it.
CurationCentral.updateGdmArticles
: Used when adding a new PubMed article to a GDM. “GDM ID” means the spelled-out group, disease, and mode.
CurationMixin.updateOmimId
: Used when updating the OMIM ID of a GDM.
Group.submitForm
: Used when creating or editing a group.
FamilyCuration.writeFamilyObj
: Used when creating or editing a family. Other objects can get modified when creating a family (e.g. Group), but those don’t get included in the history.
IndividualCuration.writeFamilyObj
: Used when creating or editing an individual. Other objects can get modified when creating an individual (e.g. Group), but those don’t get included in the history.
ExperimentalCuration.submitForm
: Used when creating or editing experimental data. Other objects can get modified when creating an individual (e.g. Annotations). Extra spaces between the parens exist because of markdown limitations — they wouldn’t exist on the site..
ProvisionalCuration.submitForm
: Used when creating or editing a provisional classification.
VariantCuration.submitForm
: Used when adding a pathogenicity to a variant. Pathogenicities have no human-readable identifying information, so it doesn’t get displayed here.
Deleted object-1 from object-2.
Two additional suggestions:
@forresttanaka @mrmin123 Thanks for the time and thought spent on elaborating a detailed spec for this feature that we discussed in our Thursday meeting. It will help us preserve historical provenance especially in the context of deletions, make user and item history more transparent and easier to access especially with multiple curators, and be able to track site metrics more easily. Nice job.
@mrmin123 Good stuff!
Updated the proposal above to add the operation type and notes about deleting objects. I also added a couple of notes at the bottom. Any changes have a little triangular flag marking them.
@forresttanaka regarding your notes, I agree with you on not using generated sentences. If generated, whatever interface we have to show these would have to do a query on the references objects whenever it wants to get a name or label. And yea, the reason I suggested the operation enum was mainly for filtering, but it would be nice to have a color/icon indicators for all the actions, especially for something like an All History view.
Wanted to make sure this was possible before going any further: can one property of an object embed more than one type of object, i.e. could the “primary,” “secondary,” and “associated” properties each embed the different kind of affected objects? Yes they can as the attached image shows. So code reading the history object can then use the embedded objects’ @type
to figure out what kind of object it is.
[Edit: To be clear, I don’t think we’re likely to initially use the “primary,” “secondary,” and “associated” properties. It simply seems dangerous not to include a reference to the very objects affected in case we find some use for them in the future. Initially though, we’ll embed links (like /curation-central/?gdm=xxx
) in the history object, and directives in the string say where to put the links]
My initial thinking was that embedding the objects would be overkill, but perhaps there are some use cases for it that I'm not thinking of right now. @forresttanaka if you can potentially see a need for it in the future, should we just go with embedded objects to begin with? Changing it in the future once in production seems like it'd be a real pain.
@mrmin123 I’ll plan on including the embedded objects for now. Then maybe once things work, we can set up some performance comparisons with and without embedding. It would probably make no significant difference when writing the objects to the DB, but I could imagine some difference when reading in an object the back end has to put together.
It's possible that the size of all returned objects with full embedding of all related objects could be huge, and might be a performance problem for the dashboard view for users with a lot of activity that needs to be queried and displayed on the dashboard.
The nice thing is that you can choose how deep to embed the objects, so really we might only include the top-level objects, none of which are all that large. If they include embedded arrays of objects it can get large, but we can simply choose not to embed those. But I could see the argument “If you’re not embedding the objects within the objects, why include the objects at all if you’re not using them?” The answer right now is just that it makes me feel better. But it may well not have any value.
At this stage I’m thinking I could put them in the schema and simply not use them. That way if we find a use in the future, it would involve a Javascript change, but no schema change.
As a progress report, I can now save history items to the DB and then use a single routine to render it with links. The string that generates the screen shot below contains:
Group {P:GRP4} added to {S:DICER1-Achondroplasia Autosomal dominant inheritance (HP:0000006)} for {A:PMID:123}
The problem of course is the italics and boldface. I’m not really sure yet the best way to encode those in the string, or to do things a completely different way. I thought about a really pared down Markdown syntax, but even really pared down, the processing might be kind of laborious.
Anyway, here’s how it looks when creating a group:
…and here’s how the history object looks in the DB. Codes in the description
property specify where the primary, secondary, and associated properties go, and what their link text says. the elements
property has corresponding objects containing the link text and a reference to the relevant objects themselves, which I don’t use and don’t get embedded.
Solved the styling problem by changing the history objects pretty much completely. Each operation being done now not only controls writing data to the history object, it also handles displaying the data. For example, groups not only control the writing of their data to the DB, a function belonging to the group module now gets called by the dashboard through the same registration mechanism we use to display pages. The dashboard only knows it’s calling whatever owns the “primary” object (still there), and that primary object’s code then knows how to display its own data with whatever styling it wants.
The bottom entry of the attached image shows the new display mechanism. The top three are the existing dashboard display mechanism.
Lots of testing needed, but here’s the dashboard displaying history objects. Lots of code that retrieved and interpreted each object removed.
[Edit: Oops — “experimental data segregation”?]
Give it a try. It should have no visual impact except on the dashboard’s “Your Recent History” section.
https://215-user-history-34b0bf4-fytanaka.demo.clinicalgenome.org
Looking great @forresttanaka - wow - it is capturing a lot of actions!
A few comments:
Good stuff @selinad! I’ll go over these tomorrow. Offhand, I don’t see a problem with any of these, though the last one will be a bit clunky to implement. The order displayed is actually the order it happens, so some information will have to skip past others. Should be doable though.
Great - thanks @forresttanaka ! Perhaps I did something funny with my order. I think it should do it in align with my actions, as you have it, so let's leave it. I do think I assessed last on the Experimental, though, and the order was different ...
Let me know if I missed any actions....thx!
@selinad No, that’s just the way it works — the assessment gets recorded, then the thing it assesses gets recorded. I agree it looks awkward. If you create something, then assess it later, then it shows in the natural order. It’s just when you create and assess on the same page and save them together, it gets recorded in the current order.
…so I’ll work on flipping that order in the history tomorrow.
Got it. In that case, flipping would be good!
@selinad
Sometimes the verb is at the front "Assessed..." and sometimes the at end "Variant pathogenicity added" - seems like it would be better to be consistent throughout.
These are two different actions though. One is assessing a pathogenicity, and the other is adding a variant pathogenicity regardless of the assessment.
Could it be:
Variant 1000 pathogenicity assessed as Supports Variant 1000 pathogenicity information added
OR
Assessed variant 1000 pathogenicity to Supports Added pathogenicity information to variant 1000
Here it is. I’ll spin up an instance shortly
Hopefully addresses all of @selinad bullet points:
https://215-user-history-5e2150d-fytanaka.demo.clinicalgenome.org/
Thanks @forresttanaka - your screen capture looks good - will take a look at instance.
Hi @forresttanaka - great!
A few things:
One kinda strange thing:
p.s. I like the way you changed around the wording - make it easier on the eyes as well since hyperlinks are in more uniform position (generally)
OK, there’s a much larger problem: it shows everyone’s recent history as your history. So I’ll fix that too.
@selinad or @kgliu0101
Would it be difficult to add the Classification level in? e.g. "Provisional classification LIMITED added to DICER1-Pleuropulmonary blastoma-Autosomal dominant inheritance with maternal imprinting"
I see autoClassification and alteredClassification. Which one should I show?
If necessary, alteredClassification is the one. In default, alteredClassification = autoClassification. It may be changed by user manually.
One thing I don’t link to in the history list are provisional classifications. I’m not sure what to link to, nor what text in the history the link should surround. Suggestions? Should a link exist for these at all?
We don't have View page for provisional yet. If must have a link, it can go to edit page (../provisional-curation/?gdm=uuid&edit=yes).
New instance featuring linked provisional classifications, and variant add events.
http://215-user-history-c5bb642-fytanaka.instance.clinicalgenome.org/
Boldfacing variant IDs: http://215-user-history-93a3503-fytanaka.instance.clinicalgenome.org/
[Edit: Might have a merge issue. Will spin up another shortly]
Fix possible merge error: https://215-user-history-9b44de8-fytanaka.demo.clinicalgenome.org/
New instance that doesn’t have the refresh problem when adding a new PMID:
https://215-user-history-9b44de8-fytanaka.demo.clinicalgenome.org
New instance with Copy OrphanetID button problem fixed:
https://488-user-history-6d64d33-fytanaka.demo.clinicalgenome.org/
Another great release (R2). Nice job and thanks for your hard work.
Currently we're relying on the
owner
(soon to besubmitted_by
) anddateTime
(soon to bedate_created
) fields to track user-made changes at the moment, which is a transient and incomplete history at best. We should incorporate a way to to track most (if not all) user-made changes for a more accurate 'Recent History' on the Dashboard and (more importantly) auditing and tracking purposes.