MaayanLab / FAIRshake

https://fairshake.cloud
Other
10 stars 6 forks source link

Rubric / Metric version control immutability #43

Open u8sand opened 6 years ago

u8sand commented 6 years ago

The modification of a rubric or metric might invalidate existing assessments using that rubric or metric. To address this, rubrics and metrics need to have version control which assessments depend on.

This may be less of an issue if only descriptions change, but if, for instance, the metrics themselves that are part of a rubric change, then assessments built on the old version will be invalidated--they'll have answers to metrics that aren't a part of the rubric and lack answers to metrics that are.

This could be mitigated in two ways:

  1. Rubric, Metric version control--assessments depend on a specific rubric version
  2. Rubric's associated metrics are fixed and cannot be modified after creation.

Perhaps (2) is a short term solution and (1) is a longer term solution.

u8sand commented 6 years ago

Also, currently anyone is able to just add metrics to an existing rubric, but that shouldn't really be allowed unless you own the metric--same thing with a project?

Should anyone be allowed to make associations if they don't own both targets of that association--I would think yes but maybe it needs to be accepted both ways?

u8sand commented 6 years ago

This extends itself to Assessments--it makes sense that any (user, project, rubric) assessment is unique--when users try to assess the same pair again, they modify their previous assessment. By tracking changes in the database, we can still keep track of results over time. Paired with tracking rubric modifications, we'll be able to match assessments with rubrics in time.

u8sand commented 6 years ago

I'm well on the ways to completing this--I've actually gotten the models all migrated cleanly (it was a bit more painful than I anticipated). Before I can merge this into master, I need to test everything to make sure it works properly:

In the future we can expose the different versions via the web ui