Update `timestampModified` of parent of a dependent record when uploaded records in the WorkBench

[grantfitzsimmons]

CSIRO wants WorkBench to automatically update the timestampModified of the Collection Object when a dependent record like determination, preparation, or attribute is uploaded and linked to it.

When adding new dependent records on the form (determination, preparation, etc.), the timestampedModified is set to be the current time once saved.

Expected Behavior:

• When a dependent record is uploaded and linked to a parent record in the WorkBench, the timestampModified of the parent record should be updated.
  • For example:
    • If preparations, determinations, or properties/attribute are uploaded using the WorkBench and linked to an existing Collection Object, the Collection Object should have the timestampModified set to the current time.
    • If addresses, groups, identifiers, or variants are uploaded using the WorkBench and linked to an existing Agent record, the Agent should have the timestampModified set to the current time.
    • If collectors are uploaded and linked to an existing Collecting Event, the Collecting Event should have thetimestampModified set to the current time.

We wanted to escalate the fix of github ticket #1806 (https://github.com/specify/specify7/issues/1806) as we didn't realise just how damaging this issue is to our collections. The fact that the system is not updating the datemodified field means we are losing data integrity as we cannot run any audits on when specimen records are being modified by users.

From our testing of this any time we upload something to the workbench that is linked to a collection object (determination, preparation, voucher relationship etc.) the creation event is logged in the audit log tables under that specific table, but there is no row in the audit log that shows that the related collection object was modified. Both the datemodified and version fields for the collection object are not updated. If you use a form to manually create/edit a determination then the audit log correctly records the event for the determination table and the collection object table, and the later has the correct datemodified and version values updated.

This bug is a breach of FAIR data policy since these edits are no longer "Findable" by our users. All of our collections need to report on their activity, and that includes reporting newly databased specimens within a time period, as well as edits/additions to any specimen.

It means we can run an easy query that identifies what has changed and allows our collections to do their annual audits of database activity. Right now doing the same action in the Workbench as in the forms DOES NOT yield the same result.

Requested by: CSIRO on Asana

[maxpatiiuk]

But this is not just for workbench, right? It would be for forms, and batch editor and agent merging and etc?

Thus, it would have to be implemented on the back-end where resources are saved, not just for workbench

[danb213]

I would like to confirm that this is accurately reflecting our request. @maxpatiiuk , just to clarify, the forms are already behaving this way. We are just looking for consistency in behaviour when adding via forms or Workbench.

One additional comment would be that purely from a database point of view the form changes to child tables of the CO should not actually change the datemodified field of the CO since technically nothing has changed in that table (apart from the version number). It would make more sense to add a new date field in tables like CO/CE/AGENT/LOAN etc. called GROUP_OBJECT_MODIFIED_DATE that could be referenced for our reporting purposes. This new field would be updated whenever the main table OR a child table object was created/edited. However I suspect this would be more work since you'd have to undo the current form behaviour, add columns to tables, and add a way for users to query on the group object modified date.

[maxpatiiuk]

just to clarify, the forms are already behaving this way. We are just looking for consistency in behaviour when adding via forms or Workbench.

ah, thank you for the clarification.

It might be that this behavior is already in place for Forms, but Workbench uses a separate mechanism for saving records, and doesn't implement this logic. In which case, indeed, this issue is only for Workbench - something we should address. Someone from the back-end side of sp7 would be able to provide a better answer.

[realVinayak]

The description / recreation steps here are a bit misleading. When you save a determination (or make a change to determination), the CO's modified date or version doesn't magically get updated due to change on determination. It only gets updated because the form for CO was open, and you were saving the CO. That is, if you were to open the determination separately (via /view/determination/ID), the CO's modified date / version will not get updated.

I don't want to add a "over-encompassing" businessrule that triggers a save on all parent objects because I can see it being very buggy -- what if you are saving a CO, the determination will trigger a save on CO, for every determination? Plus, if those determinations have dependents, then you'll have those many times number of dets needless saves on CO (we can't use update to override)? It'll be faster in an alternate implemenation where we manually save() on related. If implemented correctly, we can use that same logic in workbench, without any duplication (just need to update toOne matched). We don't care about toMany (think why)

BUT, I don't want to make this for specify in general. It'd be slow. We'd need to compare the performance. If it becomes too slow, we can add a django setting or a remote pref file. I want to add a django setting, since remove pref file fetch and read would itself be too slow, on each save of a record.