Bulk edit records

[johnvanbreda]

Support for large scale editing of records, e.g. for corrections to grid refs or dates following uploads. Thought needed on what fields should be editable and by whom, plus options for audit trail or undo.

The initial scope will only allow editing of the following fields:

• Recorder names
• Date
• Spatial reference
• Location name.

@kitenetter a few questions:

1. Given that the above fields are sample fields, what should happen if a bulk edit attempted on records where there are other records in the altered samples that were not selected for bulk edit? 2 options would be to reject the bulk edit request, or to split the alterations into new samples.
2. Should this tool only allow users to bulk edit their own records? Or can verifiers bulk edit other people's data?
3. If verifiers can bulk edit other people's data, presumably there is a need to check that the requested records fall inside one of the verifier's filters.
4. What level of feedback is needed? Is a simple count of updated samples (or occurrences if fields added to the spec in future) enough? Or a list of changes showing before/after values (which might be quite a large table of changes of course).
5. Do we need to offer a preview of the changes before actually running the operation?

An undo feature would of course add considerable complexity.

[johnvanbreda]

A first drop of code for this is now in the develop branch - I fully expect it will need further work.

[johnvanbreda]

Question 1. I've implemented a check for attempts to edit records in samples that contain other records. If this happens the user is offered an option to cancel, or proceed which will result in the samples being cloned and the edited records moved to the clones before they are altered, so the other records are unaffected.

[kitenetter]

@johnvanbreda sorry for being slow to respond to this one. The above all sounds good. To respond to your initial questions:

1. Given that the above fields are sample fields, what should happen if a bulk edit attempted on records where there are other records in the altered samples that were not selected for bulk edit? 2 options would be to reject the bulk edit request, or to split the alterations into new samples.

I think you've already dealt with this.

2. Should this tool only allow users to bulk edit their own records? Or can verifiers bulk edit other people's data?

For now at least, just for users to edit their own records. If a verifier has a need to impose a bulk verification they can contact iRecord support and we can masquerade as the relevant user if necessary (and assuming that all parties agree).

3. If verifiers can bulk edit other people's data, presumably there is a need to check that the requested records fall inside one of the verifier's filters.

Yes, but not needed at the moment.

4. What level of feedback is needed? Is a simple count of updated samples (or occurrences if fields added to the spec in future) enough? Or a list of changes showing before/after values (which might be quite a large table of changes of course). Do we need to offer a preview of the changes before actually running the operation?

Arguably an extensive preview is not needed, because if a bulk edit is done and then realised to be incorrect, a reverse bulk edit can be applied to change back again. If possible it would be good to have a summary that says "W records will have field X changed from Y to Z".

Verification status I assume that any such edits will result in the record being set back to pending? This may be problematic but we can add a warning to users not to edit verified records unless absolutely necessary.

[johnvanbreda]

Verification status - yes, altered records will be reset to pending as normal (though I suppose I could code it not to do this - but that doesn't sound like the right solution). I can add a warning to the popup. Is it worth the extra effort of doing a pre-check to see if the records about to be altered contain any that are not currently pending, so the warning only appears when relevant?

Regarding the preview, note that a bulk edit can be performed on a range of different values in one dataset, so an undo bulk edit is not going to work. The dialog currently shows the number of records about to be fed into the bulk update, but only those which have got different values will actually be updated, so a preview could be useful (though is extra effort).

[kitenetter]

@johnvanbreda "an undo bulk edit is not going to work" - that is understood, and I wasn't meaning an "undo", what I meant was that once bulk editing becomes possible, if something goes wrong it can be bulk edited again to put it right.

But I do think that a preview would be useful - is that possible within the current work or will it have to be added as a feature request for the future?

[johnvanbreda]

@kitenetter the reason behind my statement that bulk edits cannot be easily undone is that you can bulk edit heterogenous data e.g. a range of different date values can be edited to a single date value in one go, but they cannot be reset back to the range of date values that was their previously easily.

I've now implemented a preview of 10 records - the system tries to find the records with combinations of field values that are least prevalent to show in the preview to make it more likely you will spot mistakes. So, if you have 1000 records with the same date except 1 record which has slipped into your dataset with a different date, when you bulk edit it will show you the different date top of the list. It was a bit complex to code but seems to work well.