search

NHMDenmark / Mass-Digitizer

Common repo for the DaSSCo team
Apache License 2.0
1 stars 0 forks source link

Tabular remarks - condition for setting the additional columns #492

Closed jlegind closed 1 month ago

jlegind commented 3 months ago

What is the issue ?

Currently the columns are 'naively' set in post processing via the Monitoring script. Astrid has brought up the issue that these columns should only be set in case that the record has remark/notes values.

Detailed description of the issue.

A Digi app export csv will have the columns "datafile_source" and "datafile_date" and "datafile_remarks" added even if the original export contained no remarks values.

remark_cols

Why is it needed/relevant ?

A database row should be meaningful and default values should be used sparingly.

Estimate level of effort required.

easy

What is the expected acceptable result.

The datafile date should only be applied to record that in fact have a remark. On the other hand the datafile_source must be applied to all records since it was requested by the data manager for the purpose of tracking all records back to their original source.

How to approach it?

In the monitoring script insert a condition on the dataframe for when to fill in the datafile_date column.

jlegind commented 3 months ago

@AstridBVW feel free to weigh in .

AstridBVW commented 1 month ago

This ticket is based on a bit of a misunderstanding. The DaSSCo remarks field was set up in Specify to be tabular with fields added for date and source. In the post-processing script it was set up so date and source associated with the remark was filled out for all records. I pointed out that they should only be filled out if there indeed was something written in the remarks field. It was also made part of the post-processing to add information about the datafile to the datafile (i.e. the datafile name, date of the datafile and the source of the datafile), and it was decided to place the information in the tabular DaSSCo remarks table in Specify. Somehow these two things got mixed up.

The condition for the remarks has been implemented in the GREL script (ticket #506). The addition of information on the datafile has been made part of the GREL script for now also instead of a monitoring script.