search

ucsd-ccbb / qiimp

Web application to collect metadata specifications from an experimenter and produce metadata input files with appropriate constraints
3 stars 7 forks source link

Create option to keep (and pass through) or discard, the contents of a pre-filled template #98

Open adswafford opened 6 years ago

adswafford commented 6 years ago

Not sure if this is a bug or needed enhancement, but when uploading a template, there is no option to retain the data in the file. The use case is that the user creates and completes a template, realizes they need another column 'a_test_variable' and uploads their template to add this field. The output file is empty but the user can't cut and paste their filled-in old template because a) it will break the in-cell validations, and b) the column will be inserted in such a way that an easy copy and paste is amenable, especially if more than one new column is added.

I feel like this is a highly desired enhancement, but there may be issues with it that I haven't thought through just yet..

AmandaBirmingham commented 6 years ago

@adswafford This is currently prioritized as BOTH "must have" and "nice to have", which is confusing me. Could you assign it whichever one of those two best describes this issue's relevance to the beta release?

adswafford commented 6 years ago

Sorey don't he confusion, I think it should must-have since the addition of a new alphabetically sorted column will make it very challenging to copy-paste old metadata.

On Wed, Apr 4, 2018, 10:20 AM Amanda Birmingham notifications@github.com wrote:

@adswafford https://github.com/adswafford This is currently prioritized as BOTH "must have" and "nice to have", which is confusing me. Could you assign it whichever one of those two best describes this issue's relevance to the beta release?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ucsd-ccbb/cmi_metadata_wizard/issues/98#issuecomment-378670535, or mute the thread https://github.com/notifications/unsubscribe-auth/AZxBhsL4YXhpP41WtZ2ekBwTa6vlj6HOks5tlPsKgaJpZM4Rjpm1 .

AmandaBirmingham commented 6 years ago

@adswafford I am assuming this is must-have for the beta, and have labelled it as such. If that's wrong, let me know.

AmandaBirmingham commented 6 years ago

I have given this one some detailed thought, and I think we should NOT attempt to get it into the beta because it represents a large amount of effort. Here is my reasoning:

1) The existing metadata wizard NEVER READS arbitrary spreadsheet data. Loading fields from an existing template reads ONLY the contents of cell A1 of the metadata_form sheet, which is straight YAML. Therefore, adding the ability to read in actual cell-based spreadsheet data would be a significant extension of functionality. 2) Reading arbitrary data from excel is quite challenging, because users can find zillions of ways to do things wrong, or at least unexpectedly. (Austin encountered exactly this issue while working on #89 "Provide functionality to merge multiple metadata workbooks into single for qiita upload", noting "dealing with the creativity of user error scenarios is taking more time"!) For example, if the user manages to work around the spreadsheet protection and merge cells, then all bets about pasting straight from one column to another are off. Making sure that we detect problems/mismatches in transferring the data from one spreadsheet to another (let alone correct them) so that we don't mis-map the values and thus silently create garbage metadata--which is the worst possible outcome and must be avoided at all costs!--will require some serious thought and effort. 3) Right now the metadata wizard does not persist excel files that are uploaded: it reads in their field definitions (see above) and then immediately deletes them, all in one fell swoop. In order to enable transferring data from an uploaded file to a new template, we would have to change the metadata wizard to persist the uploaded file until AFTER the user had finished defining all their columns and hit submit (which could be an arbitrarily long amount of time, if they get up and wander away from the computer) and then make sure we're grabbing that same metadata file again to mine the existing metadata out of it and into the new template. Again, this is a non-trivial extension of the existing functionality, and a bit of planning should be done to guard against uploaded-file collisions, to make sure that uploaded files are cleared out once they are done being used but not before, etc.

As mentioned above, the core functionality needed for this feature (reading arbitrary data from a spreadsheet and handling user craziness) is the same as the core functionality needed for issue #89. I therefore suggest that this issue be addressed at the same time as #89, whenever that is :)