An open-use web application created for easy access, collaboration, and sharing of datasets through rich metadata search, filter, sort, and direct viewing in common industry applications or in our web-based 3D Volume Viewer.
AIBS and others have had trouble inputting their data sources since the feedback wasn't being shown to the user most of the time and even when it was it was hard to interpret as actionable. Also, several have noticed that empty file paths would lead to missing rows.
Also, @kmitcham made me realize that the app has enough data to be smarter about the kind of feedback it gives and when it can provide defaults.
Description
This does 3 things as part of ingesting a data source:
Normalizes the column names. Makes columns like "file_name" change into "File Name" so we no longer have issues with case-sensitivity
Checks for uniqueness constraint issues and/or blank values in columns where we need a value. This includes making sure either File Path or File ID are unique.
Adds default columns for File Name and File ID when not present already. This helps by guaranteeing that "File ID" is always present and is always unique since some users want "File Path" to not always be unique.
TODO: Triple check with users that default "File ID" isn't annoying. Unsure about alternatives...
Testing
I've tested this manually with several different .csvs with various casings, missing values, duplicate values and such. However, I would really appreciate as part of this review that you try this out too! It is on staging
Background
AIBS and others have had trouble inputting their data sources since the feedback wasn't being shown to the user most of the time and even when it was it was hard to interpret as actionable. Also, several have noticed that empty file paths would lead to missing rows.
Also, @kmitcham made me realize that the app has enough data to be smarter about the kind of feedback it gives and when it can provide defaults.
Description
This does 3 things as part of ingesting a data source:
Testing
I've tested this manually with several different .csvs with various casings, missing values, duplicate values and such. However, I would really appreciate as part of this review that you try this out too! It is on staging
Related Issue
Resolves #272