forc-db / ForC

Global Forest Carbon Database
https://forc-db.github.io/
Creative Commons Attribution 4.0 International
55 stars 24 forks source link

Design a system to faciliate data entry #54

Open teixeirak opened 6 years ago

teixeirak commented 6 years ago

Data entry is more cumbersome than it needs to be, and understanding the system is a learning curve.

One idea to improve this is to create a single data entry spreadsheet and R code that can draw from that to update the database. The speadsheet could include field descriptions and instructions for data entry. It could also allow for multiple measurement records or history events to be recorded quickly without requiring entry of duplicate information.

dlebauer commented 6 years ago

I think that the option to have a spreadsheet template and R function to parse it is a good one. Once the function is written some adventurous sole can implement a spreadsheet upload user interface, e.g. this feature in an R shiny web application. There could be some combination of fields and drop downs menus (e.g. to select a site and treatment) for information that is globally applicable.


Commentary

The BETYdb user interface now has a bulk upload functionality. The documentation of the bulk upload feature and data entry documentation more generally should illustrate some of the issues that may be entailed in this. One major trade off is that the web interface and database can be more difficult to use than a spreadsheet, in part because there is a lot of quality control rules enforeced - e.g. reasonable values and consistent metadata.

I am very interested in learning how this process could be implemented in a less cumbersome and more user friendly way ... one approach that would be useful but challenging to implement would be to embed the documentation as popup / helper notes within the data entry user interface, so that it is presented as needed .. eg next to each field. In my experience, researchers who are used to spreadsheets find them much easier to use while data and software oriented researchers loathe the lack of reproducibilty and automation of spreadsheets.

teixeirak commented 6 years ago

@ValentineHerr is currently working on a shiny app to do this (code will be here: https://github.com/forc-db/ForC/tree/master/shiny). It seems to be fairly essential for guiding students through the process of adding data. Bulk upload would be nice as well, but we have less need for it at this point.