search

Rothamsted-Ecoinformatics / farm_rothamsted

Custom farmOS features for Rothamsted Research.
GNU General Public License v2.0
6 stars 1 forks source link

Data Extraction: Export quantities for Plans #326

Open aislinnpearson opened 2 years ago

aislinnpearson commented 2 years ago

Following on from issue #281 you can currently view quantities for plot, land and plant assets but it's not a simple process. The plot assets are hard to find and their naming isn't intuitive (currently [experiment code]: [plot_number] which is good for computers, but maybe less obvious to people and you have to know that in order to do the search). It also becomes an issue when you want to search for the quantities associated with a large number of plots, and you have to type them all in. The important thing though is that it works for plots (as such labelled should do instead of must do), so I have added this as a nice to have data extraction issue which we can resolve after field net is replaced, following on from this comment: https://github.com/Rothamsted-Ecoinformatics/farm_rothamsted/issues/281#issuecomment-1291870737

This is also an issue with the commercial crops, where multiple csv's have to be downloaded in order to get all the information about a specific location or asset.

Update: this issue was partially addressed when we introduced 'Extract CSV' as part of the OpenTEAM work, but we now need to be able to specifically extract quantities for the experiments. See also #637

As an additional note, it would be nice to be able to extract different types of quantities, but perhaps this needs to be a separate issue?:

mstenta commented 2 months ago

farmOS v3.3.0 will include a new feature that will allow for exporting log quantities from a list of logs (rather than just from the list of quantities that farmOS offers now). This means you'll be able to open a plan, view all the logs associated with the plan, and export all the quantities associated with those logs.

Here is the PR for that: https://github.com/farmOS/farmOS/pull/861

However, we realized on a recent Rothamsted call that there is still a "gotcha" that affects Rothamsted's use of this specifically. If you export quantities of multiple types (eg: standard and material), then the bundle-specific fields are not included in the CSV. This means the material_type column will be missing in most cases.

@paul121 and I are still thinking about potential solutions to this. It will most likely need to be tackled in a follow-up to that core pull request. In the meantime, @paul121 set @aislinnpearson up with a script that can be used for CSV exports.