The ALFAM2 dataset (database) on ammonia loss through volatilization from field-applied manure. This repository contains the "ALFAM2 dataset", the code used to generate it from data files submitted by researchers who made the emission measurements, and those original data files.
The dataset consists of two files: one with plot- and one with measurement-level observations.
See the headers directory here for variable descriptions.
Files can be merged on the two plot keys or identification codes: pid and pmid.
Files are saved in a compressed format with the extension .gz (gzip) in order to reduce file size.
The easiest way to get the data into R is to read the files directly with data.table::fread().
Alternatively, various utilies can be used to extract (unzip) the files, which are regular comma-separated ASCII text files.
The ALFAM2 data are organized into uptake periods: 1 is for the original ALFAM work, 2 for the work described in this paper, and 3 for the current effort to expand the database.
The latest version will always be in the highest number (3 currently).
Earlier versions are saved to facilitate addition of data without rebuilding older dataset versions while maintaining the option for revising data submitted in an earlier uptake period.
See the Data handling tips below for more information on working with the data.
The best way to refer to this work if you use the data in a publication is to cite the version available through Zenodo. In addition to a digital object identifier (doi), this Zenodo version includes an author list. Alternatively, individual releases can be cited. In either case, be sure to note the version number used in any citation. If you have used a version not included in a release or a Zenodo dataset, please contact me (here)and I will create the necessary release or dataset. Authors may also want to cite (and read!) this paper. But again, include the dataset version number in any citation. You can find the dataset version number in the release tags or in a text file here: data-output/03/data_version.txt.
For a description of the dataset, see this paper: https://doi.org/10.1016/j.agrformet.2017.11.027. For the ALFAM2 model for ammonia emission, with parameter estimation based on the ALFAM2 dataset, see this paper: https://doi.org/10.1016/j.agrformet.2017.11.027.
The simplest way to load the data in R is with the fread() function from the data.table package.
idat <- data.table::fread('ALFAM2_interval.csv.gz')
pdat <- data.table::fread('ALFAM2_plot.csv.gz')Once these two data frames (tables) are created, they can be combined (if needed) with base::merge() or data.table::merge() function.
cdat <- merge(idat, pdat, by = c('pid', 'pmid'))(The by argument is optional here, but it is good practice to be aware of the columns used for merging.)
In Python, the read_csv() function from the pandas package can be used to read the compressed files directly.
import pandas as pd
idat = pd.read_csv('ALFAM2_interval.csv.gz')For some more information, see the analysis directory.
To inquire about submitting data to the ALFAM2 dataset, send a message to sasha.hafner AT bce.au.dk through email or else used this form.
To check submitted data, find your completed template file in this list and follow the instructions you will see at the top of the linked file.
After cloning this repo, the process of creating the ALFAM2 database from submitted data can be completely repeated by running two scripts in R (https://www.r-project.org/) sequentially:
scripts/02/main.Rscripts/03/main.RThe only obvious reason for doing so is to update the database after making a correction to submitted data or data processing code.
(Submitted data, as spreadsheet templates (usually) completed by data submitters, can be found in the data-submitted directory.)
For changes to only the most recent submitted file, only the highest numbered file, for the latest uptake period, needs to be run.
Because of changes in base R and add-on packages over time, seeing R return an error when updating earlier uptake periods, or even the latest period some time after the latest update by database developers, is possible (e.g., this problem was caused by a change in base R).
The record of R and add-on package versions in logs/*/R_package_version.txt should help sort out any of these problems.
analysis demo code for reading and plotting data and creating summariesarchive early versions of the database from before this repository was createdcontributors list of indviduals who have contributed data, used for author listdata-ALFAM1 the original ALFAM database, developed by Sven G. Sommer et al., considered "uptake period 1", and read in by scripts/02/make_database.Rdata-output output: ALFAM2 database files, with latest version in subdirectory with highest uptake numberdata-submitted data as submitted by data submitters, in spreadsheet templates, in some cases with data processing etc. by database developersdocs html logs or summaries of submitted data for checking by submitters or others through this page: https://au-bce-ee.github.io/ALFAM2-data/functions R functions for building database, some from https://github.com/sashahafner/jumbledheaders detailed description of data file headers (variable names)logs various logs created when database is builtscripts the R scripts that are used directly to build the databaseuptake-notes notes on some submitted files