International-Soil-Radiocarbon-Database / ISRaD

Repository for the development and release of ISRaD data and tools
https://international-soil-radiocarbon-database.github.io/ISRaD/
24 stars 15 forks source link

Improvements to delta-delta function #175

Closed aahoyt closed 4 years ago

aahoyt commented 5 years ago

Delta-delta: dd14c = 14c of sample - 14c of atmosphere in year of sample collection

Suggested updates (1) Use Graven et al, 2017 dataset setup for modeling purposes, rather than Hua et al, 2013 Why? -Goes further in time, avoiding issues with extrapolation -Directly provides annual means -Zones are more straightforward (three zonal bands representing the Northern Hemisphere (north of 30 N), the tropics (30 S–30 N) and the Southern Hemisphere (south of 30 S). Much more convenient than the zone shapes in Hua et al Link: http://eprints.keele.ac.uk/4844/1/2017Graven_Compiled%20records%20of%20carbon%20isotopes%20in%20atmospheric%20CO2%20for%20historical%20simulations%20in%20CMIP6.pdf -Adding an ISRaD_extra column for atm_zone would be convenient too, but less necessary if we use these simple zones

(2) Annual mean atm value should be fine, don't need to spline curve and get values at higher temporal resolution. Most studies only have obs_date_y. Higher resolution wasn't used in current function implementation anyway.

(3) Year used for atmospheric values should not always reference the same tab. (a) Use lyr_obs_date_y: -lyr_14c -inc_14c -frc_14c (want to use year sample was collected, not when the lab procedure was done. these often differ) (b) Use flx_obs_date_yr for flx_14c (no layer, referenced to profile) (c) Use ist_obs_date_yr for ist_14c (no layer, referenced to profile)

aahoyt commented 5 years ago

Simplified code chunk for Graven data, needs to be adapted to each table & by atm zone region

Read in Heather Graven's atmospheric record:

She reports mean annual values of 14c, years are midpoints.

atmGraven <- read.csv("GRAVEN_2017_TableS1_formattedForR.csv") %>% mutate(lyr_obs_date_y = Date - 0.5) #round down the year & name column the same to facilitate join

inc_data %>% left_join(atmGraven) %>% #join atm record using lyr_obs_date_y mutate(dd14c = inc_14c - NHc14) # subtract atm (varies by zone)

aahoyt commented 5 years ago

Sorry for weird formatting above. Relevant files should be attached this time.

GravenAtm.zip

jb388 commented 5 years ago

@aahoyt Should I add this to the actual codebase, i.e. the delta.delta fx? Obviously the code will need to be tweaked for each table and atm zone, as you mention above.

Also, have you looked at forecasting the curve, e.g. to 2020, with the Graven dataset? We'll still need to forecast to run delta-delta function for the most recent data in ISRaD. I doubt it matters, but possibly the higher temporal resolution of the Hua dataset was useful for improving the forecasted data.

So, in order to do this we will need to host the data in the Keeper folder and reference the directory when building the database. (Basically what I just did with the PET data---not hard). That's the simplest approach I can think of, unless we wanted to host the .csv file in the repository (since it's very small), so that it would be accessible to all users. Seems like that's pretty easy, too.

jb388 commented 5 years ago

As an alternative to hosting the Graven data on the website we could also include the dataset directly in the package. Obviously it's published, but would there be any issues with hosting the dataset on our site or including it in the package? Not sure if there are special clauses in the data sharing agreement for things like this.

@crlsierra Any suggestions or recommendations from what you did with SoilR?

crlsierra commented 5 years ago

In SoilR we added the atmospheric radiocarbon datasets as R data objects. This is a simple way to make them available as part of the package. Another option would be to simply load the dataset from a script that reads the data from a remote source, unzip, and loads in the current environment. See this discussion.

greymonroe commented 5 years ago

fixed in recent updates. Graven data is included in R package

aahoyt commented 5 years ago

I've been manually QAQC'ing the delta-delta function to make sure it is fully correct. Does the delta-delta function reference the year of sample collection for all tabs? I think it still hasn't been updated? (see code below)

Desired date ref (see mar10-c above) would be the year of sample collection, which in many cases is given by lyr_obs_date_y, rather than the date of the lab procedure (eg incubation or fractionation).

Year used for atmospheric values should not always reference the same tab. (a) Use lyr_obs_date_y: -lyr_14c -inc_14c -frc_14c (want to use year sample was collected, not when the lab procedure was done. these often differ) (b) Use flx_obs_date_yr for flx_14c (no layer, referenced to profile) (c) Use ist_obs_date_yr for ist_14c (no layer, referenced to profile)

calculate del del 14C

# flux
database$flux$flx_graven_atm<-calc_atm14c(database$flux, "flx_obs_date_y")$atm14C
database$flux$flx_dd14c <- database$flux$flx_14c-database$flux$flx_graven_atm
# layer
database$layer$lyr_graven_atm<-calc_atm14c(database$layer, "lyr_obs_date_y")$atm14C
database$layer$lyr_dd14c <- database$layer$lyr_14c- database$layer$lyr_graven_atm
# interstitial
database$interstitial$ist_graven_atm<-calc_atm14c(database$interstitial, "ist_obs_date_y")$atm14C
database$interstitial$ist_dd14c <- database$interstitial$ist_14c-database$interstitial$ist_graven_atm
# fraction
database$fraction$frc_graven_atm<-calc_atm14c(database$fraction, "frc_obs_date_y")$atm14C
database$fraction$frc_dd14c <- database$fraction$frc_14c-database$fraction$frc_graven_atm
# incubation
database$incubation$inc_graven_atm<-calc_atm14c(database$incubation, "inc_obs_date_y")$atm14C
database$incubation$inc_dd14c <- database$incubation$inc_14c-database$incubation$inc_graven_atm
jb388 commented 5 years ago

@aahoyt Are you working on this right now?

aahoyt commented 5 years ago

Yes, currently trying to figure out the issue & make sure the delta-delta is fully correct before we put it on CRAN. But no, I am not making any changes to the code - if you want to do that it would be great.

Also, I still have some questions about matching the Graven year to the ISRaD year - coming soon...

jb388 commented 5 years ago

@aahoyt @greymonroe Just to prevent duplication of work---I am fixing the delta-delta function right now.

jb388 commented 5 years ago

OK, this should be fixed now. @aahoyt mind double-checking that the data make sense now?

greymonroe commented 5 years ago

Did you rebuild the databases?

On Jun 22, 2019, at 6:09 PM, Jeff B notifications@github.com wrote:

OK, this should be fixed now. @aahoyt https://github.com/aahoyt mind double-checking that the data make sense now?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/International-Soil-Radiocarbon-Database/ISRaD/issues/175?email_source=notifications&email_token=AD7HB7HFWTLJIJDGPRDYAKTP3ZFCXA5CNFSM4G45MRZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYKMNWQ#issuecomment-504678106, or mute the thread https://github.com/notifications/unsubscribe-auth/AD7HB7ELZTQBX6R26ZLPGWTP3ZFCXANCNFSM4G45MRZQ.