what to do with storage temp and duration?

[FrederikBaumgarten]

number of spp with a storage time: 148 number of spp with storage temp: 133 number of spp with both: 66

This is probably the best way (and only quantitative way) to go forward with storage as a potential driver for germination?

[lizzieinvancouver]

See issue #14

[lizzieinvancouver]

@kengi-neer Would you be willing to take a look at the storage column and think on how we might want to make new columns from it so it is more useful? Ideally we might dream of three columns: duration, temperature, and moisture for storage but I doubt we have enough information from most papers to fill this in and a bunch of columns with NA is not super helpful for analysis. If you can take a look and see what you think, I'd appreciate it.

[kengi-neer]

@lizzieinvancouver and I discussed to integrate the current storage and chilling columns to make three new columns (names pending) using the current system used for chilling ('then' to signify order):

• chillingTemp - string of numeric values detailing temperature conditions starting from collection
• chillingDuration - string of integers detailing duration for each temperature condition starting from collection
• chillingWet - string of boolean values detailing wetness for each temperature conditions starting from collection; data from storage is based on storage humidity and data from chilling is assumed to all be wet

Also, from @DeirdreLoughnan, we could consider 'cold stratification' to be wet chilling.

If we plan on using scarification data to decide when chilling units are calculated, we could add one more column of integers to specify which stage / temperature condition (based on 'then' dividers) was scarification applied.

[lizzieinvancouver]

@kengi-neer This sounds good to me. @DeirdreLoughnan do you see any issues or concerns? I agree we can consider 'cold stratification' to be wet.

Thanks for working on this.

[kengi-neer]

@lizzieinvancouver I have finished cleaning weird values for storage temperature and duration, except for some missing papers (aldridge1992, basaran12, bibby53, li17) and updated the script to combine the storage and chill conditions which should work after updating the cleaning with the missing papers. I'll update my subsetting conditions sometime after (since sometimes not using which() may lead to subsetting some NA rows which I guess may be deleted rows still in memory but I can't be sure), but it currently works fine.

I changed the new column names to be dormancyTemp, dormancyDuration, and dormancyWet, with wetness during storage conditions being based on storageType instead and if it has the word 'moist' in it.

There seems to be a lot of estimation in terms of storage durations though, as well as some short-duration conditions and treatments (e.g. heat treatments) that may be overlooked with our current scraping. I would suggest having someone to try and do a whole "rescraping" using our current cleaned data as basis, also to check if there are any mismatched columns (a lot of study IDs, figures, less frequently temperature conditions).

EDIT: I only cleaned the weird values, not neatly checked each condition such as those mentioned in issue #44. Also, even though we might not be interested in the small duration temperature conditions, I still think it's a good idea to incorporate this "rescraping" approach as we would still need to recheck if the newly stitched storage and chill columns are fine. There are also some seemingly normal values but need some cleaning, such as chilling duration in weeks than days which I only found by chance.

[lizzieinvancouver]

@kengi-neer Sounds good! Thank you for all your work on this. I will try to check this soon (next week most likely), but if @dbuona or @DeirdreLoughnan have time sooner they could take a first look (more eyes always better).

[lizzieinvancouver]

Semi-related to this, I need a volunteer to check cleanStorage.R -- especially the addition of this very not-useful column:

> table(d$storagetemp)

  0 
283 

[lizzieinvancouver]

Thanks to @DeirdreLoughnan for checking cleanStorage.R fully!

[DeirdreLoughnan]

@kengi-neer it was great chatting with you about the combineStorageChill.R code. In reviewing this issue again, I think it best for us to continue discussing the issue here rather than create a new issue, but consolidate key points that are in issue #44.

In summary the storageTemp and storageDuration columns are cleaned, but we still have some work to do to create columns for when storage conditions are cold (between -20 and 10C, see comment in #44 from Aug 2) and "moist".

We discussed:

• revising the cleaning code to create two new columns: dormancyTemp and dormancyDuration, which should only have numeric values
• entries in these columns should be "NA" when there is no storage and no chilling treatments
• entries in these columns should be equal to the values in storageTemp and storageDuration when there is no chilling treatment, storageDuration is not "NA", storageType = c("moist", "moist/cold), and storageTemp is between -20 and 10C
• but if both chillTemp, chillDuration, storageTemp, and storageDuration have values and storageType = c("moist", "moist/cold), the values in dormancyTemp and dormancyDuration should be the combined values of both storage and chilling. In this case, dormancyDuration should be the total number of days at which a seed was between -20 and 10C, so a single integer value.

Thank for your help with this!