more speciescomplex work

[lizzieinvancouver]

@dbuona @cchambe12 Sorry for this request but can we re-write the speciescomplex code so that it works later in the code ... ? Right now we do it early but I think we could delete there and move down ... so it can be used several times in bbleadin.R

(1) On all the bb data (bb.stan.allphoto) (2) On just the expphoto data (current bb.stan)

[cchambe12]

I think this is done! I only added bb.stan.allphoto and bb.stan for now. Would someone mind checking double this to make sure it's working correctly?

[lizzieinvancouver]

@cchambe12 @dbuona Could you take another look at the speciescomplex code? We're going from 181 species to just 21 now, which is much lower than we have ever ended up before, even though the rows of data are not so different at all.... It would also be good to clean up the commenting on the speciescomplex.R code, then I can help check it (add cleaner notes throughout and overview of criteria at the very top). Thanks!

[cchambe12]

I have a guess of why this is, I can take a look!

On Mon, Oct 29, 2018 at 3:55 PM lizzieinvancouver notifications@github.com wrote:

@cchambe12 https://github.com/cchambe12 @dbuona https://github.com/dbuona Could you take another look at the speciescomplex code? We're going from 181 species to just 21 now, which is much lower than we have ever ended up before, even though the rows of data are not so different at all.... It would also be good to clean up the commenting on the speciescomplex.R code, then I can help check it (add cleaner notes throughout and overview of criteria at the very top). Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lizzieinvancouver/ospree/issues/219#issuecomment-434055597, or mute the thread https://github.com/notifications/unsubscribe-auth/ASiQ0bhkBWiFbmM3__G0u32c07YLnSEDks5up10hgaJpZM4X2yM7 .

-- PhD Candidate Harvard University | Arnold Arboretum Department of Organismic and Evolutionary Biology

[cchambe12]

@lizzieinvancouver If you use sort(unique(bb.stan$complex.wname)) then there are complexes shown. Using this method, there are 40 species/complexes for the bb.stan.allphoto and 21 species/complexes for the bb.stan. I tried to add more notes, if it still seems surprising or something is unclear let me know and @dbuona and I can sit down and go through it together!

[lizzieinvancouver]

@cchambe12 Okay. Here's what we have lost in species since June 2018:

Abies_alba Acer_complex Acer_pseudoplatanus Aesculus_hippocastanum Betula_complex Cornus_alba Cornus_mas Fraxinus_complex Fraxinus_excelsior Larix_decidua Populus_complex Prunus_avium Prunus_complex Pseudotsuga_menziesii Quercus_rubra Robinia_pseudoacacia Sorbus_aucuparia Symphoricarpos_albus Syringa_vulgaris Tilia_cordata Vaccinium_complex

Could you check where these are lost in the speciescomplex code? (Maybe they are all in ramped photoperiod studies? But this still seems a heavy hit to lose so many to such a small change in the data ... these species were all included when we were using only exp and ramped photoperiod data)

[lizzieinvancouver]

Gained (probably due to finalized chilling so not a problem): Actinidia_deliciosa Vitis_vinifera

[cchambe12]

@lizzieinvancouver Back in June, Dan and I just made a new data frame for species that included all ospree data. So the species list was made including all respvars. After we fixed this in September, the species list was parsed down significantly to the list below, which is the same as the current bb.stan.allphoto species list.

[1] "Abies_alba" "Acer_complex" "Acer_pseudoplatanus" "Acer_saccharum" "Actinidia_deliciosa"
[6] "Aesculus_hippocastanum" "Alnus_glutinosa" "Alnus_incana" "Betula_pendula" "Betula_pubescens"
[11] "Carpinus_betulus" "Cornus_alba" "Cornus_mas" "Corylus_avellana" "Fagus_sylvatica"
[16] "Fraxinus_complex" "Fraxinus_excelsior" "Larix_decidua" "Malus_domestica" "Picea_abies"
[21] "Populus_tremula" "Prunus_avium" "Prunus_complex" "Prunus_padus" "Pseudotsuga_menziesii" [26] "Quercus_complex" "Quercus_faginea" "Quercus_ilex" "Quercus_petraea" "Quercus_robur"
[31] "Quercus_rubra" "Ribes_nigrum" "Robinia_pseudoacacia" "Salix_complex" "Sorbus_aucuparia"
[36] "Symphoricarpos_albus" "Syringa_vulgaris" "Tilia_cordata" "Ulmus_complex" "Vitis_vinifera"

[lizzieinvancouver]

@cchambe12 @dbuona I am not sure this answers my query, but maybe I am missing something?

I think we need to think about the cut-offs we have imposed in speciescomplex.R and if they make sense (unfortunately I don't think we all thought about this a lot back in the spring). One way to start thinking about this would be to understand why the species I listed above are now being lost ... when in June they were included.

Another way is to plot more of the species covariates, which I will do.

[lizzieinvancouver]

@cchambe12 Could you make up a second spcomplex f(x) that only requires the species (or complex) to be present in more than one datasetID?

[cchambe12]

@lizzieinvancouver that is currently how we have the code -- needs to be in at least 2 or more datasets. I made a new spcomplex f(x) though that includes species that are in one dataset but manipulates more than one cue just to see. For the subset that includes all photoperiod treatments there are 191 complexes and for only experimental photoperiod there are 166 complexes.

As an aside, the "use.allspp" command is causing an error for me - Error in eval(ei, envir) : object 'use.allspp' not found

[cchambe12]

@lizzieinvancouver All set! It adds only 3 more species to all photoperiod types but 8 more species/complexes to our bb.stan data frame. I added the new complex code to bbstanleadin.R and added new data frames with .onecue added to the end of each data frame name so we can see how they compare to the bb.stan original versions.

[lizzieinvancouver]

@cchambe12 Thank you!

[lizzieinvancouver]

Filing some notes from the big data change we say between July and October 2018 (was in deleted file 'quickCompareBBdataNotes.txt)....

8 Jun 2018 versus 8 October 2018 ...

chavarria09 (Prunus) -- added photoperiod (was ambient) falusi96 -- added previously missing photoperiods guak98 -- added photoperiod (was ambient) jones12 -- added photoperiod (was ambient) rinne97 -- added photoperiod (was ambient) sanperez10 -- added photoperiod (was ambient) caffara11b -- changed chill portions gansert02 -- added photoperiod (was ambient) gianfagna95 -- added previously missing photoperiods gomory15 -- added photoperiod (was ambient) granhus09 --- can't figure out heide12 -- added a few chill rows (chill_portions--55.84854) lam37 -- added photoperiod (was ambient) linkosalo06 -- added photoperiod (was ambient) morin10 -- added photoperiod (was ambient) nishimoto95 -- added previously missing photoperiod partanen01 -- corrected some photoperiods (19.1673 and 10.1675 etc.) partanen98 -- corrected some photoperiods (similar to above) schnabel87 -- added photoperiod (was ambient) sonsteby13 -- added photoperiod (was ambient) sonsteby14 -- added photoperiod (was ambient) yazdaniha64 -- added photoperiod (was ambient)