Upload Poplar Data Set

[echeng7]

Data are in the all.yielddata Google doc spreadsheet here: https://docs.google.com/a/illinois.edu/spreadsheets/d/1hrvMTMc7scvQFgit7QqF68vfxxNufNC--3RJfSDhne4/edit#gid=1338120831

Yields have been updated as much as they can be. Again, ignore the grayed out ones. Thanks!

@dlebauer @gsrohde

[gsrohde]

@echeng7

Problems:

• [ ] 1. Record 7961 (line 121): This has cultivar "Beaupre" which is a cultivar of Populus trichocarpa × deltoides, not Populus trichocarpa. I'm guessing this is a typo, and that the species on this line should be Populus trichocarpa x deltoides like the lines above it (which also have cultivar Beaupre).
• [ ] 2. The species name "P. trichocarpa x balsamifera" should be "P. trichocarpa x P. balsamifera" (line 6).
• [ ] 3. In the rows where the cultivar is NM5, the species has to be "Populus maximowiczii × P. nigra" instead of "Populus nigra × P. maximowiczii". Some of the yields rows were wrong in the database used used Populus nigra × P. maximowiczii". I've correct them.
• [ ] 4. The species "Populus rasumowskyana" is not in the database. There is a "Populus × rasumowskyana" however.
• [ ] 5. There is no sitename "Kannus" in the database.
• [ ] 6. Species "P. deltoides" should be "Populus deltoides".
• [ ] 7. Site name "Wildehausen" should be "Wildeshausen".
• [ ] 8. The site name "Bullstofta" should really be "Bulltofta". I've corrected these in the database.
• [ ] 9. I changed the database entry for "P. deltoides x P. nigra" so that there is a space after each period. In the data sheet, there sometimes is and sometimes isn't a space between "P." and "nigra". There should be one always.
• [ ] 10. The entries that just say "P. deltoides x nigra" should say "P. deltoides x P. nigra" .
• [ ] 11. "P. tremula × tremuloides" should be "Populus tremula × tremuloides".
• [ ] 12. Site name "Terai belt" should be "Tarai belt".
• [ ] 13. "Popular spp." should be "Populus spp."
• [ ] 14. Species "P. trichocarpa × P. deltoides" should be "Populus trichocarpa × deltoides".
• [ ] 15. If this is like the other data set where none of the dates should be updated, then you need to elminate or rename the "date" column. You can call it "date-ignore" and the upload software won't complain about the dates being in the wrong format. On the other hand, if you want to use the date column, you have to supply a month and day for each date; dates must be in the format YYYY-MM-DD.

[echeng7]

I have fixed all the problems above. I don't know if it is just me but I keep getting an error when I check the bulk upload with rows 408, 409, 414, and 415. The cultivars are copied straight from the database with their respective species onto the spreadsheet but when it is uploaded, it changes. Can you look into this for me?

@gsrohde

[gsrohde]

@echeng7 The problem in rows 408 and 414 is that although the cultivar name "Wedesbuttel 95 x Turesson 141" exists in the database, it is associated with the species "Populus tremula × tremuloides", whereas the upload file uses the species name "P. tremuloides × P. tremula". Although the latter is a valid species name (i.e., it is listed in the species table in the database), it isn't the species name the cultivar in question is associated with. (Those two species names probably are actually different designations for the same species, but we'll leave that question aside for now.)

Similarly, in rows 409 and 415, the cultivar name "Ihlendiek 3 x Wedesbuttel 66" is in the database, but it is associated with the species with name "P. tremuloides × P. tremula" whereas the data sheet calls this "Populus tremula × tremuloides".

I think the correct solution here is to change the species name in those four rows to match what the database says is the species name corresponding to those cultivars, but please verify this with David.

I think the "mouseover" error message here needs to be changed. Instead of saying "Not found in cultivars table" it should say something like "cultivar name doesn't match species given", and the error message "Not found in cultivars table" should only be used if there is no cultivar at all in the database with that name, no matter what the species.

[echeng7]

@dlebauer

[dlebauer]

Yes, Scott's solution - change the species name - sounds correct to me

On Monday, May 11, 2015, echeng7 notifications@github.com wrote:

@dlebauer https://github.com/dlebauer

— Reply to this email directly or view it on GitHub https://github.com/PecanProject/bety/issues/275#issuecomment-101039539.

[dlebauer]

@gsrohde this dataset is ready to go.

[gsrohde]

I get a few errors on ebi-forecast. I think you must have modified a species name on pecandev but not on the live site. I assume the solution is to change "Populus tremuloides × tremuloides" to "p. tremuloides × p. tremuloides" so as to match the CSV file. Do you want me to do this? Did you have to change anything else?

[dlebauer]

I changed it to "Populus tremuloides × tremuloides" in the csv file instead of changing the database again.

[gsrohde]

OK, this is uploaded now. Again, I need to know what more to do with this set.

Just so you don't get alarmed, I should mention I've been setting the date for these sets to 1000-01-01 since I've been assuming we will copy data from the new rows to the old rows but won't copy the date column values.

[gsrohde]

I've updated the old rows and deleted the new ones.

The scripts I used to do this, and a dump of the old rows before and after the updates is stored on ebi-forecast at /home/n-z/srohde/BETYdb_database_maintenance/poplar_bulk_upload/SQL-fixes.