terraref / reference-data

Coordination of Data Products and Standards for TERRA reference data
https://terraref.org
BSD 3-Clause "New" or "Revised" License
9 stars 2 forks source link

canopy cover season 4 #283

Closed rleontie closed 4 years ago

rleontie commented 4 years ago

@dlebauer please upload canopy cover extractor results to BettyDB.

Results generated by @ZongyangLi can be found here

dlebauer commented 4 years ago

should I upload and replace existing data?

dlebauer commented 4 years ago

On its way. I've converted from fraction [0,1] to percent [0,100] and truncated values to 3 significant figures:

filenames <- dir("~/Downloads/canopy_cover_season4", full.names = TRUE, pattern = 'csv')
require(data.table) ## 1.9.2 or 1.9.3
data <- rbindlist(lapply(filenames, fread))
#hist(data$canopy_cover)
library(dplyr)
dir.create('~/Downloads/canopy_cover_season4/new/', showWarnings = FALSE)
f <- function(x){
  y <- fread(x)
  z <- y %>% mutate(canopy_cover = signif(canopy_cover*100, digits = 3),
                    method = 'Green Canopy Cover Estimation from Field Scanner RGB images')
  readr::write_csv(z, gsub('2017', 'new/updated_2017', x))
}
lapply(filenames, f)

command used to upload data:


for f in updated*.csv 
do curl -H "Content-Type: text/csv" -X POST --data-binary \
   @$f https://terraref.ncsa.illinois.edu/bety/api/v1/traits.csv?key=SECRET
done
dlebauer commented 4 years ago

Uploading finished - 54710 new records!

dlebauer commented 4 years ago

updating treatments

begin;
update traits set treatment_id = 6000000024 where site_id in (
select id from sites where sitename ~ 'Column (5|6|7|8|12|13|14|15|16)' 
              and sitename like 'MAC Field Scanner Season 4%') and treatment_id is NULL;
update traits set treatment_id = 6000000026 where site_id in (
select id from sites where sitename ~ 'Column (1|2|3|4|9|10|11)' 
              and sitename like 'MAC Field Scanner Season 4%') and treatment_id is NULL;
commit;