search

terraref / computing-pipeline

Pipeline to Extract Plant Phenotypes from Reference Data
BSD 3-Clause "New" or "Revised" License
23 stars 13 forks source link

shape files for seasons MAC field scanner seasons 1 and 2 #361

Closed rickw-ward closed 6 years ago

rickw-ward commented 7 years ago

Description

We want to ensure that the plot-level means we generate for Terra MAC UAV flights are made using polygon coordinates and plot/entry attribute information that correspond to the equivalent used in the batch extractors under development. Is it possible for your team to generate shp file sets for seasons 1 and 2?

Completion Criteria

dlebauer commented 7 years ago

Unless @jterstriep or @max-zilla has a better idea, I can do this using psql2shp funcition in PostGIS following (https://gis.stackexchange.com/questions/55206/how-can-i-get-a-shapefile-from-a-postgis-query).

If you prefer it is also possible to directly connect qgis or arcmap to the database. https://terraref.gitbooks.io/terraref-documentation/content/accessing-betydb-with-arcmap.html

related: see also https://github.com/terraref/reference-data/issues/81

rickw-ward commented 7 years ago

I'll figure out the direct connection, but not this week. Hoping to generate plot level means tomorrow.

I assume the shp file sets will contain range and column information- additional info on location attributes is welcome.

dlebauer commented 7 years ago

@rickw-ward see if these work mac_plots.zip

Here is the code that I used: 

pgsql2shp -f season1.shp -h localhost -u bety -P bety bety \ 
         "SELECT sitename, geometry FROM sites WHERE sitename like '%Season 1 Range%'"
pgsql2shp -f season2.shp -h localhost -u bety -P bety bety \
         "SELECT sitename, geometry FROM sites WHERE sitename like '%Season 1 Range%'"

Note

Let me know if you need additional information

rickw-ward commented 7 years ago

checking now

rickw-ward commented 7 years ago

@dlebauer please compare shp file set you sent with those in zips below to see offset in E/W axis. Ours are for inner two rows only, but that doesn't explain offset. The link below is to an in-season NDVI orthomosaic that shows actual rows (and foundations of gantry and sprinkler heads).

MAC 2 row season 2 sorghumexpfall2016v5_lblentry_1to7.zip

foundation and sprinkler shp files.zip

https://drive.google.com/open?id=0B5btztQSCEFxa1hJS3FpTm5mUnM

rickw-ward commented 7 years ago

@dlebauer This screen capture shows shift between our two plot polygons. Sorry- I still can't consolidate into one directory.

capture

dlebauer commented 7 years ago

I'll take a closer look at the files you sent later today.

Neither version eeds to capture the plot perfectly. I know we discussed at length last year... but perhaps we should create a new set that excludes the range ends and is better centered between columns.

rickw-ward commented 7 years ago

I'll use my .shp files for season 2. Perhaps the E/W shift of the shp file you sent is due to the conversion formula between coordinate reference systems?

dlebauer commented 7 years ago

Hopefully it isn't because of conversion between coordinate reference systems. There are obvious limitations to using different definitions of plot boundaries.

My understanding is that our database contains definitions from the F13_TERRA_field_book that were measured with the gantry and were to be considered the canonical plot definitions. That does not make these 'better', and I can change our definitions, but this would need to be done ASAP as we are launching our first re-analyses tomorrow or Friday.

dlebauer commented 7 years ago

looks like this discussion from last fall: https://github.com/terraref/computing-pipeline/issues/187 should have been resolved by https://github.com/terraref/computing-pipeline/issues/196.

It is unclear why there is still a discrepancy. In https://github.com/terraref/computing-pipeline/issues/187 I proposed that we could have multiple definitions of plots, but now I think we should try to identify a canonical definition of the plots.

dlebauer commented 7 years ago

@rickw-ward for season 2, it looks like we have defined the same plots as those in your shape files in addition to the other set of plots that do not align. In the image below you can see the plots from your shapefile (grey) with the plots in the database on top (thin red line):

image

I've deleted the plot definitions that do not align with the ones that you have from the database:

delete from sites where sitename like '%Field Plot%Season 2'
delete from citations_sites where site_id in (select id from sites where sitename like '%Field Plot%Season 2')
delete from sites_cultivars where site_id in (select id from sites where sitename like '%Field Plot%Season 2')
rickw-ward commented 7 years ago

Good news David. Thanks.

dlebauer commented 6 years ago

Here are the shapefiles for season 1, 2, 4

season124shpfiles.zip

Let me know if these work for you