search

afsc-gap-products / akgfmaps

Make AFSC bottom trawl survey maps and retrieve map layers
Other
17 stars 6 forks source link

Regional inconsistencies in field names #101

Open sean-rohan-NOAA opened 7 months ago

sean-rohan-NOAA commented 7 months ago

Issue

@Ned-Laman-NOAA @Duane-Stevenson-NOAA

I would like to correct inconsistencies in naming conventions and data types for regional shapefiles during FY24. Different regions use different naming conventions for field names. However, these should be standardized among regions to provide consistent behavior.

Examples

library(akgfmaps)

slope_layers <- akgfmaps::get_base_layers(select.region = "ebs.slope", set.crs = "EPSG:3338")
ebs_layers <- akgfmaps::get_base_layers(select.region = "ebs", set.crs = "EPSG:3338")
goa_layers <- akgfmaps::get_base_layers(select.region = "goa", set.crs = "EPSG:3338")
ai_layers <- akgfmaps::get_base_layers(select.region = "ai", set.crs = "EPSG:3338")
chukchi_layers <- akgfmaps::get_base_layers(select.region = "ecs", set.crs = "EPSG:3338")
ebs_survey_bathy <- akgfmaps::get_survey_bathymetry(select.region = "ebs", set.crs = "EPSG:3338")
Stratum field names
names(slope_layers$survey.strata)
names(ebs_layers$survey.strata)
names(goa_layers$survey.strata)
names(ai_layers$survey.strata)
names(chukchi_layers$survey.strata)

names(slopelayers$survey.strata) [1] "B5" "STRATUM" "geometry" names(ebs_layers$survey.strata) [1] "OBJECTID" "Id" "F_AREA" "Stratum" "Precise_Ar" "SURVEY" "Shape_Leng" "Shape_Area" "geometry"
names(goa_layers$survey.strata) [1] "GOA_STRATA" "GOA_STRA_1" "CONTACT" "CREATE_DAT" "LAST_UPDAT" "STRATUM" "AREA_KM2" "geometry"
names(ai_layers$survey.strata) [1] "AREA" "PERIMETER" "AISTRATA" "AI_STRATA1" "AREA_KM2" "STRATUM" "geometry"
names(chukchi_layers$survey.strata) [1] "AREA_KM2" "LAST_UPDAT" "SURVEY_REG" "geometry"

Survey grid field names
names(slope_layers$survey.grid) # No grid
names(ebs_layers$survey.grid)
names(goa_layers$survey.grid)
names(ai_layers$survey.grid)
names(chukchi_layers$survey.grid) # No grid

names(slope_layers$survey.grid) # No grid NULL names(ebs_layers$survey.grid) [1] "STATIONID" "geometry" names(goalayers$survey.grid) [1] "AREA" "PERIMETER" "GOAGRID" "GOAGRID_ID" "AREAKM2" "PERIMETER" "STRATUM" "ID" "geometry"
names(ailayers$survey.grid) [1] "AREA" "PERIMETER" "AIGRID" "AIGRID_ID" "ID" "STRATUM" "geometry" names(chukchi_layers$survey.grid) # No grid NULL

Survey area names

names(slope_layers$survey.area)
names(ebs_layers$survey.area)
names(goa_layers$survey.area)
names(ai_layers$survey.area)
names(chukchi_layers$survey.area)

names(slope_layers$survey.area) [1] "OBJECTID" "Shape_Leng" "Shape_Area" "geometry"
names(ebs_layers$survey.area) [1] "SURVEY" "AREA_KM2" "PERIM_KM" "geometry" names(goa_layers$survey.area) [1] "STRATUM" "geometry" names(ai_layers$survey.area) [1] "ID" "geometry" names(chukchi_layers$survey.area) [1] "AREA_KM2" "LAST_UPDAT" "SURVEY_REG" "geometry"

EBS 'survey' bathmetry versus regional file based on 1998/1999 GEBCO

Note how many unnecessary fields are included in the bathymetry file.

names(ebs_layers$bathymetry)
names(survey_bathy)

names(ebslayers$bathymetry) [1] "FNODE" "TNODE" "LPOLY" "RPOLY_" "LENGTH" "SHELFGEB" "SHELFGEB1" "DESC" "SOURCE"
[10] "CONTACT" "PREC" "CREATE_DAT" "LAST_UPDAT" "METERS" "geometry"
names(ebs_surveybathy) [1] "FNODE" "TNODE" "LPOLY" "RPOLY_" "LENGTH" "EBSSHELF_A" "EBSSHELF_1" "geometry"

Class types differ among shapefiles

class(ebs_layers$survey.strata$Stratum)
class(bss1$survey.strata$STRATUM)

class(ebs_layers$survey.strata$Stratum) [1] "character" class(bss1$survey.strata$STRATUM) [1] "numeric"

Duane-Stevenson-NOAA commented 7 months ago

Sean,

This sounds like a great idea, and I'm all for it. But I must admit I don't have a comprehensive grasp of what the ramifications might be, or how to go about making these changes.

Thanks for starting the discussion.

Duane

On Thu, Dec 7, 2023 at 8:21 AM Sean Rohan @.***> wrote:

Issue

@Ned-Laman-NOAA https://github.com/Ned-Laman-NOAA @Duane-Stevenson-NOAA https://github.com/Duane-Stevenson-NOAA

I would like to correct inconsistencies in naming conventions and data types for regional shapefiles during FY24. Different regions use different naming conventions for field names. However, these should be standardized among regions to provide consistent behavior. Examples

library(akgfmaps)

slope_layers <- akgfmaps::get_base_layers(select.region = "ebs.slope", set.crs = "EPSG:3338") ebs_layers <- akgfmaps::get_base_layers(select.region = "ebs", set.crs = "EPSG:3338") goa_layers <- akgfmaps::get_base_layers(select.region = "goa", set.crs = "EPSG:3338") ai_layers <- akgfmaps::get_base_layers(select.region = "ai", set.crs = "EPSG:3338") chukchi_layers <- akgfmaps::get_base_layers(select.region = "ecs", set.crs = "EPSG:3338") ebs_survey_bathy <- akgfmaps::get_survey_bathymetry(select.region = "ebs", set.crs = "EPSG:3338")

Stratum field names

names(slope_layers$survey.strata) names(ebs_layers$survey.strata) names(goa_layers$survey.strata) names(ai_layers$survey.strata) names(chukchi_layers$survey.strata)

names(slopelayers$survey.strata) [1] "B5" "STRATUM" "geometry" names(ebs_layers$survey.strata) [1] "OBJECTID" "Id" "F_AREA" "Stratum" "Precise_Ar" "SURVEY" "Shape_Leng" "Shape_Area" "geometry" names(goa_layers$survey.strata) [1] "GOA_STRATA" "GOA_STRA_1" "CONTACT" "CREATE_DAT" "LAST_UPDAT" "STRATUM" "AREA_KM2" "geometry" names(ai_layers$survey.strata) [1] "AREA" "PERIMETER" "AISTRATA" "AI_STRATA1" "AREA_KM2" "STRATUM" "geometry" names(chukchi_layers$survey.strata) [1] "AREA_KM2" "LAST_UPDAT" "SURVEY_REG" "geometry"

Survey grid field names

names(slope_layers$survey.grid) # No grid names(ebs_layers$survey.grid) names(goa_layers$survey.grid) names(ai_layers$survey.grid) names(chukchi_layers$survey.grid) # No grid

names(slope_layers$survey.grid) # No grid NULL names(ebs_layers$survey.grid) [1] "STATIONID" "geometry" names(goalayers$survey.grid) [1] "AREA" "PERIMETER" "GOAGRID" "GOAGRID_ID" "AREAKM2" "PERIMETER" "STRATUM" "ID" "geometry" names(ailayers$survey.grid) [1] "AREA" "PERIMETER" "AIGRID" "AIGRID_ID" "ID" "STRATUM" "geometry" names(chukchi_layers$survey.grid) # No grid NULL

Survey area names

names(slope_layers$survey.area) names(ebs_layers$survey.area) names(goa_layers$survey.area) names(ai_layers$survey.area) names(chukchi_layers$survey.area)

names(slope_layers$survey.area) [1] "OBJECTID" "Shape_Leng" "Shape_Area" "geometry" names(ebs_layers$survey.area) [1] "SURVEY" "AREA_KM2" "PERIM_KM" "geometry" names(goa_layers$survey.area) [1] "STRATUM" "geometry" names(ai_layers$survey.area) [1] "ID" "geometry" names(chukchi_layers$survey.area) [1] "AREA_KM2" "LAST_UPDAT" "SURVEY_REG" "geometry"

EBS 'survey' bathmetry versus regional file based on 1998/1999 GEBCO

Note how many unnecessary fields are included in the bathymetry file.

names(ebs_layers$bathymetry) names(survey_bathy)

names(ebslayers$bathymetry) [1] "FNODE" "TNODE" "LPOLY" "RPOLY_" "LENGTH" "SHELFGEB" "SHELFGEB1" "DESC" "SOURCE" [10] "CONTACT" "PREC" "CREATE_DAT" "LAST_UPDAT" "METERS" "geometry" names(ebs_surveybathy) [1] "FNODE" "TNODE" "LPOLY" "RPOLY_" "LENGTH" "EBSSHELF_A" "EBSSHELF_1" "geometry"

Class types differ among shapefiles

class(ebs_layers$survey.strata$Stratum) class(bss1$survey.strata$STRATUM)

class(ebs_layers$survey.strata$Stratum) [1] "character" class(bss1$survey.strata$STRATUM) [1] "numeric"

— Reply to this email directly, view it on GitHub https://github.com/afsc-gap-products/akgfmaps/issues/101, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANKDWAV2G6U4RCSWSLZT763YIHUH3AVCNFSM6AAAAABALKN436VHI2DSMVQWIX3LMV43ASLTON2WKOZSGAZTCMBZHAZDSNY . You are receiving this because you were mentioned.Message ID: @.***>

-- Duane Stevenson, Ph.D. Supervisory Fish Biologist Groundfish Assessment Program NMFS, Alaska Fisheries Science Center

Ned-Laman-NOAA commented 7 months ago

From the Gulf-Aleutian group we are beginning discussions involving Zimm about re-gridding the Gulf and Aleutian survey areas. The historic grids were are presently using are not reproducible and so need to be brought into the current century of spatial data. We can address concurrency of terminology across those survey area grids when we get to the implementation and operationalization of the new sampling grids. Including the Bering spatial information in this discussion will be important at that time.

sean-rohan-NOAA commented 7 months ago

@Ned-Laman-NOAA @MarkZimmermann-NOAA

Glad to hear you're thinking about regridding the GOA and AI survey grids. In case it's helpful, here is a link to the metadata file for the current EBS grid that was generated on October 28, 2009 when Angie Grieg created a survey grid for the combined EBS/NBS regions:

Although there are some issues with both the metadata and the grid (#105), the metadata describes how the files was created and has useful for sorting out issues when folks have seen differences in the grids that are floating around. The grid itself is larger than the actual survey grid, but it includes stations outside of the grid that we've sampled in the past for special projects (e.g. near-shore stations in Bristol Bay in 2012).

As you're regridding, I would request that consideration be given to including descriptive metadata in the files. Ideally, it would be helpful to be able to determine how and when the files were created, along with a description of the process used to create the file. That sort of metadata is missing from the current AI (2023 version; no metadata) and the metadata for the current GOA grid shapefile (link) is relatively sparse. However, it seems like the file was created on April 14, 2009.