This repo is taking shape based on discussions at the 2023 PICES Annual Meeting and the 2024 World Fisheries Congress. Check back frequently and leave some feedback. All components are up for debate at this stage, so join in early to shape the project!
Poster from WFC 2024 in Seattle and an overview presentation of the approach from PICES 2023 in Seattle are available here.
This repository currently covers 7 data sets of environmental covariates:
See the summary of current coverage for an inventory of environmental covariates extracted from those sources so far.
See the comparison of PDO covariates for an example of a closer look at some of the data, set up for easy collaborative editing.
See the scope of current data sets for a high-level comparison of the source data sets.
Suggestions for additional data sets can be added on the To Do List.
Important Warning: Environmental covariates in this repository are mapped onto the year of measurement. Before use, they need to be lined up properly for each specific salmon spawner and recruit data set based on the assumed mechanism of interaction and the life history of the stock. This also means that direct comparisons between diverse environmental indicators need to be approached with caution (e.g., pair-wise correlations).
We've used a lightweight, human-centered data management system to compile and manage source data for Yukon River Chinook Salmon run reconstruction (Pestal et al. 2022). 33 people from 11 organizations contributed to the data compilation and review process, consolidating quality-controlled records and detailed meta-data for 41 individual assessment projects covering a basin of 850,000 km2. Data summaries and model input files had to be constantly updated as data review and model development progressed concurrently. A worked example of the approach is available in a github repository. The worked example explains the structure of the underlying data management system and how the components fit together. It includes tips on getting started with git/github and for setting up automated reports using markdown, plus wiki pages with background information on human-centered design and excerpts from interesting papers.
We are now trying to test whether this approach can be scaled up from a relatively contained setting to a more typical data management situation. The Yukon Chinook data compilation was part of high-priority project with clear terms of reference for the project, a large technical working group, and dedicated resources. More typically, data management systems are not implemented for just a single specific analysis or project, but developed and maintained as a more general-use resource (e.g., a data base of all the regional salmon spawner estimates). This changes the human dynamics of contribution, from a highly-focused and time-constrained deliverable for targeted outcome, to a long-running commitment, often without any clear and direct result for the individual contributors.
In this type of setting, a data management system that minimizes procedural and technical hassles has the potential to greatly improve the data resource.
An open-source data resource is a good test for this idea, because individual contributors are not obligated to work through a steep learning curve or drop other tasks because a senior manager made a request.
Data management systems are easier to design and maintain if their scope is clearly bounded. For this project, we are currently working with the following bounds:
For now, we are excluding biological covariates, such as copepod diversity, winter ichthyoplankton biomass, or catch/abundance of other salmon species (e.g., using pink salmon abundance as a covariate in sockeye salmon models). In cases where biological variables are part of a multi-variable index, such as the NOAA Ocean Conditions Index, we include the overall index, and the individual environmental components of the index.
Along the way, we are also compiling an inventory of any interesting sources of environmental information that come up, but are out of scope for this project. These are stored on a wiki page.
DATA Folder: Includes all the contributed data with detailed descriptions of what they are and how they have been used.
CODE Folder: includes all the functions and code needed to process the contributed data into a single consistent data set and generate various summaries.
OUTPUT Folder: includes the merged data set and summaries.
NOTES Folder: includes a short summary of the available data and brief explorations of points of interest. Notes are kept in markdown to directly pull in plots generated in other parts of the repository and allow for easy collaborative editing.
You have four options for browsing through this repository:
If you are mainly interested in getting the data, browse through the descriptions of each data set in the DATA, then download either an individual data file or the full merged data set in the OUTPUT folder. To download a data set, click through until you see the contents of the csv file displayed, then click on the icon for "download raw file" in the top right corner.
If you just want to have a quick look at the data structure and code, follow the links above for specific examples of file structure and code.
If you want a local copy of all the key files, click on the green "<> Code" button near the top of this page, and select "Download Zip".
If you really want to dig into the details and are familiar with RStudio and git, just clone this repository, then open the RStudio project file Open-Source-Env-Cov-PacSalmon.Rproj and start looking through the code scripts.
If you have any questions, comments, or ideas for extensions, you can leave a note on the issues page by clicking on New Issue. Make sure to give it an informative title.
You can also scroll through any other open issues to follow the discussion and contribute ideas.