josephholler
commented
12 months ago Some other revision suggestions:
- [x] You could streamline the code by loading a CSV with the acs_variables, spielman_acs_variables, and alias
- [x] is there an option to wrap code in juypter code blocks?
- [x] I think it'd be OK to split the layer metadata into two files: geographic (everything up to the 'variables' and 'data_dictionary'). Then you could load the geographic metadata in from .md file to print in the notebook and avoid writing or saving it in the notebook. The 'variables' field could then just say "see data dictionary"
USA Counties shapefile
- [x] temporal coverage: this is important, because boundaries change. too bad we don't know, other than it must be prior to when the file was committed.
- [x] data quality: normally these shapefiles are designed for a particular scale ratio. without documentation, we could guess... or just say that it was not documented. It might be pretty obvious based on file size alone
USA Counties Cartographic Boundaries
- [x] data quality: if counties is a function, it may have a default option for the scale. see scales here: https://www.census.gov/geographies/mapping-files/time-series/geo/cartographic-boundary.html
Other things
- [x] rounding the land area to the nearest tenth would have fixed the 3 non-null errors. would it also make more errors?, e.g. in calculating pop density?
- [x] the workflow diagrams are really clear. The procedure to calc SOVI and outputs is essentially identical for FEMA regions and states as it was for National-level-- if you changed the first "select" to a generic description, you could probably just use the one workflow.
- [x] the mismatched SoVI scores are all -4.53, suggesting that unless differences in the hundredths place is driving the different rank, then it might just be a difference in assigning a rank when there is a tie.
- [x] FEMA Region SoVI Scores & Rankings: edit "This results was successfully reproduced." to "these results" or "this result"
- [x] confusing file system: change data_metadata.csv to procedure_metadata.csv to more closely match the example
- [x] conclusions: also note that a fully executable research compendium would reduce the risk of transcription errors.
- [x] metadata lineage: let's replace the text link to the original repository with the URL to the repository, so that the source will be clear even if the report is printed or stored in a system with no hyperlinks
doabell
environmentfolder? Can the template.ipynbfile also store environment information in the environment folder?Abstract
The Social Vulnerability Index
The Spielman et al (2020) paper is in turn a reproduction and reanalysis of:
Spielman et al (2020) developed methods to evaluate the internal validity and construct validity of the Cutter, Boruff and Shirley (2003) Social Vulnerability Index (SoVI). First, they reproduce a national SoVI model and validate it against validated our SOVI calculation function against an SPSS procedure provided by the original research group (Hazards Vulnerability Research Institute at University of South Carolina). The original SoVI uses 42 independent z-score normalized variables from the U.S. Census, reduces the data to factors using Principal Components Analysis, selects the first XX factors, inverts factors with inverse relationships to social vulnerability, and sums the factors together, and calculates a z-score. The reproduced SoVI model was slightly different than the original model due to changes in U.S. Census data, using only 28 variables.
Spielman et al. modify the geographic extent of the SoVI calculation by recalculating SoVI for each of ten Federal Emergency Management Agency (FEMA) regions, and again for a single state or cluster of states within each of the ten regions, resulting in 21 total indices. Internal validity is assessed by calculating the spearman rank correlation coefficient of the SoVI score for counties in the state model compared to the FEMA region model and national model. Construct validity is assessed by summing the loadings for each input variable across the PCA factors in each model and calculating the variables sign (positive/negative) and the rank of the variable's total loading compared to the other variables. These signs and ranks are summarized across all 21 versions of the SoVI model with regard to the number of times the sign is different from the national model and the distributions of ranks.
In this reproduction study, we will attempt to reproduce identical SoVI model outputs by comparing to the outputs in the original GitHub repository, identical correlations between SoVI models as shown in Table 2, and identical reversals, and mean and range ranks as shown in Figure 2.