azavea / fb-gender-survey-dashboard

Interactive exploration of global results from the Facebook "Survey on Gender Equality At Home
https://gender-survey-dashboard.netlify.app/
Apache License 2.0
2 stars 2 forks source link

Data is indexed incorrectly due to different ordering of variables in codebook vs data #33

Closed mmcfarland closed 3 years ago

mmcfarland commented 3 years ago

Overview

The issue is in the order of the variables as they appear in the Codebook vs in the Data tab. In the codebook, the variables are in alpha/numeric order (b1, b2, ..., b10) but in the data, the columns are in alphabetical order only (b10, b1, b2). You can check b10_yes in the region file for a specific example of the difference. This is important because we're taking an agnostic approach to the data columns, and using the codebook to drive the existence and index position of the variables referenced. Since they aren't actually in the same order, this ends up getting out of sync after the first discrepancy.

Either FB will provide a new dataset version that preserves ordering in both tabs, or we will need to update the data generation script to account for the mismatches.