search

NYCPlanning / db-factfinder

data ETL for population fact finder (decennial + acs)
https://nycplanning.github.io/db-factfinder/factfinder/
MIT License
2 stars 3 forks source link

Questions: remaining for 2010/2019 #212

Closed SPTKL closed 3 years ago

SPTKL commented 3 years ago
  1. For medians, if cumulative frequency = 0, do we set the estimate to NULL or 0?
  2. if P = 100, do we set Z to null or 0?
  3. missing variables, based on metadata comparison > these variables were replaced by different variables/variable codes
SPTKL commented 3 years ago

When comparing to metadata from previous years, we are missing the following variables:

missing variables:

variablename release_year profile category base
mhnw 2006-2010, 2013-2017 Social Household Type hh1
mhnwchu18 2006-2010, 2013-2017 Social Household Type hh1
fhnh 2006-2010, 2013-2017 Social Household Type hh1
fhnhchu18 2006-2010, 2013-2017 Social Household Type hh1
mv80t89 2013-2017 Housing Year Householder Moved Into Unit ochu3
mvbf79 2013-2017 Housing Year Householder Moved Into Unit ochu3

language spoken at home variables are all missing:

this is understandable

variablename|release_year|profile|category|base ----|-----|-----|----|---- lgoenlep1|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl1 lgfrcr1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgfrcr1 lgfrclep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgfrcr1 lgit1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgit1 lgitlep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgit1 lgprt1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgprt1 lgprtlep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgprt1 lgyid1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgyid1 lgyidlep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgyid1 lgowg1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgowg1 lgowglep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgowg1 lgscd1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgscd1 lgscdlep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgscd1 lggrk1|2006-2010, 2013-2017|Social|Language Spoken at Home|lggrk1 lggrklep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lggrk1 lgrus1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgrus1 lgruslep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgrus1 lgpol1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgpol1 lgpollep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgpol1 lgscr1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgscr1 lgscrlep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgscr1 lgoslv1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoslv1 lgosvlep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoslv1 lgarm1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgarm1 lgarmlep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgarm1 lgprs1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgprs1 lgprslep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgprs1 lgguj1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgguj1 lggujlep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgguj1 lghdi1|2006-2010, 2013-2017|Social|Language Spoken at Home|lghdi1 lghdilep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lghdi1 lgurd1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgurd1 lgurdlep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgurd1 lgoind1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoind1 lgoinlep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoind1 lgjap1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgjap1 lgjaplep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgjap1 lgmkhm1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgmkhm1 lgmkmlep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgmkhm1 lghmg1|2006-2010, 2013-2017|Social|Language Spoken at Home|lghmg1 lghmglep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lghmg1 lgthai1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgthai1 lgthalep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgthai1 lglao1|2006-2010, 2013-2017|Social|Language Spoken at Home|lglao1 lglaolep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lglao1 lgopi1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgopi1 lgopilep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgopi1 lgnav1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgnav1 lgnavlep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgnav1 lgonna1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgonna1 lgonalep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgonna1 lghung1|2006-2010, 2013-2017|Social|Language Spoken at Home|lghung1 lghnglep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lghung1 lgheb1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgheb1 lgheblep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgheb1 lgafr1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgafr1 lgafrlep1|2006-2010, 2013-2017|Social|Language Spoken at Home|lgafr1 lgfrcr2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgit2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgprt2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgyid2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgowg2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgscd2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lggrk2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgrus2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgpol2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgscr2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgoslv2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgarm2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgprs2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgguj2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lghdi2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgurd2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgoind2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgjap2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgmkhm2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lghmg2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgthai2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lglao2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgopi2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgnav2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgonna2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lghung2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgheb2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgafr2|2006-2010, 2013-2017|Social|Language Spoken at Home|pop5pl2 lgfrclep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgitlep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgprtlep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgyidlep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgowglep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgscdlep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lggrklep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgruslep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgpollep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgscrlep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgosvlep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgarmlep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgprslep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lggujlep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lghdilep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgurdlep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgoinlep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgjaplep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgmkmlep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lghmglep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgthalep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lglaolep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgopilep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgnavlep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgonalep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lghnglep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgheblep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2 lgafrlep2|2006-2010, 2013-2017|Social|Language Spoken at Home|lgoenlep2
EricaMaurer commented 3 years ago

If P=100, Z= 0

The missing variables were all replaced by different variables/variable codes

SPTKL commented 3 years ago

New Variables:

variablename release_year profile category base
pu18bwpv_pct Y2015-2019 Economic nan
pbwpv_pct Y2015-2019 Economic nan
p65plbwpv_pct Y2015-2019 Economic nan
mvbf89 Y2015-2019 Housing ochu3
mhns Y2006-2010, Y2015-2019 Social hh1
mhnschu18 Y2006-2010, Y2015-2019 Social hh1
lgruspol2 Y2015-2019 Social pop5pl2
lgruspol1 Y2015-2019 Social lgruspol1
lgrsplep2 Y2015-2019 Social lgoenlep2
lgrsplep1 Y2015-2019 Social lgruspol1
fhns Y2006-2010, Y2015-2019 Social hh1
fhnschu18 Y2006-2010, Y2015-2019 Social hh1
cvnid_vsnp Y2015-2019 Social cvnipop1
cvnid_scrp Y2015-2019 Social cvnipop1
cvnid_ildp Y2015-2019 Social cvnipop1
cvnid_hrgp Y2015-2019 Social cvnipop1
cvnid_cogp Y2015-2019 Social cvnipop1
cvnid_ambp Y2015-2019 Social cvnipop1
cu18dvsnp Y2015-2019 Social cvniu18_1
cu18dscrp Y2015-2019 Social cvniu18_1
cu18dhrgp Y2015-2019 Social cvniu18_1
cu18dcogp Y2015-2019 Social cvniu18_1
cu18dambp Y2015-2019 Social cvniu18_1
c1864dvsnp Y2015-2019 Social cni1864_1
c1864dscrp Y2015-2019 Social cni1864_1
c1864dildp Y2015-2019 Social cni1864_1
c1864dhrgp Y2015-2019 Social cni1864_1
c1864dcogp Y2015-2019 Social cni1864_1
c1864dambp Y2015-2019 Social cni1864_1
c65pldvsnp Y2015-2019 Social cvni65pl
c65pldscrp Y2015-2019 Social cvni65pl
c65pldildp Y2015-2019 Social cvni65pl
c65pldhrgp Y2015-2019 Social cvni65pl
c65pldcogp Y2015-2019 Social cvni65pl
c65pldambp Y2015-2019 Social cvni65pl
EricaMaurer commented 3 years ago

The variables with a base of cvnipop1, cvniu18_1, and cvni65pl are not new, the P was just pulled from the data this time instead of calculated (in this case for boroughs and NYC). No change needed on the front-end.

mvbf89 is not a new variable. Already displayed in the front-end as 'Moved in 1989 or earlier'

The remaining variables from the social profile that changed were submitted in a user story with the new front-end wording to be displayed.

SPTKL commented 3 years ago

@EricaMaurer are we supposed to use these p variables? e.g. the p value for c65pldamb should come out of c65pldambp etc for borough and city level? Data Engineering didn't know about this

EricaMaurer commented 3 years ago

You can ignore this