search

DataUSA / datausa-tracker

0 stars 0 forks source link

re-ingest 2016 language tables (B16001) #254

Closed davelandry closed 4 years ago

davelandry commented 5 years ago

it's missing data at certain levels: https://flint-api.datausa.io/ui/#eyJkcmlsbERvd25zIjpbWyJHZW9ncmFwaHkiLCJDb3VudHkiLCJDb3VudHkiXV0sImN1dHMiOltdLCJjdWJlIjoiYWNzX3lnbF9sYW5ndWFnZV9zcG9rZW5fYXRfaG9tZV9ieV9lbmdsaXNoX2FiaWxpdHlfMjAxNl81IiwibWVhc3VyZXMiOlsiTGFuZ3VhZ2VzIFNwb2tlbiJdfQ==

hwchen commented 5 years ago

I'm going to leave this to @MarcioPorto for now; I think he's also doing this table for the shared acs.

@MarcioPorto feel free to ping me about ingesting stuff into datausa.

jspeis commented 5 years ago

@hwchen just for future context - would you be able to add some notes in terms of what the likely cause of the error is and the likely fix? (was there a download error?)

hwchen commented 5 years ago

ah, sorry. The data appears to have been updated between the time this data was ingested, and the latest download. So no error, as far as I can tell, just updated api.

On Wed, Jun 26, 2019 at 4:21 PM Jonathan Speiser notifications@github.com wrote:

@hwchen https://github.com/hwchen just for future context - would you be able to add some notes in terms of what the likely cause of the error is and the likely fix? (was there a download error?)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Datawheel/datausa-tracker/issues/254?email_source=notifications&email_token=AA43JC2VGLMRXFBZIDY3PQDP4PFWLA5CNFSM4HZCTQJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYUWQ4Y#issuecomment-506030195, or mute the thread https://github.com/notifications/unsubscribe-auth/AA43JC37KEYSYLZY6ZESKLDP4PFWLANCNFSM4HZCTQJQ .

hwchen commented 5 years ago

From dave: but the _2016 tables seem to be missing data at certain levels (tested with County and MSA): https://flint-api.datausa.io/ui/#eyJkcmlsbERvd25zIjpbWyJHZW9ncmFwaHkiLCJDb3VudHkiLCJDb3VudHkiXV0sImN1dHMiOltdLCJjdWJlIjoiYWNzX3lnbF9sYW5ndWFnZV9zcG9rZW5fYXRfaG9tZV9ieV9lbmdsaXNoX2FiaWxpdHlfMjAxNl81IiwibWVhc3VyZXMiOlsiTGFuZ3VhZ2VzIFNwb2tlbiJdfQ==

hwchen commented 5 years ago

It looks like I was mistaken, I must have looked at 1yr instead of 5yr. So there is no data for 5yr for msa and county. I even reingested just to make sure.

@davelandry how would you like to proceed? Do you need annotations?

davelandry commented 5 years ago

Curious if @jspeis agrees, but I feel the solution would be to completely remove the 5yr table (or at the very least add a hide_in_ui annotation). This is causing errors in the vizbuilder, which doesn't use the logic layer, which means that encoding something like "force 1yr if MSA or County" is not trivial.

jspeis commented 4 years ago

@davelandry @innerstage I have a feeling this is the same issue that was recently noticed with regard to Cambridge as @innerstage mentioned there was no data in the source.