dw8547
commented
8 years ago @xriss, I've got 'https://github.com/devinit/digital-platform/blob/master/country-year/poorest20pct-percentages.csv' as not being used in the DH:
ddw_development=# SELECT
ddw_development-# *
ddw_development-# FROM public.di_concept_in_dh
ddw_development-# WHERE id = 'poorest20pct-percentages';
concept_id | series | id | in_dh
------------+--------------+--------------------------+-------
218 | country-year | poorest20pct-percentages | 0
(1 row)@akmiller01 helped me figure out which files were/are actually being used in the DH here: https://github.com/devinit/ddw-data/issues/124. As this one came out as in_dh = 0 I didn't import the table.
@timstrawson, can you help please? Do we leave 'poorest20pct-percentages.csv' out? Do I need to create a table it?
timstrawson
I can see that some of the data is available with slightly different table names, eg
fact.population_by_age_0_14 -> country-year/population-0-14
@xriss, yes. The tables that have names different from the file names in the GitHub repository are (not accounting for possible typos):
In 6 & 7 we've changed uppercase to lowercase only.
(I do a simple replace of _ - and then a string compare of the filename to try and auto find the matching csv table so these are not going to be picked up until I add them in explicitly.) @xriss, OK, got it. The above will need to be added then.
There is the oda and oda_donor which should be split and we should use the _2012 version (right?), @xriss, that's right:
however, it still looks like we are missing some data.
Take a look at all the other MISSING FROM WAREHOUSE lines below and please advise where that data should come from. @xriss, OK, having a look now.
@kriss, that's right, the above DW tables are not needed for your purposes.
@kriss, the above DW tables are needed, see table at the top with summary of name changes.
@kriss, that's right, the above DW tables are not needed for your purposes.
@kriss, that's right. My understanding is that these data series/.csv files are not even used in the DH so do not need to be replaced with DW data. @timstrawson, can you please confirm?
@kriss, these do need to be replaced with DW data. The data for these is in fact.oda_donor_2012. To get at the individual donor data, we need to filer on the di_id for the relevant years so:
@xriss, watch out for:
Delete:
Use:
instead.
@xriss, more to watch out for:
More info about this here: https://github.com/devinit/digital-platform/issues/243
@kriss, that's right. These do not need to be replaced with DW data.
@kriss, this one does need to be replaced with DW data, but is genuinely missing from DW. I will correct this & let you & @notshi know when I have.
@kriss, these do need to be replaced with DW data, see table at the top with summary of name changes.
@kriss, that's right. These do not need to be replaced with DW data.
@xriss, yes, that's right.
@xriss, yes, that's right.
@xriss, no, do (filtering on year BETWEEN 2006 AND 2014):
@xriss, yes, that's right.
@xriss, we need these too, see table at the top with summary of name changes.