DWP-Family-Resources-Survey-Disability

[LPerryman]

• [x] Create Jenkins job (in Jenkins).
• [x] Create a Jenkinsfile.
• [x] Figure out landing page.
• [x] Create main.py and check scraper works.
• [ ] Determine eventual Tidy data column names and add to info.json:
  • [x] Period column should use government-year/{y1}-{y2} #29
• [ ] Map column names to codelists (existing and newly required), adding mapping to info.json.
• [x] Add definition to reference/columns.csv
• [x] Add newly required codelists (under reference/codelists) and add info to reference/codelists-metadata.json
• [x] Add component definitions to reference/components.csv
• [x] Write a transformation to Tidy CSV and output metadata dataset.trig and schema.
  • [x] Draft
  • [ ] Finalised

[grace-spitzer-wong]

linked to https://github.com/GSS-Cogs/family-disability/issues/30

[ajtucker]

There are some duplicates in the Tidy CSV, see https://ci.floop.org.uk/job/GSS_data/job/Disability/job/DWP-Family-Resources-Survey-Disability/22/execution/node/58/log/

These are duplicates in that multiple rows have the same set of dimensions but different values. Can you check where the numbers are coming from and figure out what the distinction is between the observations and whether there's a missing dimension?

[LPerryman]

This is Js pipeline, have you sent this to him as well? i'm happy to have a look, just wondering if you meant to send it to him instead :-)

On Mon, 9 Dec 2019 at 08:52, Alex Tucker notifications@github.com wrote:

There are some duplicates in the Tidy CSV, see https://ci.floop.org.uk/job/GSS_data/job/Disability/job/DWP-Family-Resources-Survey-Disability/22/execution/node/58/log/

These are duplicates in that multiple rows have the same set of dimensions but different values. Can you check where the numbers are coming from and figure out what the distinction is between the observations and whether there's a missing dimension?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/GSS-Cogs/family-disability/issues/11?email_source=notifications&email_token=AKZPUJSJLGX6KO3FSNHK3ULQXYBOVA5CNFSM4JM5HPN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGIK2IA#issuecomment-563129632, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKZPUJVDVBVYM7Y24FKLNWLQXYBOVANCNFSM4JM5HPNQ .