Open ajtucker opened 4 years ago
Having massive issues with transforming as data is huge especially the first file JTS0501: Travel time, destination and origin indicators for Employment centres by mode of travel, Lower Super Output Area (LSOA), England (ODS, 67.3MB)
Have published journey times from first dataset: JTS0501: Travel time, destination and origin indicators for Employment centres by mode of travel, Lower Super Output Area (LSOA), England (ODS, 67.3MB) 295596 observations
All 9 Journey time datasets have been published on PMD4. note only the journey time has been published other information has been left out at this time. Can only publish one measure type at present.
Journey times to key services by lower super output area: Employment Centres - JTS0501 https://staging.gss-data.org.uk/cube/explore?uri=http%3A%2F%2Fgss-data.org.uk%2Fdata%2Fgss_data%2Ftowns-high-streets%2Fdft-journey-times-to-key-services-by-lower-super-output-area%2Femployment-centres-catalog-entry
Journey times to key services by lower super output area: Primary Schools - JTS0502 https://staging.gss-data.org.uk/cube/explore?uri=http%3A%2F%2Fgss-data.org.uk%2Fdata%2Fgss_data%2Ftowns-high-streets%2Fdft-journey-times-to-key-services-by-lower-super-output-area%2Fprimary-schools-catalog-entry
Journey times to key services by lower super output area: Secondary Schools - JTS0503 https://staging.gss-data.org.uk/cube/explore?uri=http%3A%2F%2Fgss-data.org.uk%2Fdata%2Fgss_data%2Ftowns-high-streets%2Fdft-journey-times-to-key-services-by-lower-super-output-area%2Fsecondary-schools-catalog-entry
Journey times to key services by lower super output area: Further Education - JTS0504 https://staging.gss-data.org.uk/cube/explore?uri=http%3A%2F%2Fgss-data.org.uk%2Fdata%2Fgss_data%2Ftowns-high-streets%2Fdft-journey-times-to-key-services-by-lower-super-output-area%2Ffurther-education-catalog-entry
Journey times to key services by lower super output area: General Practices - JTS0505 https://staging.gss-data.org.uk/cube/explore?uri=http%3A%2F%2Fgss-data.org.uk%2Fdata%2Fgss_data%2Ftowns-high-streets%2Fdft-journey-times-to-key-services-by-lower-super-output-area%2Fgeneral-practices-catalog-entry
Journey times to key services by lower super output area: Hospitals - JTS0506 https://staging.gss-data.org.uk/cube/explore?uri=http%3A%2F%2Fgss-data.org.uk%2Fdata%2Fgss_data%2Ftowns-high-streets%2Fdft-journey-times-to-key-services-by-lower-super-output-area%2Fhospitals-catalog-entry
Journey times to key services by lower super output area: Food Stores - JTS0507 https://staging.gss-data.org.uk/cube/explore?uri=http%3A%2F%2Fgss-data.org.uk%2Fdata%2Fgss_data%2Ftowns-high-streets%2Fdft-journey-times-to-key-services-by-lower-super-output-area%2Ffood-stores-catalog-entry
Journey times to key services by lower super output area: Town Centre - JTS0508 https://staging.gss-data.org.uk/cube/explore?uri=http%3A%2F%2Fgss-data.org.uk%2Fdata%2Fgss_data%2Ftowns-high-streets%2Fdft-journey-times-to-key-services-by-lower-super-output-area%2Ftown-centre-catalog-entry
Journey times to key services by lower super output area: Pharmacies - JTS0509 https://staging.gss-data.org.uk/cube/explore?uri=http%3A%2F%2Fgss-data.org.uk%2Fdata%2Fgss_data%2Ftowns-high-streets%2Fdft-journey-times-to-key-services-by-lower-super-output-area%2Fpharmacies-catalog-entry
Due to the amount of data being transformed and time issues the source data has been processed offline and output to google docs. The python script processes these files instead of from source. Will cause issues with new releases but DEs are experimenting with code to speed thing up.
Employment Centres https://drive.google.com/file/d/1SeekTbw2ShjSws_I5G5bTG8va0hhJ5wg/view?usp=download Primary Schools https://drive.google.com/file/d/1cPkyBV-YdqaA6z15ew-M_qEW1pL5yDNS/view?usp=download Secondary Schools https://drive.google.com/file/d/1f9TshV-To_t913j6lfMIajmFLpK_X86Q/view?usp=download Further Education https://drive.google.com/file/d/1Y0nMpj9b8rftTT4I7h3IGIvGUzkin4zr/view?usp=download General Practices https://drive.google.com/file/d/15KNDFI7WYcEmfj-6yTAO8LrzTZlC7NUE/view?usp=download Hospitals https://drive.google.com/file/d/1sl301ieObzhPiHAIRRVyETH6ucyvjOBC/view?usp=download Food Stores https://drive.google.com/file/d/1hwAQuZAazD6xEQcG9OdKiOWIpL7RkRjQ/view?usp=download Town Centre https://drive.google.com/file/d/1xmoeawBJ2rB24Nn1GpwvjaOej2lvC6XA/view?usp=download Pharmacies https://drive.google.com/file/d/1lcFBa3KUsaBd4P0wJdobD5uCjxcXbq4I/view?usp=download
BA comments:
The dataset has been split into 9 different datasets on PMDv4 due to the size of the data and only currently contains travel times due to issues with multiple measures.
Stage 1 & 2 tech spec combined
There are no active filters.
Incorrect contents issue date of 11/12/2018 has been used the latest published date is 19/12/2019.
Is Field Code the correct name for a dimension?
Travel time to 15+ decimal points?
'Employment Centre Size' dimension has been added to the 'Employment Centre' dataset and 'Field Code' dimension renamed 'Mode of Travel' and codelist altered to ('PT/Walk', 'Cycle', 'Car'). All 9 datasets have been republished.
Hi.
It doesn't look like the pickers are working for any of these datasets: JTS0501 for example.
Indeed it looks like the dimension properties haven't been defined (or at least any that are defined aren't using the same URIs as those in the DSD).
Also seems to be a typo with the employment-centre-size codelist:
?code skos:inScheme <http://gss-data.org.uk/def/concept-scheme/lemployment-centre-size>
Ought to be:
?code skos:inScheme <http://gss-data.org.uk/def/concept-scheme/employment-centre-size>
The slug looks right in code and graph URIs. I wonder if it's possible to DRY-up the generation process to avoid copy-paste mistakes?
4 ODS files. 1 Sheet per file but still a table that needs to be transformed