agiorguk / gemini

Resources relating to the UK Gemini metadata profile
5 stars 3 forks source link

align GEMINI Data quality element with the new UK Government Data Quality Framework #63

Open playarragorn opened 3 years ago

playarragorn commented 3 years ago

UK Government have just issued a Framework for Data Quality and we need to incorporate this into UK Gemini *and other metadata standards)

PeterParslow commented 3 years ago

The UK Government Data Quality Framework is based on the DAMA approach. You can find a brief description of how they expect data quality to be communicated at https://www.gov.uk/government/publications/the-government-data-quality-framework/the-government-data-quality-framework-guidance#communicating-data-quality-to-users.

The relevant bit seems to be that they consider data quality under six "dimensions":

Plus they recommend giving the "background to the data" - describing its lifecycle, how it was collected & by whom.

We could probably provide a general section on 'how to use GEMINI to communicate an assessment of the data against the DAMA/UK Government data quality dimensions". Something like:

And to provide information on the background of the data, use Lineage

PeterParslow commented 3 years ago

I missed the government's "consistency" dimension; I'm not sure there is an equivalent in ISO 19157, although the example would fit in with ISO 19157/GEMINI's "Quantitative attribute correctness".

Also, the government description of their "timeliness" dimension differs between web pages:

https://www.gov.uk/government/publications/the-government-data-quality-framework/the-government-data-quality-framework says "Timeliness describes the degree to which the data is an accurate reflection of the period that they represent, and that the data and its values are up to date.

Some data, such as date of birth, may stay the same whereas some, such as income, may not.

Data is timely if the time lag between collection and availability is appropriate for the intended use."

The first sentence of that seems to be a part of Domain consistency!

https://www.gov.uk/government/publications/the-government-data-quality-framework/the-government-data-quality-framework-guidance#communicating-data-quality-to-users says "Timeliness depends on the intended use of the data. Telling users when the data was collected will inform them of the period the data reflects, which will help them decide if the data is appropriate for their needs. Timeliness is sometimes in a trade off with accuracy or the amount of data available to users. Timely data may have been collected and processed more quickly, limiting the amount of data collected and the quality assurance done. You can communicate this trade-off when informing users about data quality and provide any justifications for it." - which lead me to the Temporal extent end being something that the user can use to determine whether the data is 'timely' for their use..

PeterParslow commented 3 years ago

Here's a more detailed mapping done by a colleague in OS; it isn't an "official" OS position (and I think she missed a bit), but we hope it might be useful. Data Quality Dimensions Mappings.xlsx

playarragorn commented 3 years ago

Peter,

Can you explain the term in the spreadsheet:

DAMA DM BOK

Regards

Alexander D. Ramage CITP MBCS Head of Management Information Systems Finance and Corporate Services

T: 0141 272 7281 Blackberry: 07920 477 576 Teams: @.*** transport.gov.scothttp://transport.gov.scot/ Please see our privacy policyhttps://www.transport.gov.scot/privacy-policy/ to find out why we collect personal information and how we use it

Transport Scotland, Buchanan House, 58 Port Dundas Road, Glasgow, G4 0HF

@.**@*.**@*.**@*.**@*.***D2716A.B4AD6420]https://www.instagram.com/transportscotland/

Transport Scotland, the national transport agency Còmhdhail Alba, buidheann nàiseanta na còmhdhail

From: Peter Parslow @.> Sent: 12 July 2021 11:34 To: agiorguk/gemini @.> Cc: Ramage ADR (Alex) @.>; Author @.> Subject: Re: [agiorguk/gemini] align GEMINI Data quality element with the new UK Government Data Quality Framework (#63)

Here's a more detailed mapping done by a colleague in OS; it isn't an "official" OS position (and I think she missed a bit), but we hope it might be useful. Data Quality Dimensions Mappings.xlsxhttps://github.com/agiorguk/gemini/files/6800654/Data.Quality.Dimensions.Mappings.xlsx

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/agiorguk/gemini/issues/63#issuecomment-878166509, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJXIZNIIPGQ74WZOKL3UFT3TXLAIHANCNFSM47WRSDBQ.


This e-mail (and any files or other attachments transmitted with it) is intended solely for the attention of the addressee(s). Unauthorised use, disclosure, storage, copying or distribution of any part of this e-mail is not permitted. If you are not the intended recipient please destroy the email, remove any copies from your system and inform the sender immediately by return. Communications with the Scottish Government may be monitored or recorded in order to secure the effective operation of the system and for other lawful purposes. The views or opinions contained within this e-mail may not necessarily reflect those of the Scottish Government.


PeterParslow commented 3 years ago

DAMA is the Data Management Association:

They own/manage/publish (at a price) the DMBOK, their "Data Management Body of Knowledge": https://www.dama.org/cpages/body-of-knowledge

As I mention above / a week ago, the UK Government Data Quality Framework is based on that (although I notice they don't cite it) - you can see from the spreadsheet how similarly the DAMA & "UK Government" "dimensions" are, and the "principles" are at least as close.