GSBPM "Analyse" Phase

[InKyungChoi]

General comment on this phase

• It is unclear when the statistical product is micro-data, what part of the validation and control fall under phase 5 or phase 6 ? Phase 6 (France)
• Time series-related statistical activities (such as time series preparation, seasonal adjustment, temporal disaggregation, etc.) are currently not visible in Phase 6. Analyze. We suggest to consider adding a sub-process for these statistical activities (Hungary)
• We suggest it should be revised to accommodate new types of outputs: it treats outputs more-or-less the traditional way (data sets). This is no longer true: visualized outputs get more and more ground. (Hungary)
• In this phase, it should be directly indicated that in first place must be carried out an Exploratory Data Analysis (EDA), because this is a well-known process for data analysis. For this reason, it is important that in the document the EDA process is referred with its name as it is commonly used. (Ecuador)

"Prepare draft outputs" sub-process

• 6.1 : Aggregates from 5.7 can be statistical outputs. May be add the precision because the rest of phase 6 only use the word statisticals outputs which is defined in sub-procces 6.1. In that case of aggregates being the statistical outputs the sub-process is still relevant but limited to recording the quality criteria ? If 6.1 should be skipped because aggregates being the statistical outputs where should this precision be added ? (France)

"Validate outputs" sub-process

• 6.2 Validate outputs: renaming suggestion to macrovalidation (Hungary)

"Interpret and explain outputs" sub-process

• Sub-process 6.2 & 6.3: Regarding 6.2 and 6.3, responsibility of NSO explaining ML-based results was mentioned during meetings of the GSBPM "Task" task team and the GeoGSBPM task team (GSBPM task team):

  • "Results produced by machine learning techniques have nature quite different from traditional methods, efforts would be needed to explain the results internally and externally" (GeoGSBPM meeting notes from December 16, 2020)
  • "Participants mentioned that sometimes validation means that if a new method/technique is used to produce statistics, then the NSO also needs to explain this to the users, basically needs to be more transparent. This can be considered also as some kind of validation. Example: NSO starts to produce statistics using ML while previously it produced the very same statistics using "traditional" processes. In this case, the use of ML needs to be explained and the NSI needs to validate the use of a new technique. This is especially important when users are not familiar with new techniques or even skeptical about some of them: ML could be an example for this case." (proposal from MWW2022 participant on the GSBPM "Task" task team work)

• Regarding 6.2 and 6.3 again, I think the scope of work between these two sub-processes is unclear; groups working on GSBPM always needed to discuss this to clarify e.g., (GSBPM task team)

  • "For sub-processes 6.2 and 6.3, members discussed the difference between validation and interpretation of outputs. It was mentioned that validation is about checking the requirement with rules/standards, whereas interpretation has to take into account other information and put the results in context." (GSBPM task team meeting notes from 14 March 2022)

  • "While 6.2 is more about intrinsic validation check (e.g., by statistician), this sub-process 6.3 is using domain knowledge and external sources (e.g., by economists). " (Linking GSBPM-GSIM task team meeting note from October 26, 2021)

"Apply disclosure control" sub-process

• We suggest complementing the following sentence in the point 93: “The degree and method statistical disclosure control may vary for different types of outputs in which finalised outputs from geospatial data may raise new confidentiality challenges in terms of personal data protection and disclosure risk”. (Portugal)
• (this is a comment on phase 5, but might be relevant) Pseudonymisation (or anonymisation) is not mentioned in the GSBPM. In Finland, the Data Management Act (based on GDBR) requires that personal data must be pseudonymised. Statistics Finland interprets pseudonymisation as part of the sub-process 5.1. Pseudonymisation requires units to be combined with the corresponding units of the other datasets (and base register), so that multiple pseudo-identifiers are not created for the same unit and so that the unit's data from different sources is properly integrated. (Finland)
• GSBPM does not mention ethical considerations here, even though there are. The GSBPM revision team should analyse how the ethical considerations are affecting the statistical work and how those perspectives can be built into the model. (GSBPM task team)

[InKyungChoi]

Action points from the Meeting of September 18th

Action: InKyung to check what was the original rationale for including the term “accessibility” in version 5.1 of GSBPM.

=> It seems it has nothing to do with disability.... it came from GEOSTAT2 (see sub-process 6.1 in Annex 4 of the GEOSTAT 2) project and stayed somehow. If people are not happy with "accessbility measure", we can remove.

Action: Giorgia to develop a sentence or two about methodology for machine learning methods

=> Giorgia made updates on google doc

Action: InKyung offered to revisit 5.1 in Geo GSBPM to see if there is anything that should be mentioned in 5.1 for the next version of GSBPM.

=> GeoGSBPM has quite some addition under this sub-process 5.1 Integrate data. This part seems useful to add including the footnote:

Geospatial information (e.g. address, x- and y-coordinate, or a geographical name) can play an important role in bringing together information from various domains by enabling integration of datasets from different sources using the location information as a matching key variable (e.g. integrating administrative data with survey data using address or postal code that exists in both datasets). To ensure the quality of the integration, standardising the geospatial information in the different datasets is critical. This standardisation would normally take place before the integration of datasets and can be done through, for example, matching location information in the datasets with a centralised standard system (e.g. address matching, geocoding) (footnote) which should be part of the national spatial data infrastructure.

footnote: The address matching and geocoding can be seen as data integration on their own (e.g. matching address in a dataset with valid address in an address register or geocode database). However, in this paper, the term “data integration” is used for the datasets that are of primary interest for the current business process and the address matching or geocoding are considered as a part of the standardisation or coding activity for this integration to take place. Note also that address matching and geocoding can be modelled as a process that uses a chain of sub-processes in Phase 5 (e.g. matching and linking address in survey dataset with geocoded address (sub-process 5.1), assigning the corresponding geocode (sub-process 5.2) while iteratively reviewing the address (sub-process 5.3) and editing / imputing (sub-process 5.4) where necessary