Introduce world infrastructure data

[JaquM-HZI]

Situation Description

UPDATE 15.03.2021: Document listing Europe (continent) with sub-continents, countries and regions, with ISO-2 codes and hierarchies: https://docs.google.com/spreadsheets/d/1ejttFVpS9HEavP2kWM5lrHgLS1T4j38tSDaKQBbLBpI/edit?usp=sharing

---

For general infrastructure, the following fields should be added to the infrastructure data: Continent, Continental Subarea. Values for country and region (Bundesland) already exist and need to be extend with world regions.

Basically, all data should be stored, for all continents, continental subareas, countries and regions. The data can be found in the SurvNet meta dat SurvNet3MetaPublikationCVD_200219 (1).xlsx The values from ISO-3166 should be used (e.g.: ISL - Iceland) supplemented by ISO-3166-2 (regions) so that it defines the respective values and is stored under ISO code. The entries should not be editable but stored for all. I think this catalog can be used worldwide? @bernardsilenou @Candice-Louw what do you think?

The hierarchy should not be one-sided in the selection, but should work in both directions. It should be possible to select a country without first selecting the continent. When selecting the country, the paths of the continental sub-area and the continent should fill automatically.

Feature Description

• add values for continent, continental subarea, and region.
• fill these areas with the data stored in the catalog of the metadata sheet
• GERMANY ONLY: map these values bidirectional within Sormas to SurvNet https://github.com/hzi-braunschweig/sormas-survnet-interface/issues/59

Example:

Continent	Continental Subarea	Country	Region	ISO-Value
Europe	-	-	-	-
-	Central Europe	-	-	-
-	-	Swiss	-	CHE
-	-	-	Aargau	CH-AG
-	-	-	Appenzell-Ausserrhoden	CH-AR
-	-	-	Appenzell Innerhoden	CH-AI
-	-	-	Basel-Landschaft	CH-BL
-	-	-	Basel-Stadt	CH-BS
-	-	-	Bern	CH-BE

Possible Alternatives

Additional Information

There must be a solution for migrating or updating existing data, for example, facilities.

@kwa20 @Candice-Louw @bernardsilenou

[Candice-Louw]

Great idea for standardization. BFS list currently in use in CH included for reference: BFS Laenderliste.xlsx Also making use of ISO.

In order to get a multi-level list as you propose may not be possible with the current Vaadin user interface, though. Until the new UI is ready, the only option may be to have various dropdown menus - able to operate independently (full lists) as well as in combination with one another (filter one list, then the next will automatically shorten its entries etc.)...

For data migration - this may relate to building up the infrastructure for SORMAS2SORMAS.

@vidi42 - is there a feasible way to do data migration when introducing the new infrastructure to SORMAS as proposed, please?

[vidi42]

@Candice-Louw regarding data migration, since the Continents and Subcontinent areas are usually the same across all SORMAS instances (I assume), we could introduce the default Continents and Subcontinent areas option similarly to the approach we did for countries. https://github.com/hzi-braunschweig/SORMAS-Project/issues/2993#issuecomment-711834774 This way existing or new instances can be populated with the default list which is already prepared.

We also leave the options for systems to import their own list of Continents and Subcontinent areas if they want to.

@MateStrysewske please correct me if I'm wrong here.

[JaquM-HZI]

@vidi42 I would not make this data editable, this causes significant problems in the interfaces or Sormas2Sormas communications.

[Candice-Louw]

Thank you @vidi42 . @markusmann-vg Please could you see if it is possible to start with Europe and its sub-continents, countries, counties/regions as a starting point, please. Link to doc in the original post above. Please let us know if you require it as a stand-alone document?

[MateStrysewske]

Follow-up issues:

4775 => Continent and subcontinent entities

4776 => UI for continents and subcontinents

4777 => Extend location with continent and subcontinent

4782 => Import for continents and subcontinents

4784 => Add country to regions

[MateStrysewske]

@kwa20 @JaquM @Candice-Louw We need to make sure that we use a list of continents and subcontinents that is as commonly used internationally as possible. Using the list that SurvNet uses might not be sufficient. One possibility could be https://en.wikipedia.org/wiki/UN_M49, even though I don't know whether it has any application beyond statistical usage.

If there's no international standard we can adhere to, I believe we should go with the 7-continent model because that's most commonly used in the world (i.e. split North and South America).

[Candice-Louw]

@MateStrysewske - you're right, but there doesn't seem to be an international accepted standard.

So, SurvNet has adopted the 6 continent model (to some extent) which is still in use (https://en.wikipedia.org/wiki/Continent):

Regardless, if we deviate from the SurvNet World values in the Excel list, we would have mapping and converter problems as SurvNet allows selecting top-level entries as individual values i.e. "Amerika" alone can be selected and should be mapped according to its unique value to and from SORMAS:

If we don't have this 1-1 mapping with SurvNet, from the top levels down, it would defeat the purpose of this exercise. For example, how do we map "Amerika" [SurvNet] in the top level when we have "North America" and "South America" [SORMAS]? For import from SurvNet to SORMAS - which one do we then choose in SORMAS? Map to North? Map to South? Choose a random one? Cannot be done. So combine them, or introduce another level. More complexity. More time. More project delays. Back to where we are now.

In short: if we cannot match SurvNet exactly, we will have problems.

This will stand for the 195 countries of the world and their provinces as there is no other reference list that we can use to make the mappings in this exact form as has already been done by us to include ISO-2 codes for each province based on the SurvNet structure and standard, for example, for Europe: WorldMap_Europe_16032021.xlsx

We are trying to assist with making lists like the one above, based on the SurvNet structure, but if you are not comfortable with using the SurvNet structure, please escalate with @markusmann-vg and @nilshellrung to GK directly. Our instruction has been to map to SurvNet. If you do not agree, it would have to be escalated to the appropriate levels above us.

@JaquM @kwa20 @HolgerReiseVSys @maximni-vg @stefanspiska @tkaefer - please add anything that may have been missed from a mapping perspective or provide alternative input to this that hasn't been considered? I hope that I have missed something somewhere and that there is a workaround/solution, but I am currently not capable of finding it and see no alternative other than exact mapping.

If no responses are given to contradict this, exact mapping according to the SurvNet list will be the final decision.

[stefanspiska]

Well from my side a easy solution would be to fill the county list with

Amerika Amerika_Nordamerika Amerika_Nordamerika_Usa Amerika_Nordamerika_Usa_Texas

This would flatten the survnet valuset to a simple list which can be put into the country structure.

Such a flattened list could be generated from the valuset description in survnet.

To make this usable for the user the the country Feld would be needed to search in (and not from the start of the values )

So if you type USA you would go to Amerika_Nordamerika_Usa and if you type Texas you would go to Amerika_Nordamerika_Usa_Texas

[HolgerReiseVSys]

Well from my side a easy solution would be to fill the county list with

Amerika Amerika_Nordamerika Amerika_Nordamerika_Usa Amerika_Nordamerika_Usa_Texas

@stefanspiska For some fields only countries are permitted, e.g. for CountryofBirthMYT

[MateStrysewske]

@Candice-Louw If there's no "official" list of continents and subcontinents that is the status quo in terms of epidemiology (and I can't find one myself), I have nothing against using the list that's also used in SurvNet.

[stefanspiska]

@HolgerReiseVSys well then you could give each County (which then should be renamed) a type wich says what it is and filter in the select according to that.