RFC: Power User configurable elements for InaSAFE 4.0

[timlinux]

Problem

This RFC describes elements that should be user configurable within the architecture of InaSAFE 4.0 and what the benefit and role of these components will be:

Current status: In progress

Value Maps (exposure classifications)

Purpose: Value maps provide a way to map user's data to known concepts in InaSAFE. For example in road exposure, we will report on a predefined list of classes of road. The default classes InaSAFE provides for roads are:

• Motorway
• Primary
• Secondary
• Local
• Path
• Other

The creation of a new value map definition would allow power users to create a new set of reporting classes (e.g. Motorway, Main, Secondary) which match the reporting requirements of the user's organization. A proposed user interface for editing value maps would look something like this:

Note: The above is a mockup and we are not planning to implement GUI's for managing these right now. Rather the user should edit the text definition as shown in the example below:

Generated code example:

generic_road_classes = {
    'key': 'generic_road_classes',
    'name': tr('Generic Road Classification'),
    'description': tr(
        'Classification of roads based on OSM.'
    ),
    'citations': [
        {
            'text': None,
            'link': None
        }
    ],
    'classes': [
        {
            'key': 'motorway',
            'name': tr('Motorway'),
            'description': tr('A road to solve the traffic and have a fare.'),
            'osm_downloader': [
                'Motorway link',
                'Motorway or highway',
            ],
            'string_defaults': [
                'motorway link',
                'motorway',
                'trunk link',
                'trunk',
            ],
            'citations': [
                {
                    'text': None,
                    'link': None
                }
            ]
        },
        {
            'key': 'primary',
            'name': tr('Primary'),
            'description': tr(
                'A road that service the main transportation with a long '
                'distance travel characteristic and high average velocity.'),
            'string_defaults': [
                'primary link',
                'primary road',
                'primary',
            ],
            'osm_downloader': [
                'Primary link',
                'Primary road',
            ],
            'citations': [
                {
                    'text': None,
                    'link': None
                }
            ]
        },
        {
            'key': 'secondary',
            'name': tr('Secondary'),
            'description': tr(
                'A road that service the transportation with a medium '
                'distance travel characteristic and medium average velocity.'),
            'string_defaults': [
                'secondary link',
                'secondary',
            ],
            'osm_downloader': [
                'Secondary link',
                'Secondary',
            ],
            'citations': [
                {
                    'text': None,
                    'link': None
                }
            ]
        },
        {
            'key': 'local',
            'name': tr('Local'),
            'description': tr(
                'A road that service the transportation with short distance '
                'travel and low average velocity.'),
            'string_defaults': [
                'local',
                'tertiary link',
                'tertiary',
                'tertiary',
                'unclassified',
            ],
            'osm_downloader': [
                'Tertiary link',
                'Tertiary',
            ],
            'citations': [
                {
                    'text': None,
                    'link': None
                }
            ]
        },
        {
            'key': 'path',
            'name': tr('Path'),
            'description': tr('A road to walk on foot aim.'),
            'osm_downloader': [
                'Cycleway, footpath, etc.',
                'Track',
            ],
            'string_defaults': [
                'cycleway',
                'footway',
                'path',
                'track',
            ],
            'citations': [
                {
                    'text': None,
                    'link': None
                }
            ]
        },
        {
            'key': 'other',
            'name': tr('Other'),
            'description': tr(
                'A road that service the transportation with short travel '
                'and low average velocity.'),
            'string_defaults': [
                'living street',
                'other',
                'pedestrian',
                'residential',
                'road',
                'service',
            ],
            'osm_downloader': [
                'Road, residential, living street, etc.'
            ],
            'citations': [
                {
                    'text': None,
                    'link': None
                }
            ]
        }
    ]
}

Outcome: Implementing a new exposure value mapping classification will make it possible for the user to select that classification in their InaSAFE settings dialog as part of their profile:

When a value mapping classification is selected in the user profile, the keywords wizard will be affected. The value mapping screen in the keywords wizard will reflect the currently selected value maps in the user profile:

The concepts in the tree to the right are taken from the value map.

Classifications (hazard classifications)

Purpose: Hazard classifications specify how a particular hazard type is deconstructed into different severity levels. For example for a flood hazard we might have these hazard classes:

• High
• Medium
• Low
• Dry

The creation of a new hazard classifications would support the development of region specific classifications for the same hazard. For example a particular country may prefer to work with four hazard classes for flood rather then three. A proposed user interface for editing hazard classes maps would look something like this:

The min/max for each class are used when the hazard data is continuous to define thresholds for each class break.

An example specification for flood classes (as a python dictionary) is shown below:

flood_hazard_classes = {
    'key': 'flood_hazard_classes',
    'name': tr('Flood classes'),
    'description': tr(
        'This is a binary description for an area. The area is either '
        '<b>wet</b> (affected by flood water) or <b>dry</b> (not affected '
        'by flood water). This unit does not describe how <b>wet</b> or '
        '<b>dry</b> an area is.'),
    'citations': [
        {
            'text': None,
            'link': None
        }
    ],
    'classes': [
        {
            'key': 'wet',
            'name': tr('wet'),
            'affected': True,
            'description': tr('Water above ground height.'),
            'string_defaults': ['wet', '1', 'YES', 'y', 'yes'],
            'numeric_default_min': 1,
            'numeric_default_max': 9999999999,
            'citations': [
                {
                    'text': None,
                    'link': None
                }
            ]
        },
        {
            'key': 'dry',
            'name': tr('dry'),
            'affected': False,
            'description': tr('No water above ground height.'),
            'string_defaults': ['dry', '0', 'No', 'n', 'no'],
            'numeric_default_min': 0,
            'numeric_default_max': (1 - small_number),
            'citations': [
                {
                    'text': None,
                    'link': None
                }
            ]
        }
    ]
}

Outcome:

• During processing of continuous hazard data, the values within each class break will be remapped to produce a classified layer based on the defined classes.
• During processing of categorical hazard data, the concepts for the classes can be mapped to the user's data. The defaults list will be used to automatically map user data to InaSAFE data if the concepts can be found.
• Implementing a new hazard class mapping classification will make it possible for the user to select that classification in their InaSAFE settings dialog as part of their profile:

When creating keywords for a hazard layer, the active classification profile for the user will be used.

Fields

Purpose: Fields allow you to establish which columns in the hazard, exposure or hazard dataset map to specific concepts. For example you you have an aggregation layer, defining which field contains the adult ratio will allow InaSAFE to scan through the concepts you have defined and map them to well known concepts within InaSAFE. With this mechanism we would have a semantic match between your field (perhaps called 'ratio_of_adults' in your own dataset) to the concept 'adult_ratio' within InaSAFE. Once this semantic match has been made, we can determine which post-processors (see below) can be used.

Notes:

• If the user defines a field with the same key value in their local working directory (e.g. ~/.qgis2/inasafe), that field definition will mask the one provided by InaSAFE.
• These fields are used when defining layer keywords. The fields are filtered by context (e.g. hazard, exposure or aggregation specific fields) to determine if they are applicable to a layer. If applicable, the user has the option of identifying all the applicable fields during execution of the keywords wizard.

Note: The above mockup is missing a field category column - see below.

adult_ratio_field = {
    'key': 'adult_ratio_field',
    'name': tr('Adult Ratio'),
    'field_name': 'adult_ratio',
    'type': QVariant.Double,
    'length': default_field_length,
    'precision': default_field_precision,
    'absolute': False,
    'description': tr('Attribute where the ratio of adult people is located.'),
    'citations': [
        {
            'text': None,
            'link': None
        }
    ]
}

Note that there are four types of fields that you can define:

1. exposure fields
2. hazard fields
3. aggregation fields
4. impact fields

   Post processors

Post processors are the core of InaSAFE. Post processors operate at an exposure feature level. After preparing the intermediate impact layer, InaSAFE will check the keywords of this layer to see which fields are defined, as well as checking the fields of the aggregate hazard layer. The cross product of these layers will determine which post processors can be run.

Note: In versions of InaSAFE prior to 4.0 the list of post processors could be user selected at run time just before running an impact function. In InaSAFE 4.0 we changed this behaviour so that post processors alway run if they have sufficient fields available.

The post processor results are all written to the final impact layer into fields that are again defined in the fields definitions. Thus a post processor is defined as one or more intermediate impact fields that are processed to produce one or more final impact fields.

Another important thing to note about post processors: Post processors are atomic. This means that no post processor may require the outputs of another post processor as an input requirement.

Note: The order in which post-processors is processed is also non-deterministic.

Below you can see an example of how we might support the creation of new post-processors by power users through a simple user interface:

For ease of understanding, we can divide the post processors below into a number of different subgroups. These are described below:

Vulnerabilities

Vulnerability post processors define specific calculations that relate to the vulnerability characteristics of exposure data. For example people can be more vulnerable if they are disabled, aged, visually impaired, pregnant etc. Roads may be vulnerable if they are at ground level for floods, yet the same road may be less vulnerable in the event of an earthquake. Conversely an overpass or elevated road may be less vulnerable in floods but more so in the case of an earthquake.

Note: As of 4.0 not all of these nuanced rules are implemented, but they are on the roadmap for future versions of InaSAFE.

Demographics

Demographic post processors seek to break down the impact report by population groups. For example they might produce new fields for counts of children, adults, elderly people. These numbers can be produced when an exposure layer provides a population column (e.g. building exposure with a count of inhabitants, or a census layer with a count of inhabitants per census area). There are many ways that population based counts can be derived. Here is an example of a post processor that performs a gender analysis:

post_processor_gender = {
    'key': 'post_processor_gender',
    'name': tr('Gender Post Processor'),
    'description': tr(
        'Post processor to calculate the number of affected woman'),
    'input': {
        'population': {
            'value': population_count_field,
            'type': 'field'
        },
        'gender_ratio': {
            'value': female_ratio_field,
            'type': 'field'
        }
    },
    'output': {
        'women': {
            'value': women_count_field,
            'formula': 'population * gender_ratio'
        }
    }
}

You can see in this example that the gender processor requires two inputs: a population column and a gender ratio column. Note that we have a system of global defaults that can be used to autogenerate e.g. the gender ratio column if it is not present in the exposure dataset.

The output columns are also defined in the post processor definition. In this case the post processor only defines one output column: 'women', although there is no limit to how many output columns a post processor may write.

One thing to note is the formula defined in the output section of the post processor. This is where the actual calculation is defined.

population * gender_ratio

In other words: Multiply the content of the population column by the content of the gender_ratio column and write the outputs into a column named "women_count_field".

Disabilities

Building properties

Minimum needs

DaLA

Exposure Definitions

Hazard Definitions

[timlinux]

@mbernasocchi has plans to start implementing these tools too....

[Charlotte-Morgan]

Hi @samnawi , @ivanbusthomi & @felix-yew - this is an RFC about the next important phase of UI development for InaSAFE. Please review this proposal and make comments. What UI tools do you think are most relevant to DM users in Indonesia; what should be a priority for DMI to fund and support? This work will be our priority in the next quarter and for InaSAFE 4.3

[Charlotte-Morgan]

@timlinux - do you still want input to this in the light of the InaSAFE roadmap discussion in September?