Failed to build map report

[akbargumbira]

Problem

Hi @lucernae @myarjunar, I have a weird problem. I am trying to build a map report (the same with test_qgis_map_pdf_report in test_impact_report.py). But the problem seems that when extracting context for map_elements, 2 out of 3 layers are None). Looking at composer_extractor and debugging it, it happens at line 248

The hazard and exposure are None because the provenance ID is different with the ID in layer registry:

Any idea what's wrong?

[timlinux]

@akbargumbira Are you perhaps loading / unloading the layers in your processing flow? Can you share your full code? @myarjunar can you dip and and take a look (on WB time please)

[akbargumbira]

Yes, to the map layer registry

    hazard_layer = get_layer(cli_arguments.hazard, 'Hazard Layer')
    exposure_layer = get_layer(cli_arguments.exposure, 'Exposure Layer')
    layers = [hazard_layer, exposure_layer]
    aggregation_layer = get_layer(
        cli_arguments.aggregation, 'Aggregation Layer')
    if aggregation_layer:
        layers.append(aggregation_layer)
    layer_registry = QgsMapLayerRegistry.instance()
    layer_registry.addMapLayers(layers)
    layer_registry.addMapLayers(impact_function.outputs)

    # create impact report instance
    report_metadata = ReportMetadata(
        metadata_dict=update_template_component(component=map_report)
    )
    impact_report = ImpactReport(
        iface,
        report_metadata,
        impact_function=impact_function)
    # get the extent of impact layer
    impact_report.qgis_composition_context.extent = \
        impact_function.impact.extent()
    # set the ouput folder
    impact_report.output_folder = cli_arguments.output_dir

    return impact_report.process_components()

[akbargumbira]

Put aside why the hazard ID is different in provenance and the one registered in layer registry, in this line https://github.com/inasafe/inasafe/blob/develop/safe/report/extractors/composer.py#L248, why is it necessary to match them? Why not for example just get the hazard layer from impact report instance? i.e impact_report.impact_function.hazard

[myarjunar]

@akbargumbira because it's a different layer object, https://github.com/inasafe/inasafe/blob/develop/safe/report/extractors/composer.py#L248 will return a layer object from the map canvas (registry), and composer need the object layer from the map canvas. I've checked the qgis source code, in qgis 3 we can simply use this impact_report.impact_function.hazard because they have a capability to automatically check whether the layer object appear in the map registry or not, but not with the previous version. CMIIW

[lucernae]

Hi @akbargumbira sorry if I missed this.

Put aside why the hazard ID is different in provenance and the one registered in layer registry, in this line https://github.com/inasafe/inasafe/blob/develop/safe/report/extractors/composer.py#L248, why is it necessary to match them? Why not for example just get the hazard layer from impact report instance? i.e impact_report.impact_function.hazard

@myarjunar is right. Previously there is some weird case in QGIS 2.14 (the version on where I wrote this). That line is just me being paranoid to make sure that we get the actual layer stored in layer registry to be used. The way impact_function worked previously, there is no guarantee that impact_function.hazard is the one that is being loaded in layer registry, because there is many preprocess happens on the fly (add/delete columns), the hazard object might not be the same python object even though the important content might be the same. This is what I remembered.

Just curious @akbargumbira what QGIS version are you using?

Also, I don't see places where you execute your analysis? I noticed that this one:

hazard_layer = get_layer(cli_arguments.hazard, 'Hazard Layer')

and this one:

impact_function.hazard

Can be a different layer object. The one that you get from cli_arguments is what I assumed the original layer that you passed along. However the one that is being processed by impact_report is the one that is stored in impact_function. Also, layer that is referenced by impact_function are layers that undergo preprocess/postprocess, so it can be a different Layer object. To illustrate it simply, if the original hazard layer is a vector layer that contains 12 columns. impact_function.hazard might be stripped of unnecessary columns. Something like that.

What I might suggest is to just pass hazard and exposure layer from impact_function into layer_registry instead of from cli_arguments.

[Gustry]

I'm half reading the thread, not sure I got the problem.

Layers which are loaded in the IF and layers that you get at the end for exposure, hazard and aggregation are definitely different. The impact function is creating new layers and we are replacing old layers with the new one.

[akbargumbira]

@lucernae I am using QGIS 2.18 and it's executed from here https://github.com/inasafe/inasafe_cli/blob/master/inasafe_cli/test_inasafe.py#L90

[Gustry]

FYI, in 4.3 we replace the hazard/exposure and aggregation by the original layer at the end of the impact function, because we can now serialize/unserialize an impact function by taking original layers.

@myarjunar What should we do with this ticket?