incorrect scaling of emissions by spatial proxy leads to failing tests

[prayner]

The problem

Versions of the prior which scale a national emission by a spatial proxy are failing because the spatial sum of the proxy over the domain does not match the denominator in the scaling equation local-emission = national-emission times proxy /proxy.sum() where proxy is a 2d field. This is wrong but not simply a code bug. In the first place we should not expect it to work for domains less than national. That is dealt with by the CI tests running on the national domain. But secondly because of the different projections of the proxy and the OM domain there are always significant parts of the proxy lying outside the OM domain. There may also be parts of the OM domain lying outside Australia (e.g. parts of PNG and Indonesia) and these should not be included. Thus we will need to integrate any proxy over an entity we can define as Australia. We will need this even for subdomains since the above spatial-emission equation needs proxy.sum() for Australia, not tjust the chosen domain.

Definition of "done"

Tests are passed and the spatial-emission equation is calculated correctly for national and subdomains. We will need:

• some way of defining whether a given point in a proxy lies inside or outside Australia
• the inclusion of a national sum of proxies whether or not we are using the national domain.

  Additional context

  A possible route is the landuse data file already provided since it is naturally limited to Australia. It is an inefficient way of calculating inclusion though, especially at the high resolution of some of the proxies.

[aethr]

@prayner I think I understand the problem.

Perhaps we need to understand how the "national emissions" we're referencing determine the boundaries or inclusions in their data set. Let's discuss in person next week.

[prayner]

A little more detail on the solution: The key problem is how we calculate the sum of a proxy over a domain which is (1) consistent with the national inventory, i.e Australia and (2) consistent with the land mask if it is a masked field. Here is a proposed algorithm:

• turn the config.layer_inputs.land_use_path tif image into a mask, 1 for land, 0 for ocean
• Use remap_raster to map it to the domain
• Normalise this field by dividing by its maximum value. This should generate something close to a land mask but limited to Australia
• Multiply this by the land mask so it's consistent with how we will treat fields later
• Set all values > 0 to 1 so it is a true mask
• OPtionally store this so we don't need to regenerate it. This should be in the intermediates directory
• Now when proxying any field, either for the national domain or a subdomain:
• Remap the proxy to the national domain
• Multiply by this Australian land mask.
• Calculate the sum, this will be the denominator in the emissions equation
• Calculate the proxy across the whole domain but mask it with this land mask.

Consequences:

• Australian totals should match independent of the subdomain we are using
• Emissions for this category will be zero for areas outside Australia (not ideal)

Some requirements:

• A field representing the Australian land mask.
• A pointer to a national domain file even if we are using a subdomain

Questions:

• where should the Australian land mask be stored, in the national om-domain-info file or in an intermediate file?
• Should we try to cover areas inside the OM domain but outside Australia?

[prayner]

Some small extra thoughts: I would name this new mask something like inventory_mask. Remember that it has to be attached to a national domain, potentially with a different extent and resolution than the subdomain we are using. I suggest adding a attribute inventory_domain_ds to the configuration and populating it as we do domain_ds. If we don't find an inventory_mask field in the inventory_domain we should create it (see prev post). This needs a function read_or_create_inventory_mask acting on the config. For testing I suggest replacing this with a stub that just copies the land_mask. This will already change results since we will be summing only over the land_mask. Once that's working we implement the algorithm for defining the inventory_mask.