Peril code grouping

[stufraser1]

Explore whether adjustment to mapping file can address problems found in testing, with handling grouped peril codes in one model format (AIR) and separate peril codes in another (OED). Documentation should highlight this as a known issue for this translation.

[stufraser1]

@tristanlloyd @benhayes21 Please could you add an example of the codes and the transformation direction requiring code splitting / direction requiring combination for this issue, to assist understanding the logic required to implement it in the mapping file. Thanks

[stufraser1]

Previous feedback from Tristan: (This example is closely related to, but not exactly the same as, splitting a grouped peril code to a string of peril codes, so I may be conflating two issues)

Trying to implement something like; transformation: 1 when: search(LayerPeril,‘PWH’) to split an AIR peril string like ‘PWH+PSH+PPH’ etc., to create separate peril columns with 1 or blanks (as required in our Sequel Impact Import). The string manipulation functions implemented in the converter ONLY work for the transformation: part, and can’t be used in the when: clauses. Can only put the search() function in the transformation: part, to return TRUE / FALSE. Then needed to post-process the resulting file to convert to 1 / ’’, [Couldn’t nest the search() within a replace(), i.e., replace(search(LayerPeril,‘PWH’),’TRUE’, 1,’FALSE’,’’) ].

[stufraser1]

The best reference is https://docs.air-worldwide.com/Database/CEDE/9.0/topic137.html Grouped peril code: PWS equates to PWH+PSH Windstorm PEQ equates to PES+PFF+PTS+PLS+PSL All Earthquake perils PFH equates to TS PFL+PSH Inland and Coastal Flood

Is the best way to deal with this, to include the original and destination in the peril code listing, i.e. transform the grouped code to string in the same way as we transform a single code to a single code (is any other logic actually needed?

[OmegaDroid]

Spliting these into multiple transformations is the way to go here, something like

transformation: 1 
when: search(LayerPeril,‘PWH’) 

is correct, is this not working?

The reason this wouldn't work

replace(search(LayerPeril,‘PWH’),’TRUE’, 1,’FALSE’,’’)

is that the first argument to replace must be a string and search returns a boolean.

[stufraser1]

Running this on a basic test does not appear to work as expected. Requires tests to confirm correct structuring of transformation using multiple where clauses, and documentation of how to perform these types of conditional statements with clear examples. Per https://github.com/OasisLMF/OpenDataTransform/issues/27

Issues found in running tests using the one-way-transformation:

• Using 'tab' to indent the where: clause, returns error: found character '\t' that cannot start any token in "/Users/stufraser/OEDtransform/examples/one-way-transformation/A-B.yaml"

• Using where: b >= 10 return error: Expecting: {'GT', 'IS', 'PLUS', 'ANON_0', 'STAR', 'AND', 'MINUS', '__ANON_2', 'SLASH', 'GTE', 'LT', 'ANON_1', 'OR', 'LTE'}

• Using where: b = 10 return error: Expecting: {'GT', 'IS', 'PLUS', 'ANON_0', 'STAR', 'AND', 'MINUS', '__ANON_2', 'SLASH', 'GTE', 'LT', 'ANON_1', 'OR', 'LTE'}

• Using where: b == 10 return error: Expecting: {'GT', 'IS', 'PLUS', 'ANON_0', 'STAR', 'AND', 'MINUS', '__ANON_2', 'SLASH', 'GTE', 'LT', 'ANON_1', 'OR', 'LTE'}

• Using where: b GTE 10 return error: Expecting: {'GT', 'IS', 'PLUS', 'ANON_0', 'STAR', 'AND', 'MINUS', '__ANON_2', 'SLASH', 'GTE', 'LT', 'ANON_1', 'OR', 'LTE'}

• • Using where: b IS 10 return error: Expecting: {'GT', 'IS', 'PLUS', 'ANON_0', 'STAR', 'AND', 'MINUS', '__ANON_2', 'SLASH', 'GTE', 'LT', 'ANON_1', 'OR', 'LTE'}

• Using when: b is not in [10] works but is not as intuitive as using where: b <> 10, for example

Running the following, the transformation succeeds with no error, but only the 1st transformation is completed; second one is blank.

• transformation: b + 10 when: b is not in [10]
• transformation: 99 when: b is 10

Running the following, the transformation succeeds with no error, but all values in d return 99

• transformation: 99 when: b is 10
• transformation: b + 10 when: b is not in [10]