shapefile not plotted in latest version

[brancomat]

Newer version(s?) of arkimaps apparently doesn't plot anymore the include shapefile Sottozone_allerta_ER. Last known version of arkimaps working: 1.19-1

Example recipe: https://github.com/ARPA-SIMC/arkimaps/blob/master/recipes/mslp.yaml

Test data: https://drive.google.com/file/d/1qOUvuO2LdIyPLUP49MQ910JIuTH1ay1X/view?usp=sharing

Test command (after dispatch):

./arkimaps preview dispatched_data/ --flavour emro mslp+0

Expected result: the following settings in flavours/emro.yaml should be honored

     add_user_boundaries:
       shape: sottozoneAllertaER

The shapefile name slightly differs (./arkimapslib/static/shapes/Sottozone_allerta_ER.shp) but even using the full path doesn't seem to work

[spanezz]

I confirm that:

• in <workdir>/renderers/renderer000.py, "sottozone" is not mentioned
• if I run ./arkimaps document-recipes --flavours emro, in the documented recipe "sottozone" is also not mentioned

If I add an empty - step: add_user_boundaries to mslp.yaml, document-recipes, then it appears both in the recipe documentation and in the renderer.

In theory this is expected: if there was no such step in the recipe, the flavour does not know where to add it. I suppose it's why emro_web is configured to skip this step.

I looked at the code of 1.19-1 and both the emro flavour and the mslp recipe seem similar, and I couldn't find where it the code it would add a step based on the flavour. that remains a mistery to me

[spanezz]

Checking the difference between the rendering scripts in 1.19-1 and the current master I do not see "sottozone" mentioned in the 1.19-1 rendering script either

[brancomat]

I apologize, wrong recipe example.

Using cape data (test data: https://drive.google.com/file/d/1TijXxfOXqSU-FuTzVZrsojJc-3LjHfDM/view?usp=sharing recipe: cape.yaml) and the command ./arkimaps preview t1/ --flavour emro cape+0 (after dispatch) the resulting image doesn't have the shapefile plotted (but now the file is mentioned in the rendering script renderer000.tar.gz ):

[spanezz]

Thanks, now it all makes more sense. The difference between 1.19-1 and current master rendering scripts is:

before:

parts.append(macro.mcoast(map_user_layer='on', map_user_layer_colour='blue', map_coastline_resolution='high', map_user_layer_name='/media/arkimaps/arkimapslib/static/shapes/Sottozone_allerta_ER'))

after:

parts.append(macro.mcoast(map_coastline_resolution='high', map_user_layer_colour='blue', map_user_layer_name='/media/arkimaps/arkimapslib/static/shapes/Sottozone_allerta_ER'))

Note that map_user_layer='on' is gone. I'll now check where it went

[spanezz]

The way values from flavours, recipes, and step defaults were merged did not recurse into the params dict, but would replace them, causing in many cases the DEFAULTS dicts in https://github.com/ARPA-SIMC/arkimaps/blob/master/arkimapslib/steps.py to be ignored.

map_user_layer='on' was exactly in that DEFAULTS dict.

I pushed a fix, that does the merging recursing into dictionaries