NOAA-EMC / wxflow

Tools for Weather Workflows
https://wxflow.readthedocs.io/en/latest
GNU Lesser General Public License v3.0
0 stars 7 forks source link

Expand jinja environment #26

Closed DavidHuber-NOAA closed 4 months ago

DavidHuber-NOAA commented 4 months ago

Description This adds ~three~ new features to the Jinja class: 1. Template.substitute_structure is now available as a filter. This is useful for parsing jinja templates that reference multiple similar variables (e.g. ensemble member directories). A template for the ensemble members can be passed into the jinja context dictionary and used to construct the final variables based on iterators, cycle date/time, etc.

~{% set cycle_YMD = to_YMD(current_cycle) %}~ ~{% set cycle_HH = strftime(current_cycle, "%H") %}~ ~{% for mem in range(1,21) %}~ ~{% set mem3 = '%03d' % mem %}~ ~{% set tmpl_dict = {'YMD': cycle_YMD, 'HH': cycle_HH, 'MEM': mem3} %}~ ~{{ ANALYSIS_tmpl | template_substitute_structure(DOLLAR_CURLY_BRACE, tmpl_dict.get) }}~ ~{% endfor %}~

  1. Add the jinja.ext.do extension to Jinja. This enables {% do ... %} statements which execute a command for which the output should not be saved. A useful example is appending to a list. Since lists in Python return None, "None" would be written to the parsed jinja template in the following example: {% set a_list = [] %}{{ a_list.append("foo") }}. Instead, one can now use do:

    {% set a_list = [] %}
    {% do a_list.append("foo") %}
  2. Lastly, this updates the lambda function for the add_to_datetime filter so any SilentUndefined input will return a SilentUndefined, preventing obscure failures.

This is needed for noaa-emc/global-workflow#2491 Type of change

How Has This Been Tested?

Checklist

codecov[bot] commented 4 months ago

Codecov Report

Attention: Patch coverage is 50.00000% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 47.87%. Comparing base (2c48c56) to head (1873fee).

Files Patch % Lines
src/wxflow/jinja.py 50.00% 0 Missing and 1 partial :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #26 +/- ## =========================================== + Coverage 47.84% 47.87% +0.03% =========================================== Files 18 18 Lines 1649 1650 +1 Branches 335 335 =========================================== + Hits 789 790 +1 Misses 801 801 Partials 59 59 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

DavidHuber-NOAA commented 4 months ago

After a conversation with @aerorahul, I removed the Template-based filter. I will instead write a method in the Jinja class to handle substitution using regex in a separate PR.