Issue with YAML sub-editor (in normal Automations UI). Converting | (literal) to >(folding), and reforming YAML

[bryanbr23]

Checklist

• [X] I have updated to the latest available Home Assistant version.
• [X] I have cleared the cache of my browser.
• [X] I have tried a different browser to see if it is related to my browser.

Describe the issue you are experiencing

Using the Automation Editor, I prepared a notification automation. Because the automation required templating, I switch to the YAML subeditor (within Automation UI), and UI is reformatting | (literal operator) to > (folding) YAML editor.

Describe the behavior you expected

In YAML edit (from UI editor)

service: notify.twilio data: title: "⚠️ *Greetings " target:

• message: > # (HERE IS THE PROBLEM) Here is the current house state and weather forecast{{"\r\n"}}

Steps to reproduce the issue

1. Edit the Automation, Add back the | literal operator, remove extra spacing.
2. Exit Editor.
3. Return to YAML editor and see that the > folding operator has been added back in, and replaces my properly formatted YAML. Tried on Edge browser (Private window), and Brave (Private window).

What version of Home Assistant Core has the issue?

2023.12.1

What was the last working version of Home Assistant Core?

No response

In which browser are you experiencing the issue with?

Microsoft Edge Version 119.0.2151.97 (Official build) (64-bit)

Which operating system are you using to run this browser?

Win11 Enterprise 22621.2792

State of relevant entities

######################################################################
# Original Text, properly formatted with | Literal Operator
######################################################################
message: |
  Here is the current house state and weather forecast{{"\r\n"}}
  Outdoor Temp: {{states('sensor.outdoor_temp_and_humidity_temperature') }}
  Living Room Temp: {{states('sensor.living_room_temperature_temperature') }}
  Living Room Humidity: {{ states('sensor.living_room_temperature_humidity') }} 
  Basement Water Leak: {{ states('binary_sensor.stu_basement_water_leak') }} 
  Kitchen Water Leak:     {{ states('binary_sensor.stu_kitchen_water_leakl')}}

  Forecast: {% if states.weather.openweathermap is defined %}
  {% set days_to_show = 5 %}
  {% for forecast in
  states.weather.openweathermap.attributes.forecast[:days_to_show] %}
    {% set date = as_datetime(forecast.datetime) %}
    {{ date.strftime("%A") }}: High: {{ forecast.temperature }}°C, Low: {{ forecast.templow }}°C
    Condition: {{ forecast.condition.capitalize() }} {{ '%0.2f'|format(forecast.precipitation | float / 10.0) }} cm
    {% endfor %}
  {% else %}
    No forecast data available.
  {% endif %}

########################################################################################
# Upon Saving, closing browser, and opening again | Literal Operator is switched to > folded operator
########################################################################################
message: >
  Here is the current house state and weather forecast{{"\r\n"}}

  Outdoor Temp:             
  {{states('sensor.outdoor_temp_and_humidity_temperature') }}

  Living Room Temp:      
  {{states('sensor.living_room_temperature_temperature') }}

  Living Room Humidity: {{ states('sensor.living_room_temperature_humidity')
  }} 

  Basement Water Leak: {{ states('binary_sensor.stu_basement_water_leak') }} 

  Kitchen Water Leak:     {{ states('binary_sensor.stu_kitchen_water_leakl')}}

  Forecast: {% if states.weather.openweathermap is defined %}

  {% set days_to_show = 5 %}

  {% for forecast in

  states.weather.openweathermap.attributes.forecast[:days_to_show] %}
    {% set date = as_datetime(forecast.datetime) %}
    {{ date.strftime("%A") }}: High: {{ forecast.temperature }}°C, Low: {{ forecast.templow }}°C
    Condition: {{ forecast.condition.capitalize() }} {{ '%0.2f'|format(forecast.precipitation | float / 10.0) }} cm
    {% endfor %}
  {% else %}
    No forecast data available.
  {% endif %}

Problem-relevant frontend configuration

Here are examples of the YAML that is changing.  The | literal operator was used, and properly formatted, then when save/different page viewed, the automation is re-written to use the > folding operator.  

######################################################################
# Original Text, properly formatted with | Literal Operator
######################################################################
message: |
  Here is the current house state and weather forecast{{"\r\n"}}
  Outdoor Temp: {{states('sensor.outdoor_temp_and_humidity_temperature') }}
  Living Room Temp: {{states('sensor.living_room_temperature_temperature') }}
  Living Room Humidity: {{ states('sensor.living_room_temperature_humidity') }} 
  Basement Water Leak: {{ states('binary_sensor.stu_basement_water_leak') }} 
  Kitchen Water Leak:     {{ states('binary_sensor.stu_kitchen_water_leakl')}}

  Forecast: {% if states.weather.openweathermap is defined %}
  {% set days_to_show = 5 %}
  {% for forecast in
  states.weather.openweathermap.attributes.forecast[:days_to_show] %}
    {% set date = as_datetime(forecast.datetime) %}
    {{ date.strftime("%A") }}: High: {{ forecast.temperature }}°C, Low: {{ forecast.templow }}°C
    Condition: {{ forecast.condition.capitalize() }} {{ '%0.2f'|format(forecast.precipitation | float / 10.0) }} cm
    {% endfor %}
  {% else %}
    No forecast data available.
  {% endif %}

########################################################################################
# Upon Saving, closing browser, and opening again | Literal Operator is switched to > folded operator
########################################################################################
message: >
  Here is the current house state and weather forecast{{"\r\n"}}

  Outdoor Temp:             
  {{states('sensor.outdoor_temp_and_humidity_temperature') }}

  Living Room Temp:      
  {{states('sensor.living_room_temperature_temperature') }}

  Living Room Humidity: {{ states('sensor.living_room_temperature_humidity')
  }} 

  Basement Water Leak: {{ states('binary_sensor.stu_basement_water_leak') }} 

  Kitchen Water Leak:     {{ states('binary_sensor.stu_kitchen_water_leakl')}}

  Forecast: {% if states.weather.openweathermap is defined %}

  {% set days_to_show = 5 %}

  {% for forecast in

  states.weather.openweathermap.attributes.forecast[:days_to_show] %}
    {% set date = as_datetime(forecast.datetime) %}
    {{ date.strftime("%A") }}: High: {{ forecast.temperature }}°C, Low: {{ forecast.templow }}°C
    Condition: {{ forecast.condition.capitalize() }} {{ '%0.2f'|format(forecast.precipitation | float / 10.0) }} cm
    {% endfor %}
  {% else %}
    No forecast data available.
  {% endif %}

Javascript errors shown in your browser console/inspector

None

Additional information

[karwosts]

I don't believe using the visual editor we guarantee that your YAML is preserved exactly as you typed it, it is converted to JSON and back a few times and the original formatting may not be preserved.

Does this reformatting actually break the output in some undesirable way, or is this issue just purely that you don't like that the yaml has been changed?

[bryanbr23]

Hi,

Thanks for the reply. I wasn't aware the visual editor was not guaranteeing YAML and is more best-attempt.
I had properly formatted a template to send SMS's with numbers lining up, etc. And when the visual editor switching the actual YAML and replacing some operators, that's where things fell down. So, yes the reformatting actually broke my desired output.
It sounds like the current work-around is to edit YAML in the automations file and not use the Visual Editor/Automations UI. If that's the case, that's fine.

Thanks for the hard work on HA!
~Bryan

[karwosts]

I'm actually not too knowledgeable about this area, but I might have expected the folded yaml to produce the same output as the literal yaml, given how it is substituting extra newlines.

Care to share an example of the output formatting, and how it is incorrect?

[bryanbr23]

Hi, Sure, I'm happy to share a sample, and maybe I'm doing something wrong.

My understanding is per the following (cp from StackOverflow): `Use > most of the time: interior line breaks are stripped out, although you get one at the end:

key: > Your long string here. Use | if you want those linebreaks to be preserved as \n (for instance, embedded markdown with paragraphs).

key: |

Heading

* Bullet
* Points

Use >- or |- instead if you don't want a linebreak appended at the end.`

Here is sample from folding > block operator:

Here is sample from literal | block operator:

Both use Twilio to send SMS. But basically you can see spaces that get munged or extra lines added.
Here is literal operator:

Here is the folding operator:

I think I'm fine editing the automations file with VSCode. But if you want feature parity in Visual YAML editor, maybe there might be some fixes needed.

Again, thank you for your help! Ping back with any future questions. ~Bryan

[karwosts]

That's interesting as in folded yaml, it says you can still get an explicit line break by adding two newlines, which I see that the editor has tried to add. (note how it added an extra newline next to each of your sensors when switching to folded).

Wonder why that did not work.

[Soukyuu]

I don't believe using the visual editor we guarantee that your YAML is preserved exactly as you typed it, it is converted to JSON and back a few times and the original formatting may not be preserved.

Does this reformatting actually break the output in some undesirable way, or is this issue just purely that you don't like that the yaml has been changed?

Chiming in on this: I mostly do not like how the formatting is changed. I expect switching to the YAML sub editor to keep my YAML as I formatted it. There is even some completely useless changes like changing

variables:
  airing_list: |
    {% set hum_values = {
      "sensor.arbeitszimmer_raumklima_humidity": "sensor.komfort_arbeitszimmer_absolute_humidity",
      "sensor.badezimmer_raumklima_humidity": "sensor.komfort_bad_absolute_humidity",
      "sensor.toilette_raumklima_humidity" : "sensor.komfort_toilette_absolute_humidity",
      "sensor.esszimmer_raumklima_humidity" : "sensor.komfort_esszimmer_absolute_humidity",
      "sensor.kuche_raumklima_humidity" : "sensor.komfort_kuche_absolute_humidity",
      "sensor.kinderzimmer_raumklima_humidity" : "sensor.komfort_kinderzimmer_absolute_humidity",
      "sensor.wohnzimmer_raumklima_humidity" : "sensor.komfort_wohnzimmer_absolute_humidity",
      "sensor.schlafzimmer_raumklima_humidity" : "sensor.komfort_schlafzimmer_absolute_humidity"} %}
    {% set hum_out = "sensor.komfort_aussen_absolute_humidity" %}
    {% set hum_rel_tres = states("input_number.luften_schwelle_feuchtigkeit") | float %}
    ...
    {% for name in ns.room_names %}
      {% set ns.room_string = ns.room_string + "\n - " + name %}
    {% endfor %}
    {{ ns.room_string }}

to

variables:
  airing_list: >
    {% set hum_values = {
      "sensor.arbeitszimmer_raumklima_humidity": "sensor.komfort_arbeitszimmer_absolute_humidity",
      "sensor.badezimmer_raumklima_humidity": "sensor.komfort_bad_absolute_humidity",
      "sensor.toilette_raumklima_humidity" : "sensor.komfort_toilette_absolute_humidity",
      "sensor.esszimmer_raumklima_humidity" : "sensor.komfort_esszimmer_absolute_humidity",
      "sensor.kuche_raumklima_humidity" : "sensor.komfort_kuche_absolute_humidity",
      "sensor.kinderzimmer_raumklima_humidity" : "sensor.komfort_kinderzimmer_absolute_humidity",
      "sensor.wohnzimmer_raumklima_humidity" : "sensor.komfort_wohnzimmer_absolute_humidity",
      "sensor.schlafzimmer_raumklima_humidity" : "sensor.komfort_schlafzimmer_absolute_humidity"} %}
    {% set hum_out = "sensor.komfort_aussen_absolute_humidity" %}
    {% set hum_rel_tres = states("input_number.luften_schwelle_feuchtigkeit") |
    float %}
   ...
   {% for name in ns.room_names %}
      {% set ns.room_string = ns.room_string + "\n - " + name %}
    {% endfor %} {{ ns.room_string }}

Why on earth would it wrap and combine short lines while there is a much longer line? It makes the code completely unreadable.

I would like to not have the YAML editor reformat my code on save. Pretty please with a cherry on top?

I think it is already possible if the code entered cannot be viewed with the visual editor. It then does not touch the content, I think?

[jantman]

FYI this appears to be the same as #14368

I'm pretty sure I'm having problems related to this as well, as it seems to be adding line breaks in the middle of template tags for me (also for notifications), and my actual notifications never show anything past the first line break (presumably because Jinja doesn't allow line breaks inside of a template tag).

Example of this, for me:

message: Roomba ERROR {{ trigger.to_state.attributes['error_code'] }}: {{ trigger.to_state.attributes['error'] }} (status {{ trigger.to_state.attributes['status'] }})

gets converted to:

  message: >-
    Roomba ERROR {{ trigger.to_state.attributes['error_code'] }}: {{
    trigger.to_state.attributes['error'] }} (status {{
    trigger.to_state.attributes['status'] }})

And this just shows a persistent notification of Roomba ERROR 104:

[chupacabra71]

same here... it's been driving me nuts! could not figure why the folding operator keeps getting injected in the script i am building. i set a bunch of variables in a template:

variables:
  options: "{{ [option_one, option_two, option_three] | selectattr('enabled') | list }}"
  expect_response: |-
    {{ enable_timeout or option_one_enabled or option_two_enabled or
    option_three_enabled }}
  notification_data: |-
    {% set payload = namespace(data={ 'apple_device': true }) %} 
    {% set push = namespace(d={}) %} 
    {% set payload.data = dict( payload.data, **{ 'subtitle': subtitle }) %} 
    {% set push.d = dict(push.d, **{ 'interruption-level': interruption_level })%} 
    {% if notification_link | length %} 
    {% endif %} 
    {% if attachment_type == 'camera_entity' %} 
      {% set payload.data = dict(payload.data, **{ 'entity_id': attachment_camera_entity }) %}
    {% endif %} 
    {% set payload.data = dict(payload.data, **{ 'push': push.d }) %} 
    {% set payload.data = dict(payload.data, **{'actions': options }) %} 
    {% set payload.data = dict(payload.data, **{'tag':tag }) %} 
    {% if group|length %}
      {% set payload.data = dict(payload.data, **{'group': group }) %} 
    {% endif %}
    {{ payload.data }} 
alias: Setup Mobile Payload

but then the output shows the folding operator:

options: []
expect_response: true
notification_data: >-
  [{'apple_device': True, 'subtitle': Undefined, 'push': {'interruption-level':
  Undefined}, 'actions': [], 'tag': Undefined}]

[bryanbr23]

Work around is to edit the yaml manually and avoid the visual editor in the UI.

[chupacabra71]

Work around is to edit the yaml manually and avoid the visual editor in the UI.

I tried, but seem to be getting the same results... this is the script (notice it is yaml only):

still yields the same result:

 Executed: April 14, 2024 at 10:33:16 AM
Error: expected dict for dictionary value @ data['data']
Result:

params:
  domain: notify
  service: mobile_app_seb_m1mbp
  service_data:
    title: NO MESSAGE SELECTED
    message: THE MESSAGE
    data: >-
      {'apple_device': True, 'subtitle': '', 'push': {'interruption-level':
      Undefined}, 'actions': [], 'tag':
      'script.beta_unified_notification_script-01HVEGDNDY6CD4SR977N4GN8PS'}
  target: {}
running_script: false

[chupacabra71]

i was looking at the trace and it appears that something is converting the yaml code as if it was built in the UI... this is the code in the yaml file where the errors happen:

    - alias: Setup Payload
      variables:
        options: >-
          {{ [option_one, option_two, option_three] | selectattr('enabled') | list }}
        expect_response: "{{ enable_timeout or option_one_enabled or option_two_enabled or option_three_enabled }}" # If there's no timeout nor options, then no point waiting. Stop script.
        notification_data: |-
          {%- set payload = namespace(data={ 'apple_device': true }) -%}
          {%- if payload.data.apple_device -%}
            {# iOS #}
            {%- set push = namespace(d={}) -%}
            {%- set payload.data = dict(payload.data, **{ 'subtitle': subtitle }) -%}
            {%- set push.d = dict(push.d, **{ 'interruption-level': interruption_level }) -%}
            {%- if notification_link|length %}{% set payload.data = dict(payload.data, **{ 'url': notification_link }) -%}
            {%- endif -%}
            {%- if attachment_type == 'camera_entity' %}{% set payload.data = dict(payload.data, **{ 'entity_id': attachment_camera_entity }) -%}
            {%- endif -%}
            {%- set payload.data = dict(payload.data, **{ 'push': push.d }) -%}
          {%- endif %}
          {# Common  #}
          {%- set payload.data = dict(payload.data, **{ 'actions': options }) -%}
          {%- set payload.data = dict(payload.data, **{ 'tag': tag }) -%}
          {%- if group|length %}{% set payload.data = dict(payload.data, **{ 'group': group }) -%}
          {%- endif -%}
          {{ payload.data }}

this is the resulting code in the trace. I think it's the extra whitespaces that are causing the issue. i tried to mitigate that by using the chomping methods, but the spaces persist.

alias: Setup Payload
variables:
  options: '{{ [option_one, option_two, option_three] | selectattr(''enabled'') | list }}'
  expect_response: >-
    {{ enable_timeout or option_one_enabled or option_two_enabled or
    option_three_enabled }}
  notification_data: >-
    {%- set payload = namespace(data={ 'apple_device': true }) -%}

    {%- if payload.data.apple_device -%}
      {# iOS #}
      {%- set push = namespace(d={}) -%}
      {%- set payload.data = dict(payload.data, **{ 'subtitle': subtitle }) -%}
      {%- set push.d = dict(push.d, **{ 'interruption-level': interruption_level }) -%}
      {%- if notification_link|length %}{% set payload.data = dict(payload.data, **{ 'url': notification_link }) -%}
      {%- endif -%}
      {%- if attachment_type == 'camera_entity' %}{% set payload.data = dict(payload.data, **{ 'entity_id': attachment_camera_entity }) -%}
      {%- endif -%}
      {%- set payload.data = dict(payload.data, **{ 'push': push.d }) -%}
    {%- endif %}

    {# Common  #}

    {%- set payload.data = dict(payload.data, **{ 'actions': options }) -%}

    {%- set payload.data = dict(payload.data, **{ 'tag': tag }) -%}

    {%- if group|length %}{% set payload.data = dict(payload.data, **{ 'group':
    group }) -%}

    {%- endif -%}

    {{ payload.data }}

[bryanbr23]

karwosts is one of the devs, and helping answer my initial question.
Unfortunately, the UI editor (even while viewing YAML) will load/store multiple times.

The Dev team knows there are some problems in this space.

My workaround was to craft the automations in the UI editor as needed. Then edit automations.yaml in a text editor, fix the literals that got messed with, then run the automation. Do not edit YAML in the UI editor, otherwise expect it to be messed with. It was a while ago that I had this problem, and perhaps I was able to view things, but could not save or edit, otherwise the literals would be whacked again.

I hope this helps. Not a great story, but a work-around. I"m sure the devs are focused on bugs/features with most demand. And this area probably isn't high priority.

[karwosts]

@chupacabra71 - I'm not quite sure I understand what you're getting hung up on, internally all this is stored as JSON, which has no concept of folding or nonfolding operators.

Only when we want to present JSON data as yaml it is dumped, and the dumper will choose whatever block style indicators it wants.

You say "the output shows the folding operator", that's just an artifact of how it is displayed in that specific dialog screen, there's no actual operator stored internally in your result or anything.

What is the actual problem you're getting stuck on?

[chupacabra71]

@karwosts I guess my issue is that somehow when the automation that is built in YAML runs, the dictionary that i am building in the code becomes corrupt and interpreted as something other than a dictionary. the data is there according to the traces, but it is not read as a dictionary:

    data: >-
      {'apple_device': True, 'subtitle': '', 'push': {'interruption-level':
      Undefined}, 'actions': [], 'tag':
      'script.beta_unified_notification_script-01HVEGDNDY6CD4SR977N4GN8PS'}

I have run the code through a couple of jinja parsers, and the code works fine, so my thought is that somethign is mangling either the code or the response.

[karwosts]

I'd probably try to look for some community support either on Discord #templates channel or somewhere on the forums and see what some of the template pros think. I doubt the issue you're seeing is related to folded vs non-folded yaml, it's probably some templates issue.

[github-actions[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.