Structure field content with Unicode characters and leading whitespace breaks the Panel

lemmon commented 2 years ago

Description

Issue with combination of whitespace and non-latin characters when parsing/encoding the structure field.

To reproduce

Simply create a page with following content:

Menu:

-
  title: >
     Lorem ipsum dolor sit amet,
    čonsectetur adipiscing elit. Nam sed
    blandit orci.

Note the whitespace before "Lorem" and character "č" on the second [multi]line. This happened completely by accident when my client created an entry containing this east-eu characters and inserted a whitespace at the beginning of created text.

I understand that this issue is related to parsing or encoding yaml structures, or possibly both. I am not sure what is the correct handling of such situations according to yaml spec.

I have tried the same content without the special characters -- Kirby panel didn't break but the data wasn't parsed correctly.

It would help if Kirby added an option to trim the additional whitespace when creating content. It happens to me a lot that my clients adding unnecessary spaces and this, as demonstrated, can lead to unintended consequences.

Your setup

Kirby Version
3.6.5

lukasbestle commented 2 years ago

I can reproduce this issue. The issue already begins when the YAML string is decoded in Data\Yaml::decode().

With the string from your example the parsed array is:

array (size=1)
  0 => 
    array (size=1)
      'title' => string 'Lorem ipsum dolor sit amet, �onsectetur adipiscing elit. Nam sed landit orci.' (length=77)

With the space at the beginning removed, the result is:

array (size=1)
  0 => 
    array (size=1)
      'title' => string 'Lorem ipsum dolor sit amet, čonsectetur adipiscing elit. Nam sed blandit orci.' (length=79)

So it looks like our YAML parser Spyc butchers the Unicode character in the string. This in turn breaks the Panel.

Regarding solutions: We have plans to replace Spyc with Symfony YAML, which is much more robust. You are right that this particular issue can be fixed by stripping the whitespace, but this would break use cases where the whitespace is actually intended. And it isn't a full solution as the Spyc library was causing various problems over the years. So we rather want to fix the source of the issue.

lemmon commented 2 years ago

Yes, I understand that automatic whitespace management is not good. I ment it as an optional feature. Still would love to see it, regardless of this issue.

getkirby / kirby