Diazo theme with subrequest show wrong characters (Python 3)

[mauritsvanrees]

In Diazo rules, you can use <replace href="@@some-fragment" /> to insert the contents of a some-fragment view. We use this in a Plone 5.1 site to show some text that Managers can edit in a control panel. Works fine, also with non-ascii. On plone 5.2 with Python 3, it works with ascii, but shows very ugly when the fragment contains non-ascii.

To reproduce:

• Take Plone coredev buildout 5.2 with Python 3.
• In the ZMI add a simple Page Template with id non-ascii. Add some non-ascii in it, for example Greek: <p>Αρχική Σελίδα</p>. (Pronounced roughly like "Archikí Selída", apparently meaning "Home". No, I can't read this.)
• In the theming control panel copy the barceloneta theme, activate the new theme, and edit the rules.xml.
• Comment out the part with <replace css:theme-children="#portal-footer .doormat">
• Instead, add this: <replace css:theme-children="#portal-footer .doormat" css:content="body" href="/non-ascii" />

What I expect to happen:

The non-ascii Greek characters should show up readable. (If you can read Greek, anyway.)

What actually happened:

This showed:

Αρχική Σελίδα

Even Greeks can't read this.

Note: I have the same problem when I use French and a simple accented character. Actualités is rendered as Actualités.

What version of Plone/ Addons I am using:

Plone 5.2.1 on Python 3.7. With Python 2.7 it works fine.

I guess this is a problem somewhere in plone.app.theming or Diazo. In plone.app.theming this code in utils.py is called. With a subrequest I get a response in bytes with a charset utf-8. At the end of this method, result is a string and I can print it just fine.

Any idea?

[csanahuja]

This is what I reported in plone.app.theming in case it helps: https://github.com/plone/plone.app.theming/issues/162

[mauritsvanrees]

Thanks for the pointer. That helps. I made a PR: https://github.com/plone/plone.app.theming/pull/180