Views Incorrectly Encode UTF Characters as HTML Entities

[vladaman]

Issue

The current behavior in the views section of the application involves encoding UTF characters as HTML entities.

Actual Behavior

When displaying Accounts, Notes and Contacts views the links are showing with HTML entities.

Possible Fix

Decode HTML entities into UTF-8 Behavior is working correctly on SuiteCRM 8.3.0 at https://suite8demo.suiteondemand.com/ in English Version.

Steps to Reproduce

1. Change SuiteCRM to non-english language (issue may be in
2. Create an Account with non-english characters like "šččřž"
3. Display list of Accounts

Your Environment

• SuiteCRM Version used: 8.2.4
• Browser name and version (e.g. Chrome Version 51.0.2704.63 (64-bit)): Version 1.51.118 Chromium: 113.0.5672.126 (Official Build) (64-bit)
• Environment name and version (e.g. MySQL, PHP 7): PHP 8.1
• Operating System and version (e.g Ubuntu 16.04): Ubuntu 22.04

[johnM2401]

Hey @vladaman

I've had a test on both Demo and locally using the characters you provided, only the first one seems to be encoded incorrectly:

(i'll mark this as a bug as there do seem to be some characters that are encoding incorrectly)

However, most other characters i've tried DO seem to be encoding well.

Do you have any further examples of characters that you found encode poorly?

As the issue seems much more prevalent on your local environment, it might be worth checking a few things:

• Do you have the mbstring enabled for PHP? You can check which modules are enabled by running the following on your server: php -m If you see "mbstring" in the list shown, it is enabled correctly.

• What charset is PHP currently using? You can confirm by running: php -i | grep default_charset (For example, I see: default_charset=> UTF-8 => UTF-8 )

Thanks!

[lukio]

Hi @johnM2401 @vladaman I can confirm this issue at the field description. At detail/edit seems to render as expected, but when it's rendered at listview the description field renders with wrong enconding.

DetailView

I also capture the graphql response and at detail (when getRecord) is called the description field came with the characteres as expected.

ListView

The graphql response (getRecordList) brings the description field with wrongs characters as you can see.

I verify and the output of the php -i and php -m are OK.

[lukio]

I also verify the issue with parent_name field at https://suite8demo.suiteondemand.com/

[pgorod]

What about the DB? IS it wrong in the DB?

There should be no HTML escaping in the database. If we do this wrong (which we do in v7!) all over the place, then we need to revert the damage in hundreds of places in the UI, one by one.

Then, when we fix the central problem, we will need to fix back all the previous fixes, bug by bug, one by one. We should really get a grip on this "security clean-ups" issue, otherwise we'll just keep adding to the technical debt...

[lukio]

Hi! I believe that those kind of change is too mucho. Just for now, we would re-encode those fields when retrieving data as getRecord should be doing.

[pgorod]

I'm tying to reproduce the issue locally but I can't. I'm on 8.4.1, maybe that's why.

Is it really necessary to change the SuiteCRM language to something non-english? That's a lot of work which I am trying to avoid. Did anyone reproduce this in a plain English version? How, exactly?

Thanks

[yunusyerli1]

Hi @johnM2401 @vladaman, I have tested this issue with a new charachters as "Şebnem Çakğır šččřž 안녕하세요감사합니다ΚαλημέραΕυχαριστώสวัสดีनमस्ते". And I realized that the issue exists only on listviews. If network tab is checked, parant_name is encoded incorrectly. I have checked the db as it is encoded correctly.

[chris001]

@yunusyerli1 Nice work. Can you double check, when you look at the ListView's HTML page source code (right click on list view, click View Page Source), is the field double-encoded, or single-encoded?