search

keboola / php-utils

A set of utility functions
MIT License
0 stars 1 forks source link

Less aggressive utf16 striping #1

Open odinuv opened 6 years ago

odinuv commented 6 years ago

This code: https://github.com/keboola/php-utils/blob/master/src/Keboola/Utils/jsonDecode.php#L41 https://github.com/keboola/php-utils/blob/master/src/Keboola/Utils/stripInvalidUtf16.php#L12

removes even valid utf16 chars if an invalid one is encountered. This would be a better solution: https://mnaoumov.wordpress.com/2014/06/14/stripping-invalid-characters-from-utf-16-strings/

More info: https://keboola.slack.com/archives/C02JB1749/p1513775038000189

MiroCillik commented 6 years ago

I will look into.