Closed gwillem closed 3 years ago
franzliedke
commented
6 years ago Please isolate which attribute is causing the problems, by removing them one by one from the serializer that is creating the JSON. Then, please let us know what is the type / charset of the relevant database column.
gwillem
commented
6 years ago I'm afraid my PHP skills are too limited to isolate the serializer. Could you give me a few pointers?
DB:
+--------------------------+----------------------+
| @@character_set_database | @@collation_database |
+--------------------------+----------------------+
| utf8mb4 | utf8mb4_general_ci |
+--------------------------+----------------------+
luceos
commented
6 years ago Feel free to post the full payload if you have it.
franzliedke
commented
6 years ago @gwillem We need the post / discussion data to be able to make any progress here.
gwillem
commented
6 years ago Can I share the DB dump somewhere privately? Update: credentials shared with @franzliedke on Discord.
franzliedke
commented
6 years ago Thank you @gwillem! I was able to reproduce the problem with the data you shared. Now for isolating the exact problem...
franzliedke
commented
6 years ago I have tracked this down to TextFormatter, hehe. Unfortunately, I haven't been able to reduce it down to a very short example snippet that would trigger the error as well. But I have a (very long) example post that does.
@JoshyPHP Would you be interested in this example post? I could share it with you privately (on Discord?) to debug on a Flarum instance?
franzliedke
commented
6 years ago (The problem appears when JSON encoding the post's content_html attribute (which is its content sent through TextFormatter).
Down in the depths of the zend-diactoros library, we serialize the JSON with json_encode($content_html, 79). That returns boolean false in this case. json_encode($content, 79) works, though, which makes me think the problem is something TextFormatter does to the data.
JoshyPHP
commented
6 years ago @franzliedke If you have the HTML, can you make a reduced case where json_encode() returns false and attach it here?
nowfalsalahudeen
commented
6 years ago change data type 'text' to 'long text' in content field of 'post' table (flarum db). this fixed my issue. The column used to be TEXT, which is limited to 65535 characters. but LONG TEXT does the job very well if u r make long text size post
stale[bot]
commented
3 years ago This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We do this to keep the amount of open issues to a manageable minimum. In any case, thanks for taking an interest in this software and contributing by opening the issue in the first place!
askvortsov1
commented
3 years ago Let's just catch errors when rendering posts, and display some error message instead that contains the post ID, and logs the error message to system logs? IMO, anything that could brick the forum should be avoided at all costs.
Explanation
A (Vietnamese) FreeFlarum user bricked (part of) their forum by submitting a specific post. As soon as the post is rendered in discussion or tag overview:
The problem is reproducible, however I cannot establish if there is invalid UTF-8. Python's JSON renderer does not have any problems with it. I also tried copy-pasting the post title & content from the database to another forum on the same server, but that did not yield an exception.
Technical details
Flarum info
Apparently it was reported by other users of UTF-8 languages:
https://forum.flarumchina.org/d/411/19 http://discuss.flarum.org.cn/d/3536
How to debug?