unformated text quoted with <<>> are lost in private messages

[tuxun]

symptom: write <<something>> to someone. it appears <> instead of <<something>>. "<<"">>" is commonly used as "guillemet". https://en.wikipedia.org/wiki/Guillemet

The input stand: raw input. this should stay raw cause its frustrating for newcomers to dont understand why their raw text isnt printed as raw text.

[tuxun]

ahahah sure the same problem here :/

[HankG]

I've done some testing on reproducing. This is the behavior confirmation status:

• Conversations Bodies: Yes
• Conversations Titles: No
• Posts Bodies: No
• Post Comments: No
• Poll Title: No
• Poll Answer: No

our standard "raw text" however is to use the back-ticks like here on GitHub. The convenience button for it has the "<>" but that's not what is used for it in the actual markdown.

[goobertron]

You have to use the appropriate syntax for the platform you're using. diaspora* uses Markdown, as does GitHub. As Hank points out, Markdown has its own simple encoding for raw text. There's a brief guide to Markdown syntax at https://diasporafoundation.org/formatting – and a more complete guide at https://daringfireball.net/projects/markdown/syntax

[tuxun]

i do, and i also use the broken preview button. (and wikipedia )

Guillemets are used pointing outwards («like this») to indicate speech in these languages and regions:

https://en.wikipedia.org/wiki/Guillemet#Uses

[goobertron]

Ah, I see. When you spoke about 'raw text' I thought, as did Hank, that you are looking for the syntax to encode raw, unformatted text (as you had done in your opening message) like this: raw text. The syntax for this is to use back-ticks.

What you actually want is to use guillemets as quotation marks in normal, formatted text. Is that right?

If so, which pod are you using? I've just tested this on two pods, and I get the same result (the desired one) no matter whether I use actual guillemets or two less-than/greater-than signs, and no matter whether or not I space them from the quoted text.

It is also fine in the project's Discourse instance, which I also just tested.

the broken preview button

In what way is it broken? It is working fine for me. Again, which pod are you using?

[tuxun]

https://framasphere.org a fr/uk/us/klingon pod. you maybe want to check the pod version against this "bug".

Given its a language specificity, i understand its not handled by default, but i think its a pretty damn break for newcomers.

In unformatted view we type <<something>> in preview it shows <<something>> and the final visible output is <>. on github at least it render in preview what it send (and its an english site so my fault here) (i havent checked the source... i think it has been done before cause... injection and escape of course)

[goobertron]

@Flaburgan, please could you check the exact behaviour on Framasphere, and look into any possible bug? Thanks.

[tuxun]

note i also linked this issue to our podmin, and the documention linked above specify that i have to escape any markdown characters if i want to send them unformatted. for me the real problem is that our language specificity make counter instinctive the use of our keyboard :/

[goobertron]

OK, now you have specified that this relates only to private messages (a.k.a. 'conversations') and only once the message has been sent, I have been able to confirm this.

It is only <<something>> (unspaced) that is parsed as <>; << something >>, «something» and « something » all appear as expected.

[HankG]

Did some research on this. Good news/bad news. Good news, it's not on ingestion processing of the conversation. Everything is stored as requested. It's not even deep in the controller. The problem is that when we render the messages (but not the message previews) we call the markdownified method in the HAML code. This is in the _message.html.haml and _message.mobile.haml. If I take out the markdownified call it renders correctly again. More to come...

[HankG]

A little more. The heavy lifting is in the MessageRenderer being done by the Redcarpet HTML render. It seems to be the filter_html markdownify step which is interpretting that as embedded HTML which we do not want. Be curious how we want to address this. What's really odd to me is that this doesn't seem to be a problem on the status message/posts just on the conversations yet this rendering artifact is coming server side for sure.

[tuxun]

Thanks everyone for digging into it :) are you reproducing this issue with the master branch? i don't know the version of my pod or even if it's a forked and changed one

[goobertron]

Framasphere currently runs version 0.7.7.1 of the code. You can check the code your pod is running at any time by looking in the side-bar or appending /nodeinfo/1.0 to your pod's domain – for Framasphere, this is https://framasphere.org/nodeinfo/1.0

[HankG]

@tuxun my code inspection and testing today was on the develop branch and when my initial testing was on my production account pod when it was running 0.7.7.1 (I think)

[tuxun]

well i think that's all for me here, may the force be with us and all opensource softwares :)

[DeusFigendi]

It seems to be the filter_html markdownify step which is interpretting that as embedded HTML which we do not want.

This is not entirely correct, per specification markdown allows just any HTML. But of corse nobody wants <script>-elements or <embed> or <object> or similar stuff entered by users. So what you really want is a subset of HTML-elements. For example I often use <sub> or <sup> if I want to write something small (or upper/lower) for attribution of an image or to tell what software I used to post a post or something like that. There are several other elemtents people use there are some elements diaspora* does not support but friendica does send because it is not defined which elements diaspora* supports so friendica must "think" any HTML is okay because it's part of markdown.

So I guess that the markdownify removes any HTML-tag/element it doesn't know, for example <something> or it is interpreted as HTML and the browser engine removes it (that is more likely if I think about it).

Which HTML elements are wanted and which ain't… should be discussed.

[ilatypov]

Same here in the post body with the mobile interface in 7.6.0, https://freehuman.fr/posts/1571162