Discussion: Default behavior of line breaks

[Spangenhelm]

Hello everybody,

First of all thanks to @drsprite who pointed out the carriage return behavior, you can find the first discussion here: https://github.com/Jermolene/TiddlyWiki5/issues/275

I'm opening this one today to see if we can discuss about how the default behavior should be:

1) When typing regular text and you hit "return" key once it does nothing (like now in 5.0.7 version) you have to hit it twice to actually "jump" a line. You cannot do a simple carriage return without the use of paragraphs/code blocks/hard line breaks.

2) When you hit "return" key once it would do a simple carriage return like in any regular text editor without the need of paragraphs/code blocks/hard line breaks.

It seems like the same thing happened within <$list> it always leaves one empty line between two other (or is it a padding issue something like that?). Anyway it is taking more place than needed imo.

First we have to ask @Jermolene if it is even possible and then see if the majority of users is interested in changing default behavior or if they would rather leave it the way it is.

Thank you for your attention, +++ Spangenhelm +++

[pmario]

Having a propper paragraph handling, imo was one primary goal in TW5. TWclassic didn't produce <p> paragraph tags, which causes a lot of problems. eg: broken text formating on different screen sizes to name only one.

If you need hard linebreaks use:

"""
line with a hard
linebreak.
"""

---

The <$list> widgets creates "block" elements, if there is a single linebreak at the end of the first line. This caused problems with the created HTML code... Unwanted paragraphs inside the list. Paragraphs have a top- and bottom-margin, so there is too much space between the different elements, with the default CSS. ... IMO this should be fixed with 5.0.8. The code is allready pushed to the master.

[poblabs]

I suppose I'm of a different school of thought. If I type a paragraph, and that paragraph has a carriage return, I would expect it to fall onto the next line (like normal carriage return behavior).

For example, to save an email contact's signature information is quite confusing when it all bubbles up into one very long run-on sentence. Or even cataloging an entire email into Tiddly and it's just one long line. It can be confusing.

My workaround for having proper carriage return behavior is to wrap my wiki's into code ticks. Not ideal for non-code wiki entries, but it's manageable.

[Spangenhelm]

If I type a paragraph, and that paragraph has a carriage return, I would expect it to fall onto the next line (like normal carriage return behavior).

@drsprite Let's be as clear as we can: what do you mean by "paragraph" ? With or without any " ' ` ?

[poblabs]

@Spangenhelm Yes, sorry. My definition of paragraph in that reply is plain text. (e.g. just start typing). Without any sort of formatting ticks. Much like how I am typing this reply to you - I just start typing a textual entry with no formatting rules applied.

And if I hit carriage return. It treats it as such.

:smile:

Hope this helps clear my confusion.

[Spangenhelm]

Lol you are clear this time! Except the fact that there is a plain text option to treat the text differently but that's just to tease you!

Ok so to me if i am correct we think the same way and what we want is simple: 1) Open new empty tiddler without any "type" selected 2) Start typing text like in any other editor 3) And when we hit the "return" key it just goes to the next line without leaving one empty in between Is it ok ? I think i should take screenshosts it would be easier to explain i guess!

[Spangenhelm]

On the left what we want | On the right what is really happening: Note that type is empty !

---

With two "return" | Leaves an empty line that we do not want Note that type is empty !

[poblabs]

Ok so to me if i am correct we think the same way and what we want is simple: 1) Open new empty tiddler without any "type" selected 2) Start typing text like in any other editor 3) And when we hit the "return" key it just goes to the next line without leaving one empty in between

Yep, that sounds correct to me. I can appreciate the different needs to styling the wiki, which is why it's easy to get confused.

I was not aware of a "plain text" format. Would that require me wrapping my sentences within <p> stanzas? If so, that seems like too much work :wink:

[Spangenhelm]

Would that require me wrapping my sentences within <p> stanzas?

Not that i know of and since it changes the layout/style/display of text it is not what i want at least! If it is part of tw and made on purpose for compatibility reasons as @pmario pointed then i guess we are sc*****!

---

Edit: Also i have noticed that when we apply a "type" it applies to the whole tiddler which would prevent any mix between regular text and other i guess.

With "plain-text" type applied, note that it applies to the whole tiddler and prevent mixing text types

[poblabs]

Potentially. Like I said, my "fix" to keep the normal carriage returns was to wrap my plain text into code back ticks `. It's not ideal, but it works.

[Spangenhelm]

The best fix imo would be the one provided on the site and above via @pmario with triple " (paragraphs but in wikitext not html like you)

[Jermolene]

And if I hit carriage return. It treats it as such.

@drsprite GitHub actually uses a modified version of Markdown (see the link above the edit box where it says "GitHub Flavored Markdown"). As far as line breaks are concerned, it GFM still uses double line breaks to delineate HTML paragraphs that are wrapped in <p> tags. The additional rule is that single line breaks generate a <br> tag.

Reading the discussion, I think that both @Spangenhelm and @drsprite's needs would be met by adopting the additional GFM rule.

The main impact of the change would be to make it harder to copy and paste text from sources like emails that are plain text with fixed line lengths.

[Spangenhelm]

@Jermolene i'd love to try but first i need to find what to modify, can you just point us the right direction? would it be something like the "wikiparser" that we have to modify ? Thank you

[pmario]

+1 for GFM like line breaks.

So 1 line break would create a <br /> tag. 2 line breaks create a new paragraph.

see: https://groups.google.com/forum/?fromgroups=#!topic/tiddlywiki/IHtqnWQ75lk

[KolyaKorruptis]

+1

[Spangenhelm]

Any news about having gfm behavior in tw? The markdown plugin maybe?

[Jermolene]

Hi @Spangenhelm we've left things as they are, meaning that we have the same line break rules as classic Markdown, not GitHub Flavoured Markdown.

The Markdown parser in the Markdown plugin doesn't support GFM as far as I know:

https://github.com/evilstreak/markdown-js

[pmario]

@Jermolene

Any news about having gfm behavior in tw? The markdown plugin maybe?

I think the question was, if we get gfm linebreak handling with TW syntax. ... IMO the answer in one / several hangouts was no. Not possible anymore, because we have too many side effects with the <html> and paragraph handling.

[Jermolene]

@Spangenhelm @pmario

That's right, the main reason we decided against adding GFM line breaks to TW5 WikiText was that it was too hard to make it work with the other choices we had already made.

[pmario]

I think, we discussed it again in the hangout 60 and 63, which I didn't annotate yet :) ... There was a good explanation from Jeremy, why it won't work. ... So I'll post a link if I find it.

But my opinion is now, that the discussion actually is over. We have:

• 3 double quotes: """ see: http://tiddlywiki.com/#Hard%20Linebreaks%20in%20WikiText
• the html br tag: <br />
• code: with 3 backticks
• tiddler type: text/plain

So imo this ticket will be a wont-fix

[KolyaKorruptis]

In that case TW2 isn't really an option for me anymore. The current solution only allows for new paragraphs but not line breaks.

[Jermolene]

@Kolya33 do you mean TW5? If so, TW2 might actually suit you more, as it does implement hard line breaks rather than paragraphs.

Anyhow, I'd be interested to understand your usecase.

[KolyaKorruptis]

Yes, sorry I meant TW5. Our use case is a collaborative wiki in my company that I'm sharing with the department. I had brought up TW5, configured it etc and had to explain to at least a dozen people why you cannot simply make a linebreak (despite having out a tiddler with the existing solutions on the startpage). And of course I pointed out that a better solution would be coming...

In our experience the current solutions are simply not usable. If you press return you see a linebreak but it's not WYSIWYG as every forum, webmail and other online forms handle this case. Instead it behaves like HTML source code. That's not sth. most people have experience with or the patience to understand and learn. So despite the fact that I love TW5 I'll have a hard defending this behavior and we will probably switch to another solution.

[Jermolene]

Hi @Kolya33

I had brought up TW5, configured it etc and had to explain to at least a dozen people why you cannot simply make a linebreak

Perhaps the simplest approach for most users may be to tell them to always use double line breaks to end a paragraph. I guess it depends on the type of content being created.

Instead it behaves like HTML source code. That's not sth. most people have experience with or the patience to understand and learn.

TW5's behaviour was of course taken from Markdown, which appears to be pretty popular.

So despite the fact that I love TW5 I'll have a hard defending this behavior and we will probably switch to another solution.

Shame, I hope we can work something out.

[KolyaKorruptis]

Creating a new paragraph by hitting return 2 times is fine and expected. What's missing is a single linebreak. Just like you can do here on GitHub.

[pmario]

2 possibilities I see:

• use the tw2 parser plugin, that was not available as the discussion started.
  • It may introduce other styling problems, originating form the TWclassic syntax.
• We implement a GFM library plugin and
  • live with the limitations that come with markdown

[KolyaKorruptis]

I didn't know the classic parser was available for TW5, I'll check that out, thanks. https://github.com/buggyj/TiddlyWiki5/tree/tw2parser

[Spangenhelm]

@pmario

But my opinion is now, that the discussion actually is over. We have:

3 double quotes: """ the html br tag: br /> code: with 3 backticks tiddler type: text/plain

You are right we can use all of those but to me this is more like a "workaround" than "classic" behavior (by classic i mean things that usually works the same way everywhere since ever, and that we have taken the habit to use it like this, no matter if it is wrong or right) anyway those workarounds becomes really annoying when mixing datas within the same tiddler and it is getting worse when dealing with a lot of datas (like when copying-pasting text) then it just becomes unusuable (easily i mean), the issue here is that tw is working in "reverse" for new users -> we have to use "abnormals" tricks to use tw "normally". So to me it will have to change soon or later to return to a more logical use.

@Kolya33 Hi, do you know how i could have only single line break behavior, and how i could add it to my tw[v5.1.0]? tw2parser doesnt seem to fit my needs. Thank you

[pmario]

@Spangenhelm I don't know, what you mean with "datas". Do you have an example.

[pmario]

I found a very simple solution. ... Just create a stylesheet tiddler with this content.

p {
 white-space: pre-line;
}

Now a paragraph respects a single line break, how everyone expects. ...

@Jermolene The downside is, that it breaks the TW UI, since there are paragraphs scattered around everywhere. ... but IMO this is a bug, I did mention several times already.

[KolyaKorruptis]

p { white-space: pre-line; } .tc-sidebar-scrollable p { white-space: normal; }

[Jermolene]

Hi @pmario the problem with "white-space: pre-line" is that it changes the semantics of displaying linebreaks within paragraphs. Even if we removed the p tags from the sidebar, user content that included widgets within a paragraph would also end up with extraneous linebreaks.

The problem here is that there are two mutually exclusive approaches to line-break handling. We had to choose one, and the decision has been made to follow the lead of Markdown, on the basis of its popularity.

Having chosen one, we then implemented mechanisms to escape from the linebreak handling for special purposes, such as poetry. We can continue to add support for other scenarios where the linebreak handling needs to be different.

But I don't see a way that we can support both TWC-style linebreaks and Markdown-style linebreaks. We have to choose one or the other.

[KolyaKorruptis]

Serious question: How was the popularity of Markdown measured?

[Jermolene]

Serious question: How was the popularity of Markdown measured?

It's more a question of observation than measurement. Markdown is pretty ubiquitous on the web now. If you want a measurement, how's this:

http://www.google.com/trends/explore#q=markdown%2Ctiddlywiki

[KolyaKorruptis]

That might be a developer's filter bubble or a special pet. I've used millions of HTML forms on the web and Github and Stackoverflow are the only places I've ever seen offering Markdown parsing. Both are developer sites. If you wanted to go with popularity you'd actually have to use BBCode, which is far more popular as it's used by all the big internet forum softwares and hence by practically every forum out there. This is not a suggestion, just a note about the argument of popularity.

In the end I don't care about syntax much. The point is that TW breaks the principle of least astonishment by ignoring linebreaks. I press return and get the result as if I pressed space. On the left I see my linebreak and in the preview on the right it's gone. There is no sensible explanation for that if TW is supposed to be used by anyone but developers.

[pmario]

Hi @pmario the problem with "white-space: pre-line" is that it changes the semantics of displaying linebreaks within paragraphs. Even if we removed the p tags from the sidebar, user content that included widgets within a paragraph would also end up with extraneous linebreaks.

yes. but imo widgets should not use paragraphs to build DOM elements other than to display text. If paragraph is used for text only, the linebreak display can be defined by the user with CSS. And that's what the users want. ...

So we need to have a look, if widgets produce paragraphs to create UI elements. ... I think this is worth a closer look. ...

As we can see from the discussion. Adding aditional wiki syntax like eg: <br/> or \ at the end of the line, will not be accepted by users. I personaly think, hard linebreaks just produce problems, if you need to reuse your content. In the long run you waste time and money. But that's an other topic :)

[Jermolene]

Hi @Kolya33 I can see you've strongly held opinions here and I'm not particularly interested in trying to persuade you to different ones. The boat has sailed on TW5's default wikitext treatment of linebreaks. If that means that you can't use TW5 then that's useful feedback. If there's some backwards compatible change that will make things easier for you then I'll for exploring it.

[Spangenhelm]

Sorry @pmario it wasn't clear, i was obviously talking about mixing text (plain and wikified for example) Try opening a new tiddler with this (no text type selected -> default option):

"""
! This should be emphased
"""

! This is really emphased

This give us problems with triple ", triple ` and plain/text tiddler type thus we can only use wiki syntax with hard linebreak (
tag)

Thanks to you and @Kolya33 for your propositions i will try it and see if it fits my needs.

[FrinalFrantasy]

Hey, since you guys are welcoming feedback, I'd like to quick explain my usecase and how this affects me.

I have several TiddlyWikis (all TWc), totaling ~2000 individual tiddlers. I use them for all of my personal notes, including bookmarks, email drafts, to-do lists, part numbers, phone numbers, etc etc. At some point, I'd like to upgrade it all to TW5. I don't have strong feelings on the layout of TWc vs TW5; however, I want to be as future-proof as possible, and, I find TW5 easier than TWc to use on a smartphone.

The entire reason I'm delaying is TW5's handling of line breaks. Here are some examples of my tiddlers (one from my to-do list, and two from my quotes database):

The formatting of all of these completely breaks when imported into TW5. Since these are tiddlers made by me and for me, I don't adhere to any particular standard except whatever is most natural and parsable to myself. It wouldn't be very hard to fix them (such as by using [br]), but I would guess that about 75% of all my tiddlers need similar fixing. So at this point, I have resigned myself to having basically three options:

1. Manually go through every one of my tiddlers, check for single line breaks, and add a [br] before each one. Train myself to apply the same logic to every tiddler I add.
2. Install the TW2 parser plugin; manually go through every one of my tiddlers, set them all to text/x-tiddlywiki type, and hope that future TW5 functionality/plugins don't interfere with them. Modify my TW5 so that new tiddlers are automatically text/x-tiddlywiki.
3. Manually go through every one of my tiddlers, and re-write them such that they do not include single line breaks. Train myself to organize my thoughts in a way which does not include single line breaks.

1 is probably my best bet, and 1 and 2 can both probably be automated, if I find or learn to make a plugin that does so. But I hope you can understand that all three are pretty unappealing to me.

(All that said, my intention is not to complain or demand special help; TiddlyWiki is so helpful to me in my life that I don't really mind having to work around this issue. I only mention it since you guys value feedback.)

[hdevarajan]

I wanted to echo @Kolya33 and @CustooFintel's points, in particular @Kolya33's point about the treatment of linebreaks in tw5 going imho the wrong way on least astonishment.

I too have a few '000 tiddlers scattered across 10 or so tiddlywiki 2s, having gotten onboard since 2008. Through that time have converted several friends and co-workers to using this and have been eagerly anticipating the new version; also deeply appreciated the 25 year intention behind the vision. I have spent the afternoon trying to port to TW5 and ran into this thread.

The treatment of linebreaks imho is going to be a big issue during evangelization for the need to explain that to have single line breaks have to go into poetry mode.

I have followed the rationales involving markdown, and as merely a grateful consumer and not actually a contributor, have neither answers and certainly not requirements (indeed, as @CustooFintel has said, TW5 is clearly something that harks to finest traditions of open source work and esp in its long term vision). But have to say that single line break needs are so ubiquitous (with no bullets, indentations etc.) and so ubiquitously effortlessly supported in alternatives that supporting it in some natural way without having to go into explanations involving markdown will make TW5 more ready for prime time.

So quite sad to see the wontfix designation. That said, would be remiss if didn't record my thanks for the community of TW contributors, as use TW every day, so you guys have made a tremendous difference to intraday productivity.

[Jermolene]

Hi @hdevarajan

Just to be clear, marking this as wontfix makes it look like a fix is possible, but that we've chosen not to implement it.

That's not the situation. This is a cantfix -- the current situation is that we cannot see a way to introduce the desired behaviour without completely breaking a significant chunk of the existing wikitext syntax. We'd fix it if we could see a way to do so.

[hdevarajan]

That helps and thank you.

I see in another discussion that can use the TW2 parser in TW5 and if time will poke around using that. cheers

[carugnom]

Just watching these old posts... is really so hard to get TW5 to render linebreaks or newlines as expected ? I mean, I don't want to mess my text with triple quote blocks. I don't want to have to 'remember' to write that way because that parts are just simple text. It looks ugly. Also I can't mix it with other markup. Why just don't make newlines render as expected ? In Dokuwiki is what plugins like XBR do for example.

[danielo515]

Why just don't make newlines render as expected ?

That is your biased point of view. People used to markdown will expect a different behavior. Rules are rules, and they are never reasonable for everyone

[carugnom]

Well I like all these simple markup languages because the basic idea is to let the user write as few markup as posible. Of course it makes sense to use markup for headers, tables and other places where the text looks different. The only point I dislike is being forced to use markup to get newlines, because I already can make newlines in original the text. We see a number of people asking for this feature also, right ? So I think it would be good to have support for it.

2017-06-24 12:13 GMT-03:00 Daniel Rodríguez Rivero <notifications@github.com

:

Why just don't make newlines render as expected ?

That is your biased point of view. People used to markdown will expect a different behavior. Rules are rules, and they are never reasonable for everyone

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Jermolene/TiddlyWiki5/issues/443#issuecomment-310844214, or mute the thread https://github.com/notifications/unsubscribe-auth/ACnC9gYWMKSdQm9Fxm3V1PTkcQlQHOfsks5sHSeHgaJpZM4BlBOG .

[danielo515]

Well, is as simple as a custom parsing rule. Should be possible, but not for the default behavior

[carugnom]

Aha, good to know. I'll search for how to create such custom parsing rule. Thank you !

2017-06-25 11:17 GMT-03:00 Daniel Rodríguez Rivero <notifications@github.com

:

Well, is as simple as a custom parsing rule. Should be possible, but not for the default behavior

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Jermolene/TiddlyWiki5/issues/443#issuecomment-310905236, or mute the thread https://github.com/notifications/unsubscribe-auth/ACnC9nREZGOIEsqmXTc5xas0s4dU_-S7ks5sHmwKgaJpZM4BlBOG .

[pmario]

If you really want to use GFM, you should use the markdown-it plugin, that I created. It has a GFM mode, which should pretty much do, what you want, in terms of linebreaks.

The default setting is "commonmark" mode which uses "2 spaces at the end of the line to indicate a <br>". It's strange but would work too. see: 2nd paragraph in the "Test" tiddler.

It is still ALPHA, since there is no html sanitation yet. So copy/pasting html code isn't safe atm. But be aware, that markdown misses many features of the TW syntax.

---

Aha, good to know. I'll search for how to create such custom parsing rule. Thank you !

IMO it's not that simple. ...

• You don't want to use <br> which is 4 chars and does exactly what you want.
  • So it has to be less than 4 chars
• You don't want to use tripple-doublequotes .. aka "hard linebreak" block
  • It has to be less than 3 chars. eg: 2 or 1
  • so all those existing formatting chars are gone. Since they would create conflicts.
• There isn't much left. ... We used the nice combinations already ;)

[carugnom]

So.... the simple parsing rule is not so simple uh? I don't know how parsing rules work in tw5, but isn't it as easy as replacing \n with
? Any link to learn about parsing rules ?

2017-06-25 12:17 GMT-03:00 Mario Pietsch notifications@github.com:

If you really want to use GFM, you should use the markdown-it plugin https://wikilabs.github.io/editions/nightly/markdown-it.html, that I created. It has a GFM mode, which should pretty much do, what you want, in terms of linebreaks.

The default setting is "commonmark" mode which uses "2 spaces at the end of the line to indicate a ". It's strange but would work too. see: 2nd paragraph in the "Test" tiddler.

It is still ALPHA, since there is no html sanitation yet. So copy/pasting html code isn't safe atm. But be aware, that markdown misses many features of the TW syntax.

Aha, good to know. I'll search for how to create such custom parsing rule. Thank you !

IMO it's not that simple. ...

• You don't want to use
  which is 4 chars and does exactly what you want.
  • So it has to be less than 4 chars
• You don't want to use tripple-doublequotes .. aka "hard linebreak" block
  • It has to be less than 3 chars. eg: 2 or 1
  • so all those existing formatting chars http://tiddlywiki.com/#Transclusion%20in%20WikiText:%5B%5BTransclusion%20in%20WikiText%5D%5D%20%5B%5BTables%20in%20WikiText%5D%5D%20%5B%5BStyles%20and%20Classes%20in%20WikiText%5D%5D%20%5B%5BParagraphs%20in%20WikiText%5D%5D%20%5B%5BLinking%20in%20WikiText%5D%5D%20%5B%5BHard%20Linebreaks%20in%20WikiText%5D%5D%20%5B%5BDashes%20in%20WikiText%5D%5D%20%5B%5BFormatting%20in%20WikiText%5D%5D are gone. Since they would create conflicts.
• There isn't much left. ... We used the nice combinations already ;)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Jermolene/TiddlyWiki5/issues/443#issuecomment-310908576, or mute the thread https://github.com/notifications/unsubscribe-auth/ACnC9lg_TKXUhdkb1ekDkb8UY7jh0WKcks5sHnokgaJpZM4BlBOG .

[tobibeer]

The default setting is "commonmark" mode which uses "2 spaces at the end of the line to indicate a
". It's strange but would work too. see: 2nd paragraph in the "Test" tiddler.

Ha! I think it should be possible and would make sense to:

1. allow for a setting to have this apply to parsing wikitext, e.g. "transform trailing blanks more than X into linebreaks", e.g. defaulting to 1
2. to extend the parser to translate any trailing blanks at the end of a line into additional newlines <br>

That way, you can have it both ways:

1. you can use as many newlines in wikitext to separate concerns
2. but also control how many linebreaks are applied in the representation

[pmario]

commonmark uses exactly 2 spaces. ... and eats the rest. We shouldn't do different here.

Because of What if ...

I'd really like to get this going. ... even if we would break the existing TW syntax, which is my biggest concern. IMO mardown "did take over the world"