search

MacDownApp / macdown

Open source Markdown editor for macOS.
https://macdown.uranusjr.com/
9.43k stars 1.09k forks source link

Paste puts wrong line endings in #329

Open Jmuccigr opened 9 years ago

Jmuccigr commented 9 years ago

I'm on a Mac OS X 10.9.5. To reproduce this:

  1. Open a text document in BBEdit. Make sure the line endings are set to "Unix (LF)".
  2. Copy some text from that document.
  3. Paste the text into a MacDown document.

Result: the line endings in the MacDown document will be carriage returns (^M if you look in something like less).

The expected result is that the line endings get pasted in as they were in BBEdit, that is, as LF.

One consequence of this is that, while the text looks ok in MacDown's rendering, it won't work in pandoc, which will not treat the CR like a line ending and thus all sorts of chaos ensue.

A solution is for MacDown to always paste in LF and never CR or CRLF. Alternatively the line endings could be converted upon saving/closing, I suppose.

I tried a few other apps and all of the text I pasted from them seemed to have the correct line endings. Interestingly TextEdit seems to do the same thing, if I paste text into it (that is, text from BBEdit has ^M), but as soon as I save it, the newly copied line endings show up without the CR.

(My own solution right now is to replace them myself with perl -pi -e 's/\r\n?/\n/g' filename.md.)

(I'm not sure exactly what the problem is here. Maybe MacDown is somehow not grabbing the right form of the text for pasting?)

michaelsproul commented 7 years ago

Just ran into this too

uranusjr commented 7 years ago

This is a bit weird. MacDown does not convert line endings when you paste (as far as I remember), so if it gets CR it gets them from your clipboard. Does BBEdit do some awkward line ending manipulation? I have no idea.

The solution seems reasonable though. Automatically convert between CRLF and LF is a common-seen text editor feature.

Jmuccigr commented 7 years ago

BBEdit seems to be smart enough to make the change either when it pastes or when it saves. You set the file's line endings and it just takes care of it from then on. So if I paste a bunch of lines with CR endings into a document that is set for LF (the default, I think and certainly what I use), then those line endings get converted to LF. I'm pretty sure that happens right away because the endings are preserved on pasting.

MacDown now seems to save everything with CR, no matter what they had originally. I tested this by copying out of BBEdit in the various formats and pasting into new MacDown documents and saving right away.

(It also saves as ascii unless there are non-ascii characters in the file. UTF-8 might be a better default these days.)

vlerenc commented 7 years ago

Unfortunately, when I save a file, it seems MacDown adds the CRs. This has nothing to do with paste for me. I had a markdown file (that I used to edit in vi), then after having saved it in MacDown I reopened it in vi and saw the ^M everywhere. Would be great to not write them or make it configurable. Thanks in advance.

Jmuccigr commented 7 years ago

Yeah, I suspect that the solution here is that MacDown converts everything to whatever the file is set to be, with a default setting in the prefs.

FranklinYu commented 7 years ago

@vlerenc Does it happen to a specific file, or to any file? Any way to reproduce it?

@Jmuccigr That solution sounds brutal for me…

vlerenc commented 7 years ago

@franklinyu Yes, I tried to narrow it down a bit. The reason I had the "impression" MacDown is putting the wrong line endings into the file is that my file consistently had 0d0a line endings already (not sure which program on the Mac added it when I edited the file in the past). Now, all programs that I used to edit the file preserved the line endings and when I added new lines, the new lines too got a 0d0a line ending, except for MacDown. MacDown only added a 0A line ending and what vi then does it to display all the 0d0a of the other lines, because after MacDown saves the file, the line endings are inconsistent.

So for me the solution is simple: my file was broken somewhen in the past by some crazy program/editor/whatever that created 0d0a line endings and I will fix this.

For MacDown it might be safer to check the line endings and if they are consistently 0d0a, use 0d0a for modified/new lines as well, I guess.

vlerenc commented 7 years ago

P.S.: Thank you for asking back. When trying to reproduce the problem, I saw it in the hex editor and wouldn't have checked otherwise.

FranklinYu commented 7 years ago

@vlerenc I think users may want different strategies/policies, but it is still a good idea to at least provide them options:

  1. Don't convert current. Insert LF (line feed). (current)
  2. Don't convert current. Insert LF or CR LF, whichever appears more in this document.
  3. Don't convert current. Insert CR LF if it's the only terminator/separator in this document; insert LF otherwise.
  4. Convert the document into pure LF or pure CR LF, whichever appears more. Then follow that terminator/separator.
Jmuccigr commented 7 years ago

How about:

  1. Make line-ending a preference and force all line endings to that.

  2. Make line-ending a by-document preference and force all line endings in the document to that.

I wouldn't complain about option 1, but I suspect others might disagree with my choice.

Is it recommended to have more than one line-ending in a document? If not, how would there be one that occurred more than another, except by pasting?

FranklinYu commented 7 years ago

@Jmuccigr a document with more than one line-ending is clearly broken, but if MacDown is ever converting all of them (which seems aggressive to me), at least I (as a user) would expect a notice. For example, a notice with buttons "Convert", "Cancel", "Always Convert for This File" looks good to me.

Jmuccigr commented 7 years ago

If there were an option to choose the line-endings, I'd expect the app to convert all of them to that without asking me.

A "don't convert" option could be added, but why would you need that? The line endings all have to be the same, right? So either they are all the same as the default when they get pasted in, or they're not, in which case MacDown would convert them.

I don't see the use case for leaving a mix of line endings in a file. Maybe if all the lines were pasted in and they were all in the non-default ending, MacDown could ask if you wanted to keep them that way, but how often is that going to happen? Almost inevitably someone is going to edit a file by doing something other than pasting in lines. So why write code for it? (And it seems like a pesky chore to have to do.)

FranklinYu commented 7 years ago

If there were an option to choose the line-endings, I'd expect the app to convert all of them to that without asking me.

That's reasonable only when you know that the option exists, typically not the case when a user upgrades to the latest version (let's say this feature lands in MacDown 0.7). I would rather have the option set to null by default, meaning "preference unknown", and prompt the user that we provide such functionality. If the user ever sets it to something, then sure, we just follow that; but before the user shows his opinion, converting by default seems a bit aggressive.

Jmuccigr commented 7 years ago

That's fine with me. It should also mean that at some point, the null option goes away.

niklasberglund commented 7 years ago

I would rather have the option set to null by default, meaning "preference unknown", and prompt the user that we provide such functionality.

A lot of users are probably less tech savvy and have no idea what line ending is though. This kind of users are under-represented here in GitHub issue discussions 😊

FranklinYu commented 7 years ago

@Jmuccigr The null option goes away as soon as a user choose one, since we know his choice by that moment.

@niklasberglund That's true, so for non-tech guys maybe the "converting all to Unix style" just works in macOS. I have no idea whether they prefer

  1. a pop-up telling the conversion with "Convert" as default button, or
  2. silently "fix" the file.
uranusjr commented 7 years ago

From my understanding a lot of MacDown users are indeed not tech-saavy. A sane default with (somehow obscure) configurability is the better way to approach this

Jmuccigr commented 6 years ago

Getting bit by this again (somehow). Are we anywhere on it?

(My specific problem came from some exported text from FMP which puts in CR for line endings. I replace them all with LF in the FMP script, and I'm good. This is still a problem in MacDown though.)

Jmuccigr commented 5 years ago

And again I got bit by this. Not sure where it came from this time, but it's annoying. Any chance to fix it?

DealPete commented 5 years ago

This still appears to be an issue. I just saved a file that uses Unix-style line endings and got ^M everywhere.

jamesquilty commented 5 years ago

I think this is an important issue to fix because inconsistent line termination can cause other programs to crash with cryptic errors.

The one that bit me hard over the last few days is https://github.com/markdownlint/markdownlint/issues/237. The fix to markdownlint is, in some respects, a workaround for the problem: some editors are creating files with inconsistent line endings.