search

draem0507 / rietveld

Automatically exported from code.google.com/p/rietveld
Apache License 2.0
0 stars 0 forks source link

Files with mixed newline styles end up bad content #149

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Prepare a file with CRLF style EOF
2. Add an extra 'CR' at the end of a line. (This can be done by ^V^M in VIM)
3. Upload to code review server
4. Try side-by-side view of the file

What is the expected output? What do you see instead?
Expected:
Complete Side-by-side view of the file

But I got:
"Can't parse the patch" 
http://codereview.appspot.com/109116/diff/1/2
or if the line with mixed newline doesn't change, side-by-side view render 
partly and leave a 
"error: old chunk mismatch"
http://codereview.appspot.com/109116/diff/4/5

Please provide any additional information below.
The unified diff view is not effected. (With one extra blank line)
This might have been duplicated to issue37.

SVN diff reports error thus prevent the patch from being uploaded.
svn: File 'abc.txt' has inconsistent newlines
svn: Inconsistent line ending style

Git could generate diff and upload it.

The root cause is Git and Python(str.splitlines()) count "\r\r\n" as different 
number of lines
 $ git diff -r 4b0b79424 abc.txt 
diff --git a/abc.txt b/abc.txt
index 752787a..1e3f8fe 100644
--- a/abc.txt
+++ b/abc.txt
@@ -1,3 +1,3 @@
-hello
+hello^M
 ,
 world

$ python
Python 2.6.2 (r262:71600, Jul 23 2009, 00:41:09) 
[GCC 4.0.1 (Apple Inc. build 5478)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> "hello\r\r\n".splitlines()
['hello', '']

Original issue reported on code.google.com by lihaitao on 29 Aug 2009 at 1:49

GoogleCodeExporter commented 9 years ago 
You have the root cause right.  The question is, what to do about this?  You 
would
have to tell Python somehow that it should only treat CRLF as a line ending.  
(In
effect this is how the Microsoft C library interprets line endings in text 
mode.) 
This is a property of the file, so unless GIT sends along the fact that the diff
should be interpreted this way, I'm not sure how Rietveld can guess this.  Maybe
upload.py can request this info from GIT if it detects a CR anywhere in the 
file?

Original comment by gvanrossum@gmail.com on 31 Aug 2009 at 8:22

GoogleCodeExporter commented 9 years ago 
Could you provide a reproducable example? The issue links at the top are broken 
(404) and when I try 
to reproduce it locally the newlines are all normalized to '\n'. That's the 
diff I've uploaded:

[master] $ git diff
diff --git a/hello b/hello
index 6b9e844..7f65266 100644
--- a/hello
+++ b/hello
@@ -1,3 +1,3 @@
 foo
-hello
-bar
+hello^M
+bar^M
[master] $ 

The side-by-side view looks fine with this changeset.

Original comment by albrecht.andi on 1 Sep 2009 at 4:27

GoogleCodeExporter commented 9 years ago 
Strange, the links works fine for me. Can you try them again?

A file is attached with the first line end with '\r\r\n'. I can reproduce it on 
a
fresh built Git 1.5.6.5(Linux 2.6.16.60). 

To reproduce: (On Linux)
1. add this file to Git
2. change the second line
3. upload

And I can reproduced this issue on Mercurial.

Have you set git config core.autocrlf? But I can get it reproduced whether it 
is set
or not.

Original comment by lihaitao on 1 Sep 2009 at 8:46

Attachments:

GoogleCodeExporter commented 9 years ago 
Only operating systems "Commodore machines, Apple II family, Mac OS up to 
version 9
and OS-9" represents newline with CR, according to Wikipedia.
(http://en.wikipedia.org/wiki/Newline#Representations) If Rietveld isn't 
targeting
these OSs, maybe str.splitlines() can be replaced with "str.replace('\r\n',
'\n').split('\n')[:-1]"?

>>> "foo\r\r\nbar\r\n".replace('\r\n', '\n').split('\n')[:-1]
['foo\r', 'bar']
>>> "foo\r\nbar\n".replace('\r\n', '\n').split('\n')[:-1]
['foo', 'bar']

Original comment by lihaitao on 2 Sep 2009 at 6:55

GoogleCodeExporter commented 9 years ago

Original comment by albrecht.andi on 6 Apr 2012 at 7:12

GoogleCodeExporter commented 9 years ago 
This bothered me for so long, working an a multi-system team with SVN. I made 
this useful app, that navigates a folder with sub-folders looking for text 
files and converts all UNIX EOL to DOS EOL. It is an AIR app, works on both 
Windows and Mac. hope it helps

http://www.pippoflash.com/index.php/2012/06/11/svn-error-inconsistent-line-endin
g-style-nightmare-solved-download-app/

Filippo

Original comment by pippomu...@gmail.com on 12 Jun 2012 at 12:29