sshfs sometimes corrupts files

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Mount a disk with sshfs
2. Open text file in textmate
3. End of file is corruped 

Using macfuse 0.2.1 and sshfs 0.1

Original issue reported on code.google.com by marcus.r...@gmail.com on 19 Feb 2007 at 9:53

[GoogleCodeExporter]

What do you mean by "corrupted"? Can you give a specific example, and perhaps 
how to reproduce the issue?

Original comment by si...@gmail.com on 19 Feb 2007 at 11:09

[GoogleCodeExporter]

I realize that you did list the steps for reproducing the problem and I still 
asked you for information on how to 
reproduce the issue. Nevertheless, I do need to know what specific corruption 
you're seeing.

Original comment by si...@gmail.com on 19 Feb 2007 at 11:54

[GoogleCodeExporter]

I am experiencing the same issues.  The end of the file is getting cut off, and 
there is a buch of garbage in it's 
place.

As far as repoducing it.  It just seems random.  Sometimes the file will open 
correctly, other times it will not.

Original comment by bern...@gmail.com on 23 Feb 2007 at 6:47

[GoogleCodeExporter]

marcus.ramberg and berndtj:

I can't seem to make this happen on my machine, so let me ask you:

1. What is the exact sshfs command line you are using? Or are you using some 
kind of sshfs GUI front end?

2. What is the ssh server in question: a Linux machine? a Mac OS X machine? 
something else?

3. With the sshfs volume mounted, what is the output of the following command 
in the Terminal:

sysctl macfuse.version.string

4. Is TextMate configured to do "Atomic Saves" or otherwise?

5. How do you see that the end of the file is cut off? In textMate itself, or 
by doing a 'cat' etc. on the command 
line? If it's seen within TextMate, does 'cat' from the command line show the 
expected content for the file?

6. What is the general sequence of operations when you see this? Do you just 
start editing in TextMate and 
save the file inside the sshfs volume? Do you open an existing file and start 
editing it? Do you then insert text 
and save it, after which you see the corruption?

7. What is the ballpark size of the file when you start editing it? What is the 
size at the point you see 
corruption?

8. What do you mean by "bunch of garbage"? Non-ASCII characters? ASCII but 
unexpected characters? 
Something else?

Original comment by si...@gmail.com on 23 Feb 2007 at 7:53

[GoogleCodeExporter]

OK let me try and add some better info.

1) The sshfs command I'm using is:
sshfs ${host}.ironport.com: ~/SSHFS\ Volumes/${host} 
-oping_diskarb,volname=${host},workaround=rename

2) I'm connectig to freeBSD machines (different versions)

3) macfuse.version.string: 0.2.0, Feb 11 2007, 22:02:04

4) Textmate is not setup for atomic saves, but I don't think it is textmate 
related.  For instance I'm using sshfs 
with p4v (perforce GUI client).  When a file is checked out, the permissions 
are changed from read-only, to 
read-write.  This alone seems to corrupt the file, as the "revert if unchanged" 
does not revert.  Strangely, it 
seems that after reverting once, I can re-check out the file and there is no 
problem.  It seems to me that there 
may be read speed related.

5) I see the end of the file is cut off in TM or in TextEdit.  Although, I 
think the file size is maintained, as the 
end of the file is not just cut-off, there are just a bunch of non-ascii chars. 
 There are sometimes some other 
strings mixed in there, but it is always the tail of the file.  I have not 
tried cat'ing the file, but opening the file 
on the server under vim, shows the file uncorrupted.

6) The corruption happens at file open/read time.  I am in the habit of 
checking the file when I open it, to see 
if it is OK before I start editing.  These are all existing files, not new ones.

7) These are all small files ~20k

8) Yeah, just non-ascii chars, sometimes some ascii chars mixed in there.

If you have any other questions, let me know.

Thanks for all your work,

Berndt

Original comment by bern...@gmail.com on 24 Feb 2007 at 2:09

[GoogleCodeExporter]

berndtj:

Release 0.2.0 had a bug wherein if you specify 'volname', you incorrectly 
enable ACLs too. Just to rule out any 
strange issues related to that, can you please upgrade to the latest MacFUSE 
(0.2.1) and see if you still see this 
issue?

Original comment by si...@gmail.com on 24 Feb 2007 at 6:19

[GoogleCodeExporter]

berndtj:

Never mind. I think I found what the problem might be. Thanks for your help. 
I'll fix this in the next release.

Original comment by si...@gmail.com on 24 Feb 2007 at 8:55

[GoogleCodeExporter]

Fixed in 0.2.2.

Original comment by si...@gmail.com on 24 Feb 2007 at 9:20

• Changed state: Fixed

[GoogleCodeExporter]

Thank you!

Original comment by bern...@gmail.com on 25 Feb 2007 at 6:44

[GoogleCodeExporter]

This issue is not totally fixed.

The tail of files is still either cut off, or there are some non-asci chars 
attached at the end.  The number of 
chars is significantly lower.

However, as I indicated before, when opening a file through perfoce (P4V), the 
file would appear edited, upon 
opening (if it did not open cleanly).  Now, even if there is some garbage at 
the end of the file, perforce does 
not think the file has been edited, that's an improvement.  Also, opening the 
same file in vim on the server, 
shows that the file is indeed still in tact.

It looks to me as if the end of file marker is not getting written consisently.

Original comment by bern...@gmail.com on 5 Mar 2007 at 5:42

[GoogleCodeExporter]

Original comment by si...@gmail.com on 6 Apr 2007 at 3:38

• Changed state: New

[GoogleCodeExporter]

berndtj: I reopened this issue based on your feedback. I can't reproduce it 
using any of the tests that I tried 
earlier--do you have a way to deterministically reproduce it with the current 
MacFUSE release? Are you still using 
the same sshfs arguments are before?

As for the "end of file marker", that's just a conceptual thing--the file 
system doesn't write any such thing to 
storage (although applications could if they want).

Original comment by si...@gmail.com on 7 Apr 2007 at 7:23

[GoogleCodeExporter]

I was experiencing similar problems, until I realized I hadn't updated sshfs 
since v0.1. Since I updated to 
MacFUSE & SSHFS 0.3, the problem seems to have disappeared.

Original comment by renaud.d...@gmail.com on 9 May 2007 at 1:42

[GoogleCodeExporter]

No further feedback received and I still haven't seen this happen myself. Until 
information to the contrary is 
received again, marking this as fixed. If somebody runs into this still, just 
open a new ticket.

Original comment by si...@gmail.com on 11 May 2007 at 7:49

• Changed state: Fixed

[GoogleCodeExporter]

For future reference only
(not suggesting that this MacFUSE issue 103 be re-opened):

I sense a _possible_ similarity between some symptoms here 
and MacFusion issue one hundred and seven 
<http://code.google.com/p/macfusion/issues/detail?id=107> 
and to a lesser degree, MacFusion issue one hundred and eight 
<http://code.google.com/p/macfusion/issues/detail?id=108>.

The test(s) that gave rise to those two issue reports 
(neither of which is conclusively a defect) were extraordinary, I was using 
combinations of sshfs and AFP in 
most unusual ways, but the interesting thing is that I encountered an 
unexpected end of file. 

Original comment by grahampe...@gmail.com on 16 May 2007 at 11:53

[GoogleCodeExporter]

For reference only 
(not necessarily for consideration by MacFUSE developers): 
EncFS.plugin ticket 3 
<http://dev.systemred.net/trac/encfs/ticket/3>.

Original comment by grahampe...@gmail.com on 28 May 2007 at 4:14