NTFS-3g slow write-performance

[GoogleCodeExporter]

What steps will reproduce the problem?
Copying data from external eSATA HDD HFS+ to NTFS 3.1 on PCIe-HW RAID5
partition mounted using NTFS-3g 20070118 and MacFUSE-Core 0.1.9 yields very
bad performance (about 2,3MByte/s). Copying to a HFS+ partition on same
RAID device yields about 34MByte/s. CPU load is only 12% on a Core 2 Duo,
4GB RAM, OS X86 10.4.8.

NTFS-3g developers suggest a MacFUSE problem:
http://forum.ntfs-3g.org/viewtopic.php?t=162

Best Regards,
   Felix Buenemann

Original issue reported on code.google.com by Felix.Bu...@gmail.com on 29 Jan 2007 at 5:11

[GoogleCodeExporter]

I get approx 900 KBps, while writing to NTFS drives. Source is irrespective. 
Also
this is on OSX 10.4.8 with MacFuse 0.1.7 and NTFS-3g 20070118

Original comment by nitants...@gmail.com on 4 Feb 2007 at 6:17

[GoogleCodeExporter]

Here's what happens to me. I've written this in NTFS-3G forums, too, FYI.

I've seen this problem with both MacFUSE 0.1.9 + NTFS-3G 20070118 and
MacFUSE 0.2.1 + NTFS-3G 20070207.

The write speed to the NTFS partition is pretty slow. It seems to be around
700 to 900KB/s after several measurements when copying single file. Note
that it's an external hard disk that had sustained copying speed of 15MB/s
when using HFS+.

Then when I attempt copying of two (or more) files, as in selecting two files
in Finder and dragging them to NTFS volume, the speed drops to around
200KB/s.

Then it stays there. It'll retain that speed even if I copy a single file again.
I've tried unmounting / mounting, but it doesn't make any difference.

I wonder what's going on?

Original comment by wesle...@gmail.com on 17 Feb 2007 at 12:23

[GoogleCodeExporter]

What's going on:

ntfs-3g accesses your disk--the block device I mean--by opening it from user 
space and doing reads/writes 
on it. Mac OS X does not have a buffer cache for block devices when they are 
accessed like this, which makes 
things instantly super slow. That's the biggest factor. So this is largely a 
platform difference--not really a 
MacFUSE issue. ntfs-3g's I/O patterns (lots of random reads/writes) only make 
it much worse, because the I/O 
can't be coalesced, etc.--it's typically much faster to do large contiguous 
writes than lots of small random 
writes.

You can see the difference by creating a disk image, formatting it as NTFS, and 
then performing writes to it--
should give you the expected performance. You can see another aspect of the 
difference by using the 
"xmp" (pass-through) MacFUSE file system (available in the examples/ 
subdirectory of the user-space FUSE 
library), which will remount your root volume so that you can optionally go 
through MacFUSE for all I/O to the 
root volume. This will give you some ballpark idea of the type of overhead 
MacFUSE itself has.

Original comment by si...@gmail.com on 17 Feb 2007 at 12:51

[GoogleCodeExporter]

I wasn't aware Mac OS X doesn't have buffer cache either for block devices. 
People
indeed report ntfs-3g being at least 20-40x faster on Linux (which do have) 
than on
Mac OS X. 

FreeBSD has the same problem, though the latest branch added the cache, afaik 
(which
will probably arrive to Mac OS X as well, at some point). Csaba Henk, author of 
FUSE
for FreeBSD, developed a user space buffer cache library and made ntfs-3g 
patches to
use it. The patch should be fairly portable and improve performance drastically 
on
Mac OS X, in theory. It wasn't officially integrated yet for some reasons, for
instance because it decreased the performance on Linux by 2-8% which otherwise 
is in
line with the performance of the in-kernel file system drivers IO wise. For
interested parties, the ublio patches are available from
http://comments.gmane.org/gmane.comp.file-systems.ntfs-3g.devel/2

Original comment by sz...@ntfs-3g.org on 17 Feb 2007 at 9:13

[GoogleCodeExporter]

> FreeBSD has the same problem, though the latest branch added the cache, afaik 
(which
> will probably arrive to Mac OS X as well, at some point).

It's not necessarily a "problem" -- it's just a difference, at worst a feature 
that's not there on Mac OS X. So far, 
nobody, including Apple, has needed this, so nobody missed its absence. Also, 
just because a feature appears in 
FreeBSD doesnt' mean it is likely to appear in Mac OS X -- Apple tends to be 
quite selective in what they "borrow" 
from FreeBSD.

Do you guys plan to roll the ublio patch into ntfs-3g sometime in the near 
future?

Original comment by si...@gmail.com on 17 Feb 2007 at 10:43

[GoogleCodeExporter]

I've added a FAQ item for this:

http://code.google.com/p/macfuse/wiki/FAQ

Original comment by si...@gmail.com on 20 Feb 2007 at 7:19

[GoogleCodeExporter]

It's nice to see a FAQ on this issue, but is this going to be something that'll 
never
go away as long as OS X's way of handling the block device stays like it is 
right now?

Original comment by wesle...@gmail.com on 23 Feb 2007 at 12:54

[GoogleCodeExporter]

The ublio work may already solved the Mac OS X performance problem. Nobody 
tried it yet.

The code will be integrated as soon as somebody does the integration work, 
testing,
proves it's useful on Mac OS X, takes responsibility for the Mac OS X port and 
the
patch has no side-effect performance and reliability wise on Linux (e.g. it 
should be
optional). At the moment I'm not aware of anybody who would undertake this. It's
still an open position ;-)

Original comment by sz...@ntfs-3g.org on 24 Feb 2007 at 4:09

[GoogleCodeExporter]

Marking as "invalid" because this can't be fixed in MacFUSE itself.

Original comment by si...@gmail.com on 28 Mar 2007 at 3:25

• Changed state: Invalid