Getting slower and growing errors after a while.

After copying data to a SDFS 0.9.1 filesystem for a while, the filesystem
contains suspiciously more data than the underlying file system, it has
gotten very slow, and the log is full of errors like:

May 3, 2010 11:05:06 AM org.opendedup.collections.CSByteArrayLongMap getData
INFO: found no data for key [4075096b1cea75677b91334ae24c659a2abb84a818d242e5]
May 3, 2010 11:05:06 AM org.opendedup.sdfs.filestore.HashStore getHashChunk
SEVERE: unable to get hash 4075096b1cea75677b91334ae24c659a2abb84a818d242e5
java.lang.NullPointerException
        at
org.opendedup.sdfs.filestore.HashStore.getHashChunk(HashStore.java:16
7)
        at
org.opendedup.sdfs.servers.HashChunkService.fetchChunk(HashChunkServi
ce.java:74)
        at
org.opendedup.sdfs.servers.HCServiceProxy.fetchChunk(HCServiceProxy.j
ava:188)
        at org.opendedup.sdfs.io.DedupChunk.getChunk(DedupChunk.java:113)
        at
org.opendedup.sdfs.io.ReadOnlyCacheBuffer.<init>(ReadOnlyCacheBuffer.java:41)
        at
org.opendedup.sdfs.io.SparseDedupFile.getReadBuffer(SparseDedupFile.java:394)
        at
org.opendedup.sdfs.io.DedupFileChannel.read(DedupFileChannel.java:354)
        at fuse.SDFS.SDFSFileSystem.read(SDFSFileSystem.java:335)
        at
fuse.Filesystem3ToFuseFSAdapter.read(Filesystem3ToFuseFSAdapter.java:298)
11:05:06.525 Thread-659988 ERROR [fuse.SDFS.SDFSFileSystem]: unable to read
file [filename]
java.io.IOException: unable to read at [0] because
[java.lang.NullPointerException]
        at
org.opendedup.sdfs.io.DedupFileChannel.read(DedupFileChannel.java:357)
        at fuse.SDFS.SDFSFileSystem.read(SDFSFileSystem.java:335)
        at
fuse.Filesystem3ToFuseFSAdapter.read(Filesystem3ToFuseFSAdapter.java:298)

First warning and error in the log (except for a gazillion of failed
symlinks) are:
Apr 26, 2010 2:59:01 PM org.opendedup.sdfs.filestore.OpenFileMonitor run
WARNING: Unable check files
java.lang.NullPointerException
        at
org.opendedup.sdfs.filestore.OpenFileMonitor.isFileStale(OpenFileMonitor.java:94
)
        at
org.opendedup.sdfs.filestore.OpenFileMonitor.run(OpenFileMonitor.java:58)
        at java.lang.Thread.run(Thread.java:717)

13:01:30.573 Thread-203629 ERROR [fuse.SDFS.SDFSFileSystem]: unable to read
file [filename]
java.io.IOException: unable to read at [0] because
[java.lang.NullPointerException]
        at
org.opendedup.sdfs.io.DedupFileChannel.read(DedupFileChannel.java:357
)
        at fuse.SDFS.SDFSFileSystem.read(SDFSFileSystem.java:335)
        at
fuse.Filesystem3ToFuseFSAdapter.read(Filesystem3ToFuseFSAdapter.java:
298)

The log is 622 MiB, and is growing fast.  About 1/4 is INFO lines about
non-existing files (star is stating files to see if the file should be
restored or not).

One java thread is cunsuming 100% CPU (one of four cores).  A few other
java threads gets some CPU now and then.
Original issue reported on code.google.com by stur...@gmail.com on 3 May 2010 at 9:32
tst2005googlecode / opendedup

Getting slower and growing errors after a while. #14
