folbricht / desync

Alternative casync implementation
BSD 3-Clause "New" or "Revised" License
343 stars 44 forks source link

Failing sparsefile tests #262

Closed bearrito closed 8 months ago

bearrito commented 8 months ago

Sparsefile tests are flaky. There are some recent failing tests from CI with the same.

 sparse-file_test.go:103: 
            Error Trace:    /home/bstrausser/Git/desync/sparse-file_test.go:91
            Error:          Received unexpected error:
                            EOF
            Test:           TestSparseFileRead

The error originates at

func (h *SparseFileHandle) ReadAt(b []byte, offset int64) (int, error) {
    if err := h.sf.loader.loadRange(offset, int64(len(b))); err != nil {
        return 0, err
    }
        // return errs
    return h.file.ReadAt(b, offset)
}

Which calls code that looks like

               // This is the offending syscall
        m, e := f.pread(b, off)
        if e != nil {
            err = f.wrapErr("read", e)
            break
        }
        n += m
        b = b[m:]
        off += int64(m)
    }
    return

In the above the off(set) will always be equal to 2097152 when it throws.

I believe the issue is the random choice of start and offset

The test code looks like. The commented values are the value of vars on a run that throws.

// 2097152
index_length := int(index.Length())
//2089125
start := rand.Intn(index_length)

//12025
length := rand.Intn(int(index.Index.ChunkSizeMax))

fromSparse := make([]byte, length)
fromBlob := make([]byte, length)

_, err := h.ReadAt(fromSparse, int64(start))

This will cause us to attempt to read 12,025 bytes starting at 2,089,125.

There is a lot going on in creating a new sparse file, but I assume this is larger than the backing index file which has size 2,097,152