enkore
commented
8 years ago I think the culprit is to be found in _chunker.c. There are two code paths, one works on native FDs, one uses the Python file-like interface[1].
// chunker_fill
// if we have a os-level file descriptor, use os-level API
n = read(c->fh, c->data + c->position + c->remaining, n);
if(n > 0) {
c->remaining += n;
c->bytes_read += n;
}
else
if(n == 0) {
c->eof = 1;
}
else {
// some error happened
return 0;
}I never seen what read() returns on bad sectors, but I suppose EIO. So chunker_fill returns 0 on error, chunker_process, which uses chunker_fill, checks that case and returns NULL (for a PyObject return value -- I suppose this is converted to None). So this NULL/None should bubble up to borg.archive.Archive.process_file, line 555:
with os.fdopen(fh, 'rb') as fd:
chunks = []
for chunk in self.chunker.chunkify(fd, fh):
chunks.append(cache.add_chunk(self.key.id_hash(chunk), chunk, self.stats))
if self.show_progress:
self.stats.show_progress(item=item, dt=0.2)
cache.memorize_file(path_hash, st, [c[0] for c in chunks])
status = status or 'M' # regular file, modified (if not 'A' already)This should explode if chunk where None. So I guess something doesn't quite work out here. Going back a bit, what does happen with the NULL PyObject*?
[1]
// no os-level file descriptor, use Python file object API
data = PyObject_CallMethod(c->fd, "read", "i", n);
if(!data) {
return 0;
}If read raises an exception it should bubble up the stack just like any other exception.
RonnyPfannschmidt
seambeat
ThomasWaldmann
When creating a new archive with files read from a disk with some bad sectors, Borg truncates files that are not fully readable and includes them into the archive. There is no error message or a warning, the return code is 0.
Seems like Borg stops reading files on an I/O error and continues as if they were fully and correctly read. The result is an "successfully" created archive containing demaged files.
The same behavior can be reproduced using a fake block device instead of a disk with device mapper's error target on it.