bogovicj
commented
3 months ago in most cases you can just check for the 404 when you do a GET
You're probably right that we won't care in many cases, but the current design behaves differently if a block doesn't exist (return null) vs fails to read an existing block (throw an error). I think that optimizing in the way you suggest would prevent treating these cases differently (I don't think we do much different right now)
Note for later - Looks like these are logging block reads, so this is probably fundamentally coming from this code
// HEAD
if (!getKeyValueAccess().isFile(path))
return null;
// GET
try (final LockedChannel lockedChannel = getKeyValueAccess().lockForReading(path)) {
return DefaultBlockReader.readBlock(lockedChannel.newInputStream(), datasetAttributes, gridPosition);
} catch (final IOException | UncheckedIOException e) {
throw new N5IOException( "Failed to read block " + Arrays.toString(gridPosition) + " from dataset " + path, e);
}
krokicki
I noticed that the N5 Viewer seems to always send a HEAD before a GET. This is inefficient and may be unnecessary, since in most cases you can just check for the 404 when you do a GET. If this could be optimized away it would improve latency, while also reducing load on data servers.
Access log on my proxy server: