search

KhronosGroup / KTX-Specification

KTX file format source
Other
70 stars 12 forks source link

Format needs to be more directly seekable #32

Closed dewilkinson closed 5 years ago

dewilkinson commented 5 years ago

ISV feedback:

“The format is non-seekable. Excellent work that I need to read through half the file so I can start skipping to the bits I want. Point in case, I just want the lowest level mip-map. I read the file header, then I need another request to read the bytesOfDataFormatDescriptor worth of data (and the bytesOfKeyValueData), which I can then use to skip to bytesOfSupercompressionGlobalData, then I can skip over that. Which means I need two seeks and reads, or, if I’m doing HTTP, three requests assuming I don’t download the whole file right away (assuming a file with both supercompressed and non-supercompressed data is valid.)“

MarkCallow commented 5 years ago

Putting aside the hyperbole, I'm open to putting an index immediately after the header.

The miplevels are currently indexed. We could combine this and the new index at the front of the file. Since the number of miplevels is unknown until after the header has been read, the header must be read before the length of the index will be known. So 2 reads will be necessary to read in the complete index and a seek and a third read to get level data. If we don't fold in the level index, you'll need a read and a seek to find the level index, a second read to get the index and another seek and a third read to get the level data. So an index doesn't seem to provide any reduction in the number of reads & seeks.

The comments about "doing HTTP" suggest that you can do seeks and partial reads via HTTP. Is this true?

jherico commented 5 years ago

The comments about "doing HTTP" suggest that you can do seeks and partial reads via HTTP. Is this true?

@MarkCallow yes, HTTP supports range requests to fetch a subset of an asset. Our application already uses this with KTX 1 to fetch a header, and then incrementally fetch the mips

dewilkinson commented 5 years ago

Reviewed in 12/21 call

It was agreed that an index table block after the KTX2 header is a good solution to allow a reader to seek to any specific data block in the file

MarkCallow commented 5 years ago

I intend to fix this in the same change as issue #39 as they changes are closely related so I am closing this.

MarkCallow commented 5 years ago

Please review PR #53.