Turns out that reading the binary files byte by byte (or max 4 bytes) at a time is a terrible idea... We kinda only got away with it since files are really small. This is now changed to reading variable sized chunks at a time. Sometimes still really small chunks and sometimes bigger. Depends on context, usually a record sized chunk. Maps are for example read in 10-20 chunks and models in 2 chunks. Arguably reading these files (except perhaps the actual video files) at one go would still be faster. But still feels more correct way how the files are structured. And the difference would be way smaller now since we already went to only few reads instead of.... millions of reads?
RandomAccessFile wasn't really the one to blame. But it has been replaced with the newer NIO2 SeekableByteChannel. It works by spitting out chunks as ByteBuffer which we the wrap with our seasonings on top to allow easy conversion of read resources.
This will speed up the initial asset conversion. Reading maps is about 20 times faster (tested with Gonzalez map), for me it went from 400ms to 20ms.
Resolves #253
Turns out that reading the binary files byte by byte (or max 4 bytes) at a time is a terrible idea... We kinda only got away with it since files are really small. This is now changed to reading variable sized chunks at a time. Sometimes still really small chunks and sometimes bigger. Depends on context, usually a record sized chunk. Maps are for example read in 10-20 chunks and models in 2 chunks. Arguably reading these files (except perhaps the actual video files) at one go would still be faster. But still feels more correct way how the files are structured. And the difference would be way smaller now since we already went to only few reads instead of.... millions of reads?
RandomAccessFile wasn't really the one to blame. But it has been replaced with the newer NIO2 SeekableByteChannel. It works by spitting out chunks as ByteBuffer which we the wrap with our seasonings on top to allow easy conversion of read resources.
This will speed up the initial asset conversion. Reading maps is about 20 times faster (tested with Gonzalez map), for me it went from 400ms to 20ms.