Open rdb opened 4 years ago
Does it make sense to use the existing class handle of zlib compression to solve this issue?
I'm not entirely sure what you mean by "the existing class handle of zlib compression". We do already pull in the zlib library for other purposes in our C++ codebase, so we can use zlib functions if that's what you mean.
@rdb Thanks for confirmation. I would like to work on this.
I suspect our main concern for compression algorithms/libraries for BAM is decompression speed. There are better options for decompression speed than zlib (e.g., lz4 looks nice for decompression). However, zlib doesn't seem terrible for this, and it has the benefit that we already depend on zlib.
If/when someone gets around to a PR, benchmarks would be much appreciated so we can make sure we're not hurting BAM read performance with decompression.
I agree, it would be nice to see some comparisons of both size and decompression speed of the image data when (1) uncompressed, (2) compressed with zlib, and (3) compressed with a lossless image format like PNG.
If zlib proves to provide adequate compression speed without too much performance loss, we can make the zlib compression strength adjustable with a Config.prc value.
@ChinYing-Li You are welcome to work on this. Let me know if you need any assistance (or feel free to ask any questions on our Discord server in the #development channel).
... okay, I'll bite. Why a lossless format? If a .bam
is being created, the asset is being baked for distribution, and in most cases an appropriate GPU-preferred (lossy) compression format can be chosen. This is essentially what I'm doing now: packing a .txo.pz
+.bam
into a .mf
for single-file distribution. At the very least, having the option to use the existing (already compressed) texture stream would cover most use cases.
I can't speak for why @rdb mentioned a lossless format, but I can talk about why I prefer one. Panda should always preserve user data (especially in automatic operations). If a user is okay with losing information, they should explicitly say so. This could mean using an already lossy format (e.g., DDS), setting a flag on the texture, or using some form of global flag.
setting a flag on the texture
Yes, I would be fine with that - I'm already setting the compression format tag on my textures, I just need them to be written as-is to the .bam
s
At the moment, embedded texture data in .bam files is stored uncompressed. This makes these .bam files unnecessarily massive. It would be useful if Panda could use a lossless compression algorithm, such as PNG or ZLib, to store embedded textures.
We should pick a compression method that is quick to load, even if it's not perfect, since otherwise it would kill the benefits of model-cache-textures.