search

panda3d / panda3d

Powerful, mature open-source cross-platform game engine for Python and C++, developed by Disney and CMU
https://www.panda3d.org/
Other
4.44k stars 777 forks source link

Feature request: Compressing embedded texture data in .bam #745

Open rdb opened 4 years ago

rdb commented 4 years ago

At the moment, embedded texture data in .bam files is stored uncompressed. This makes these .bam files unnecessarily massive. It would be useful if Panda could use a lossless compression algorithm, such as PNG or ZLib, to store embedded textures.

We should pick a compression method that is quick to load, even if it's not perfect, since otherwise it would kill the benefits of model-cache-textures.

ChinYing-Li commented 4 years ago

Does it make sense to use the existing class handle of zlib compression to solve this issue?

rdb commented 4 years ago

I'm not entirely sure what you mean by "the existing class handle of zlib compression". We do already pull in the zlib library for other purposes in our C++ codebase, so we can use zlib functions if that's what you mean.

ChinYing-Li commented 4 years ago

@rdb Thanks for confirmation. I would like to work on this.

Moguri commented 4 years ago

I suspect our main concern for compression algorithms/libraries for BAM is decompression speed. There are better options for decompression speed than zlib (e.g., lz4 looks nice for decompression). However, zlib doesn't seem terrible for this, and it has the benefit that we already depend on zlib.

If/when someone gets around to a PR, benchmarks would be much appreciated so we can make sure we're not hurting BAM read performance with decompression.

rdb commented 4 years ago

I agree, it would be nice to see some comparisons of both size and decompression speed of the image data when (1) uncompressed, (2) compressed with zlib, and (3) compressed with a lossless image format like PNG.

If zlib proves to provide adequate compression speed without too much performance loss, we can make the zlib compression strength adjustable with a Config.prc value.

@ChinYing-Li You are welcome to work on this. Let me know if you need any assistance (or feel free to ask any questions on our Discord server in the #development channel).

Clockwork-Muse commented 2 years ago

... okay, I'll bite. Why a lossless format? If a .bam is being created, the asset is being baked for distribution, and in most cases an appropriate GPU-preferred (lossy) compression format can be chosen. This is essentially what I'm doing now: packing a .txo.pz +.bam into a .mf for single-file distribution. At the very least, having the option to use the existing (already compressed) texture stream would cover most use cases.

Moguri commented 2 years ago

I can't speak for why @rdb mentioned a lossless format, but I can talk about why I prefer one. Panda should always preserve user data (especially in automatic operations). If a user is okay with losing information, they should explicitly say so. This could mean using an already lossy format (e.g., DDS), setting a flag on the texture, or using some form of global flag.

Clockwork-Muse commented 2 years ago

setting a flag on the texture

Yes, I would be fine with that - I'm already setting the compression format tag on my textures, I just need them to be written as-is to the .bams