search

BinomialLLC / basis_universal

Basis Universal GPU Texture Codec
Apache License 2.0
2.7k stars 263 forks source link

build variations from number of cores #374

Open bmwiedemann opened 3 months ago

bmwiedemann commented 3 months ago

While working on reproducible builds for openSUSE (sponsored by the NLnet NGI0 fund), I found that our warzone2100 had files vary when build VMs had different CPU counts.

Here is an extract from the two build logs:

[ 5122s] [  3%] Generating base/texpages/bdrops/backdrop0.ktx2
[ 5122s] cd /home/abuild/rpmbuild/BUILD/warzone2100/build/data && ../3rdparty/basis_universal_host_tool/basis_install/bin/basisu -ktx2 -uastc -uastc_level 2 -uastc_rdo_l 1.0 -uastc_rdo_m -resample 2048 2048 -output_file /home/abuild/rpmbuild/BUILD/warzone2100/build/data/base/texpages/bdrops/backdrop0.ktx2 -file /home/abuild/rpmbuild/BUILD/warzone2100/data/base/texpages/bdrops/backdrop0.png
[ 5517s] Wrote output .basis/.ktx2 file "/home/abuild/rpmbuild/BUILD/warzone2100/build/data/base/texpages/bdrops/backdrop0.ktx2"
[ 5517s] Compression succeeded to file "/home/abuild/rpmbuild/BUILD/warzone2100/build/data/base/texpages/bdrops/backdrop0.ktx2" size 2128338 bytes in 395.214 secs
----
[ 5678s] [  0%] Generating base/texpages/bdrops/backdrop0.ktx2
[ 5678s] cd /home/abuild/rpmbuild/BUILD/warzone2100/build/data && ../3rdparty/basis_universal_host_tool/basis_install/bin/basisu -ktx2 -uastc -uastc_level 2 -uastc_rdo_l 1.0 -uastc_rdo_m -resample 2048 2048 -output_file /home/abuild/rpmbuild/BUILD/warzone2100/build/data/base/texpages/bdrops/backdrop0.ktx2 -file /home/abuild/rpmbuild/BUILD/warzone2100/data/base/texpages/bdrops/backdrop0.png
[ 5678s] Wrote output .basis/.ktx2 file "/home/abuild/rpmbuild/BUILD/warzone2100/build/data/base/texpages/bdrops/backdrop0.ktx2"
[ 5678s] Compression succeeded to file "/home/abuild/rpmbuild/BUILD/warzone2100/build/data/base/texpages/bdrops/backdrop0.ktx2" size 2151211 bytes in 5611.483 secs

Notice the different sizes in the log.

We had a similar issue in https://github.com/Warzone2100/warzone2100/issues/2991 before. There it helped to add -uastc_rdo_m but here it does not. I'd appreciate some patch or workaround to make build output deterministic.

richgel999 commented 1 month ago

We had a similar issue in Warzone2100/warzone2100#2991 before. There it helped to add -uastc_rdo_m but here it does not. I'd appreciate some patch or workaround to make build output deterministic.

Got it - perhaps limiting the max # of threads to some reasonable low value would help. I'll look into this.