openSUSE / obs-build

OBS build script, can be used with OBS or stand alone
GNU General Public License v2.0
133 stars 188 forks source link

KVM: corrupted source files #1030

Open bmwiedemann opened 2 months ago

bmwiedemann commented 2 months ago

For my work on reproducible builds I am using osc build --vm-type=kvm a lot and noticed that the VM occasionally gets corrupted build input files on a on Leap 15.5 system.

For .spec .changes and .rpm (BuildRequires) the build fails early, but now that I started using pbuild I also found 3 cases where the .src.rpm silently contained the corrupted file (cpanspec.yml, rpmlintrc etc). The corrupted file had the same size but contained binary garbage (and build-compare would not even report a diff).

The 2nd stage should verify all checksums of build inputs before starting to build, so silent corruption gets avoided and a consistent build error message is produced.

This was with build-20240313-150500.446.1 from openSUSE:Tools but has been going on for a longer time.

bmwiedemann commented 2 months ago

I am now trying osc build --build-opt=--vm-use-mkfs-copyin as a possible workaround. Should have results next week.

adrianschroeter commented 2 months ago

so you are building as root? (otherwise mkfs-copyin would be default).

If there are corupted files without that mode I would verify if the umount did not happen for some reason before starting the VM. (As I can not really believe that the kernel is corrupting the block cache, but that would be another option).

bmwiedemann commented 2 months ago

Yes, that was building as root. I did not see this corruption with mkfs-copyin so far.