On arm/android a zero initialized vec is not actually filled with zeros

Frando commented 1 year ago

Checklist

[X] I've looked through the issues and pull requests for similar reports

Describe your issue

Hi, when cross-compiling a crate we use (redb) to android we discovered a very weird issue with targets aarch64-linux-android and armv7-linux-androideabi. The original issue is here but I managed to create a minimal reproduction (see below).

The issue is: When repeatedly creating a vector filled with 0u32, it will happen that this vector's first element is not 0u32 but instead a different value. So basically, the TLDR is:

let data = vec![0u32; len];
assert!(data[0], 0u32)

should never fail the assertion, but it does on aarch64-linux-android with cross test.

We have been pondering this for a few days but are completely out of ideas how and why this happens. If the qemu VM runs out of memory or something, it should panic with an OOM, shouldn't it?

Example

cargo init

paste the following into src/lib.rs

#[cfg(test)]
mod tests {
    #[test]
    fn it_works() {
        let mut cache = vec![];
        let len = 8192;
        for round in 0..10000 {
            let data = vec![0u32; len];
            for i in 0..len {
                assert_eq!(data[i], 0u32, "data[{i}] is {} (at round {round})", data[i])
            }
            cache.push(data);
        }
    }
}

and run

cross test --target aarch64-linux-android

The test should pass (and does when testing directly on the host) - but the above command fails with this:

running 1 test
test tests::it_works ... FAILED

failures:

---- tests::it_works stdout ----
thread 'tests::it_works' panicked at 'assertion failed: `(left == right)`
  left: `8192`,
 right: `0`: data[0] is 8192 (at round 4097)', src/lib.rs:10:17

Additional information / notes

No response

What target(s) are you cross-compiling for?

aarch64-linux-android

Which operating system is the host (e.g computer cross is on) running?

[ ] macOS
[ ] Windows
[X] Linux / BSD
[ ] other OS (specify in description)

What architecture is the host?

[X] x86_64 / AMD64
[ ] arm32
[ ] arm64 (including Mac M1)

What container engine is cross using?

[X] docker
[ ] podman
[ ] other container engine (specify in description)

cross version

cross 0.2.5 (37c681a 2023-07-17)

Emilgardis commented 1 year ago

what qemu version are you using? I'll test this eventually myself, but would be good information to have (and also if maybe a later qemu has fixed it)

docker run --privileged --rm tonistiigi/binfmt:master --version

Frando commented 1 year ago

$ docker run --privileged --rm tonistiigi/binfmt:master --version
binfmt/82a943b qemu/v7.1.0 go/1.20.7

Emilgardis commented 1 year ago

Sorry, seems like I was mistaken in that binfmt showed the installed version... Can you do

qemu-system-x86_64 --version

instead :)

here's an example of mismatch

$ docker run --privileged --rm tonistiigi/binfmt:master --version
binfmt/82a943b qemu/v7.1.0 go/1.20.2

$ qemu-system-x86_64 --version
QEMU emulator version 8.0.0
Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers

Frando commented 1 year ago

Yes, I get the same as you:

$ docker run --privileged --rm tonistiigi/binfmt:master --version
binfmt/82a943b qemu/v7.1.0 go/1.20.7

$ qemu-system-x86_64 --version
QEMU emulator version 8.0.0
Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers

Frando commented 1 year ago

Hi @Emilgardis, do you have any further ideas or how we could go about identifying/fixing this issue?

cross-rs / cross