Hide jemalloc warnings

[mcandre]

Checklist

• [X] I've looked through the issues and pull requests for similar request
• [ ] This feature could be solved with a custom docker image (optional)

Describe your request

Can we please hide the jemalloc warnings cluttering the logs?

Describe why this would be a good inclusion for cross

That way, it will be easier to see logs more pertinent to the user's code. Especially when generating binaries for several different target triples.

[Emilgardis]

What warnings are you talking about? Please provide an example.

Do you mean this?

<jemalloc>: (This is the expected behaviour if you are running under QEMU)
<jemalloc>: MADV_DONTNEED does not work (memset will be used instead)

it's nothing controlled by cross, but rather jemalloc warning that when running under qemu there's broken behaviour, signaling MADV_DONTNEED doesn't work properly with jemalloc under qemu.

It's defined here and I'm not aware of a way to turn it off https://github.com/jemalloc/jemalloc/blob/31e01a98f159926493158cde6453cde55f21c42b/src/pages.c#L800-L801

if you're running on real hardware this warning won't happen

[Emilgardis]

Seems like

[build.env]
passthrough = ["MALLOC_CONF=trust_madvise:true"]

does the trick, however, this will most likely bring bad behaviour

[mcandre]

Intersting.

So this isn't necessary a cross-specific issue, but rather one which cargo and/or rustc could help fix, right?

[Emilgardis]

I supposed. But the fix done in cargo/rustc would be for them to not use jemalloc in distributed binaries, or for qemu to adjust/fix this behaviour. But, I think the warning is valid, as a normal user would not run under qemu and the developer should know something abnormal is happening

edit: the problem is qemu, not jemalloc, see https://github.com/jemalloc/jemalloc/commit/a943172b732e65da34a19469f31cd3ec70cf05b0

[Emilgardis]

It may actually be fixed with https://github.com/qemu/qemu/commit/892a4f6a750abceeda4c1ee8324f58f82fd6bd89 on 7.2.0, I'm not able to test that right now though

[geelen]

I've just installed cross from main and using it for the first time and seeing these warnings. Did that qemu release go out? Can cross be updated to use it?

[Emilgardis]

it's not something done by cross, qemu is installed on the host and used by docker

[geelen]

Ah, thanks for the quick reply. I'll look into how to get docker for mac using the latest qemu, cheers.

[Emilgardis]

the easiest way to install it is probably using https://github.com/tonistiigi/binfmt/pull/120

[ShaneMurphy2]

@geelen Did you find a way?

[Emilgardis]

for the record, I think I was wrong in how qemu is used, I believe it's actually using the one in the container, I thought that wasn't the case. We install qemu here

[ShaneMurphy2]

for the record, I think I was wrong in how qemu is used, I believe it's actually using the one in the container, I thought that wasn't the case. We install qemu here

Is that using QEMU 5.x?

[ShaneMurphy2]

for the record, I think I was wrong in how qemu is used, I believe it's actually using the one in the container, I thought that wasn't the case. We install qemu here

Isn't this the QEMU docker uses? The cross image is based off and amd64 ubuntu image if I understand correctly.

[Emilgardis]

Im not totally sure how qemu is eventually used I'll investigate more, but maybe you're correct and I was assuming correctly

[Emilgardis]

It actually does seem like this is solved with newer qemu versions!

update with

docker run --privileged --pull always --rm tonistiigi/binfmt:qemu-v8.0.4 -uninstall amd64 -install amd64

let me know if it works

[ShaneMurphy2]

Im not totally sure how qemu is eventually used I'll investigate more, but maybe you're correct and I was assuming correctly

I think it is the QEMU used by docker, since the warnings are being emitted when running cargo build in the container. Unless I'm mistaken, the QEMU installed by cross is used for running cargo test.

[ShaneMurphy2]

It actually does seem like this is solved with newer qemu versions!

update with

docker run --privileged --pull always --rm tonistiigi/binfmt:qemu-v8.0.4 -uninstall amd64 -install amd64

let me know if it works

Oh, does docker automatically use that container for emulation?

[ShaneMurphy2]

Also that didn't seem to solve it for me.

[Emilgardis]

no, it does some magic to install qemu for docker :)

[Emilgardis]

Also that didn't seem to solve it for me.

I get absolutely zero MADV_DONTNEED warnings when running from a clean slate with cargo clean using 8.0.4, using 7.0.0 I get many

[ShaneMurphy2]

Also that didn't seem to solve it for me.

I get absolutely zero MADV_DONTNEED warnings when running from a clean slate with cargo clean using 8.0.4, using 7.0.0 I get many

Hm, so I should: run docker run --privileged --pull always --rm tonistiigi/binfmt:qemu-v8.0.4 -uninstall amd64 -install amd64 restart docker? install cross: cargo install cross --git https://github.com/cross-rs/cross create an app: cargo new arm && cd arm run the build: cross build --target aarch64-unknow-linux-gnu

Edit: Doing the above, plus restarting my laptop for good measure fixed it! Not quite sure the mechanics, as to where the installed qemu actually is, or how docker uses it but I'm happy.

[Emilgardis]

I didn't have to restart docker. Not sure why its working for me and not for you :/

I have enabled beta feature rosetta in docker, maybe that also plays into it?

[ShaneMurphy2]

Spoke too soon, aarch64-linux-gnu-gcc segfaults when running the build for openssl-sys. At least the MADV warnings are gone 😅

[Emilgardis]

thats https://github.com/cross-rs/cross/issues/1097

[Emilgardis]

Some more usage, sometimes I still get the warnings, sometimes no, not sure whats differing

[ShaneMurphy2]

Some more usage, sometimes I still get the warnings, sometimes no, not sure whats differing

For me to get it consistent I deleted any other versions of binfmt through the docker desktop UI.