xx3::Hash64 output is not stable

usrtax commented 2 years ago

the same val , but output different hash

shepmaster commented 2 years ago

Very surprising! Please provide runnable code (as text not a picture!) that reproduces the problem, including what version of the library is in use. It’s also useful to know what platform/cpu architecture you are on.

shepmaster commented 2 years ago

/cc @flier

jnicholls commented 1 year ago

Bump. Noticed no activity on this in a month, but pretty important to nail down imho. @usrtax can you provide the code as text? If not I can type it up. But at least, what cpu/arch you ran this on? Thanks.

cgbur commented 1 year ago

Also adding to this that I have experienced the same thing and came to the conclusion the xxh3 64 bit is unstable. Arch was a skylake xeon processor (aws c5 instance).

So theres not repeated work I am making a reproducible example right now

cgbur commented 1 year ago

Update: I am trying to make a minimum example that fails using version 1.6.2 because that is the version I first encountered this bug on. The bad news is that I cannot get it to elicit this behavior. The reason the bug was hard to track down for me is that when I first wrote the code in my project all my tests were passing. It was when somebody added code elsewhere in the codebase that all of the sudden my tests started to sporadically fail. This is worrying to me as it sounds like actual undefined behavior which changes based on compilation.

When I was experiencing the unstable output like the author I could get it to trigger with code as simple as:

    let mut hasher = twox_hash::xxh3::Hash64::default();
    hasher.write(&bytes);
    let first = hasher.finish();
    let mut hasher2 = twox_hash::xxh3::Hash64::default();
    hasher2.write(&bytes);
    let second = hasher2.finish();

Hopefully @usrtax still has the code and its open source? His repo https://github.com/usrtax/xxhash-bug is empty :(

usrtax commented 1 year ago

更新：我试图做一个使用版本 1.6.2 失败的最小示例，因为这是我第一次遇到这个错误的版本。坏消息是我无法让它引发这种行为。这个错误对我来说很难追踪的原因是，当我第一次在我的项目中编写代码时，我的所有测试都通过了。当有人在代码库的其他地方添加代码时，我的测试突然开始偶尔失败。这让我很担心，因为这听起来像是实际的未定义行为，它会根据编译而改变。

当我像作者一样遇到不稳定的输出时，我可以用简单的代码来触发它：
    let mut hasher = twox_hash::xxh3::Hash64::default();
    hasher.write(&bytes);
    let first = hasher.finish();
    let mut hasher2 = twox_hash::xxh3::Hash64::default();
    hasher2.write(&bytes);
    let second = hasher2.finish();
希望@usrtax仍然有代码及其开源吗？他的仓库https://github.com/usrtax/xxhash-bug是空的 :(

I feel it has something to do with the nightly compiler, and I can't reproduce this problem after I upgrade the compiler once.

I use xxhash-rust = {version="0.8.5",features=["xxh3"]} now

shepmaster / twox-hash

xx3::Hash64 output is not stable #92