search

gltf-rs / gltf

A crate for loading glTF 2.0
Apache License 2.0
534 stars 124 forks source link

Determinism issue when serializing Primitive::attributes #336

Closed kirdaybov closed 1 year ago

kirdaybov commented 2 years ago

When serializing Gltf::document with gltf::json::serialize::to_vec we get different ordering on the attributes map of a primitive. This happens due to the dependence of the behavior of the hash function on the starting condition of the program. As a result, we get a nondeterministic result: "attributes":{"NORMAL":1,"TEXCOORD_0":2,"POSITION":0} vs "attributes":{"POSITION":0,"NORMAL":1,"TEXCOORD_0":2}

kirdaybov commented 2 years ago

Proposed fix is to switch to BTreeMap: https://github.com/gltf-rs/gltf/pull/337

alteous commented 2 years ago

Why is this an issue? A hash map is likely to have better memory locality than a binary tree.

kirdaybov commented 2 years ago

In my case I wanted to serialize the import result in our data pipeline (document + buffers + images) as a single blob. The reason is to have an almost raw data without the need for a virtual file system (since GLTF can have external links to textures or .bin file). The data pipeline expects data serialization to be deterministic.

The only thing that behaves non deterministically during serialization is the attributes. Since the amount of data stored in the attribute map is small (<10 I believe) the difference of impact on memory and performance between HashMap and BTreeMap shouldn't be significant.

kirdaybov commented 1 year ago

Closed with https://github.com/gltf-rs/gltf/pull/337