earth-mover / icechunk

Open-source, cloud-native transactional tensor storage engine
https://icechunk.io
Apache License 2.0
292 stars 17 forks source link

Node ids are random not sequential #319

Closed paraseba closed 4 weeks ago

paraseba commented 4 weeks ago

We want this to avoid conflicts in (at least) two scenarios:

This is not strictly needed, we could "renumber" the nodes as we merge them, but it would increase complexity.

The downside of this change: slightly bigger type for nodes, slightly more cloning.

WARNING: this is an on-disk format breaking change, Icechunk versions with this change cannot read repositories written with previous versions.