0xPolygonMiden / miden-vm

STARK-based virtual machine
MIT License
621 stars 158 forks source link

Refactor `MastForest` format to put all decorator data at the end #1489

Open plafer opened 3 weeks ago

plafer commented 3 weeks ago

_Originally posted by @bobbinth in https://github.com/0xPolygonMiden/miden-vm/pull/1482#discussion_r1751216993_

Not for this PR, but I wonder if there is a way to refactor this so that all decorator-related data is in one section. This way, if we wanted to drop decorators from the serialized binary, we could just truncate this section without doing any extra processing.

yasonk commented 1 week ago

I haven't looked into the serialization code in detail, but just to clarify, the intent is to make the serialization format look like this?

//! The serialization format of MastForest is as follows:
//!
//! (Metadata)
//! - MAGIC
//! - VERSION
//!
//! (lengths)
//! - nodes length (`usize`)
//!
//! (procedure roots)
//! - procedure roots (`Vec<MastNodeId>`)
//!
//! (raw data)
//! - Node data
//! - String table
//!
//! (info structs)
//! - MAST node infos (`Vec<MastNodeInfo>`)
//!
//! (All Decorator data)
//! - decorators length (`usize`)
//! - Decorator data
//! - decorator infos (`Vec<DecoratorInfo>`)
//! 
//! (before enter and after exit decorators)
//! - before enter decorators (`Vec<(MastNodeId, Vec<DecoratorId>)>`)
//! - after exit decorators (`Vec<(MastNodeId, Vec<DecoratorId>)>`)
bobbinth commented 1 week ago

I think it is a bit more complicated than that because some decorator data is also stored in the "Node data" section and also strings used by decorators are stored in the "String table" (though, I don't remember this too well now).

But the general idea is correct - we want to move everything decorator-related to the end of the file to make removing decorator data (which is going to be primarily used for debug purposes) as straight-forward as possible.