search

coreweave / tensorizer

Module, Model, and Tensor Serialization/Deserialization
MIT License
153 stars 24 forks source link

[3.0] separate header hash from tensor hash #143

Open bchess opened 2 weeks ago

bchess commented 2 weeks ago

Eta0 last week We've previously discussed separating the header hash from the tensor hash, and I think it would be a good idea to add in v3. It makes it possible to edit the header after writing the tensor data without re-hashing the tensor, and it also makes the dataflow a bit simpler to not have to worry about the header being in a mostly-finalized state around the same time that the tensor is available (unencrypted) in memory.