search

coreweave / tensorizer

Module, Model, and Tensor Serialization/Deserialization
MIT License
180 stars 25 forks source link

[3.0] Validate incremental bulk_write calls #149

Open bchess opened 3 months ago

bchess commented 3 months ago

Eta0 last week It used to be the case that a bulk write could still be incremental, and a series of bulk writes could be used to create a tensorizer file. Bulk write functions are also the only way to incrementally write structured data, namely through write_state_dict. As far as I can tell, the way the serialization code is written still allows that, so we should add some test cases for that here too.