Closed Eta0 closed 1 year ago
This change optimizes the serialization code to multithread and overlap hash computations, reduce redundant reads, and write to disk more efficiently.
This provides a ~2x speedup on serialization tests when serializing gpt-neo-2.7B.
gpt-neo-2.7B
Serialization Optimization
This change optimizes the serialization code to multithread and overlap hash computations, reduce redundant reads, and write to disk more efficiently.
This provides a ~2x speedup on serialization tests when serializing
gpt-neo-2.7B.