intelligent-machine-learning / glake

GLake: optimizing GPU memory management and IO transmission.

Apache License 2.0

351 stars 32 forks source link

[Roadmap] GLake checklist #14

Open ruizhang1230 opened 7 months ago

ruizhang1230 commented 7 months ago

Training

[ ] Multi-stream Memory Reuse: Done, will be released
[ ] Compatible with Expandable Segment
[ ] Memory Pattern Profiling tool
[ ] DoubleOverlapping(for finetune): Done, will be released
[ ] Multipath with Specific Scenario
[ ] Compression (Lossless/Lossy for finetune)

Inference

[ ] LLM KV Cache optimization: Almost Done, will be released
[ ] MoE Inference Optmization
[ ] Other Optimization for Specific Scenario (not fragmentation)

Refactor

[ ] Support TensorFlow
[ ] Support ONNX RUNTIME