Paper Reading

MLSys

DL Compiler

Survey

• [x] The Deep Learning Compiler: A Comprehensive Survey: [Paper] [Note]

Graph Optimization

• [x] Rammer: Enabling Holistic Deep Learning Compiler Optimizations with rTasks[OSDI'20]: [Paper] [Note]
• [ ] Pet: Optimizing Tensor Programs with Partially Equivalent Transformations and Automated Corrections[OSDI'21]: [Paper]

Kernel Fusion

• [x] AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures[ASPLOS'22]: [Paper] [Note]
• [x] BOLT: BRIDGING THE GAP BETWEEN AUTO-TUNERS AND HARDWARE-NATIVE PERFORMANCE[MLSys'22]: [Paper] [Note]
• [x] ROLLER: Fast and Efficient Tensor Compilation for Deep Learning[OSDI'22]: [Paper] [Note]
• [x] Cocktailer: Analyzing and Optimizing Dynamic Control Flow in Deep Learning[OSDI'23]: [Paper] [Note]
• [x] Welder: Scheduling Deep Learning Memory Access via Tile-graph[OSDI'23]: [Paper] [Note]
• [x] Chimera: An Analytical Optimizing Framework for Effective Compute-intensive Operators Fusion[HPCA'23]: [Paper] [Note]
• [x] Effectively Scheduling Computational Graphs of Deep Neural Networks toward Their Domain-Specific Accelerators[OSDI'23]: [Paper]
• [ ] Operator Fusion in XLA: Analysis and Evaluation: [Paper]

Compiler Design

IR Design

• [x] MLIR: Scaling Compiler Infrastructure for Domain Specifific Computation[CGO'21]: [Paper] [Note]
• [x] AMOS: Enabling Automatic Mapping for Tensor Computations On Spatial Accelerators with Hardware Abstraction[ISCA'22]: [Paper] [Note]
• [x] Graphene: An IR for Optimized Tensor Computations on GPUs[ASPLOS'23]: [Paper] [Note]

Auto Schedule

Space Exploration

Software-Hardware Co-Design

Polyhedral

Inter-Operator Optimization

Distrubited Training

Accelerator Architecture

Sparse Model

On-device Training

• [x] PockEngine: Sparse and Efficient Fine-tuning in a Pocket[MICRO'23]: [Paper] [Note]

LLM Infer

Deep Learning

• [x] Attention is all you need[NIPS'17]: [Paper] [Note]
• [x] Big bird: transformers for longer sequences[NIPS'20]: [Paper] [Note]
• [x] Gated Linear Attention Transformers with Hardware-Efficient Training[arxiv]: [Paper] [Note]

HPC/CUDA

Parallelism

Computer Architecutre

Compiler

• [x] Honeycomb: Secure and Efficient GPU Executions via Static Validation[OSDI'23]: [Paper] [Note]
• [x] HIDA: A Hierarchical Dataflow Compiler for High-Level Synthesis[ASPLOS'24]: [Paper] [Note]

OS

• [x] RedLeaf: Isolation and Communication in a Safe Operating System[OSDI'20]: [Paper] [Note]
• [x] Unikraft: Fast, Specialized Unikernels the Easy Way[EuroSys'21]: [Paper] [Note]
• [x] The Deminkernel Datapath OS Architecture for Microsecond-scale Datacenter Systems[SOSP'21]: [Paper] [Note]

Hypervisor

• [x] HyperBench: A Benchmark Suite for Virtualization Capabilities: [Paper] [Note]
• [x] Security and Performance in the Delegated User-level Virtualization[OSDI'23]: [Paper] [Note]
• [x] Nephele: Extending Virtualization Environments for Cloning Unikernel-based VMs[EuroSys'23]: [Paper] [Note]
• [x] Honeycomb: Secure and Efficient GPU Executions via Static Validation[OSDI'23]: [Paper] [Note]

RISC-V