search

flexflow / FlexFlow

FlexFlow Serve: Low-Latency, High-Performance LLM Serving
https://flexflow.readthedocs.io
Apache License 2.0
1.59k stars 218 forks source link

Convert allocator to arena #1419

Open reyna-abhyankar opened 1 week ago

reyna-abhyankar commented 1 week ago

Have Allocator track the memory it has allocated and de-allocate upon going out of scope

Also see #1416