flexflow / FlexFlow

FlexFlow Serve: Low-Latency, High-Performance LLM Serving
https://flexflow.readthedocs.io
Apache License 2.0
1.59k stars 218 forks source link

Change `Allocator` ownership model #1416

Open reyna-abhyankar opened 2 weeks ago

reyna-abhyankar commented 2 weeks ago

Instead of passing around an Allocator &, we should pass around std::unique_ptr<IAllocator> const &.

It expresses that someone else owns it and that you don't get to take ownership of it (but you can share the reference to it), and allows you to modify the IAllocator - @lockshaw