Open isidentical opened 7 months ago
Essentially I'd like to keep the graph itself in GPU since it is a single graph, but offload the weights back and forth between GPU and RAM and re-use the same oneflow graph for all the weights/pipelines (so kinda related to #581)
Offloading compiled pipelines back and forth (or at least their weights) is a useful feat when dealing with multiple models on a single device with limited VRAM (but a lot of RAM). This doesn't seem possible right now: