In my case, I have multiple SegmentAnythingUltraV2 nodes, processing different images, and I wanted to use cache_model to improve speed, but found that the final GPU usage was N* cache_model .
Can we use a global cache to make it easier to use and speed up the process?
In addition, it is recommended that after the model is used, it is unloaded to the CPU-memory and does not occupy the GPU. The switching speed between the two is very fast. It is not recommended to use this kind of tool-type resident GPU.
There is indeed this issue, as the nodes develp in the early stages did not take these into consideration. To change this situation, a new node needs to be created
In my case, I have multiple SegmentAnythingUltraV2 nodes, processing different images, and I wanted to use cache_model to improve speed, but found that the final GPU usage was N* cache_model .
Can we use a global cache to make it easier to use and speed up the process? In addition, it is recommended that after the model is used, it is unloaded to the CPU-memory and does not occupy the GPU. The switching speed between the two is very fast. It is not recommended to use this kind of tool-type resident GPU.