Closed rickstaa closed 4 months ago
This pull request gives Orchestrators the ability to enable the DeepCache optimization method. This method can provide a 50% inference speed up for multi-step inference (See https://livepeer-ai-spe.productlane.com/roadmap?id=899de8b9-812f-461c-9866-820af7e27438).
This pull request gives Orchestrators the ability to enable the DeepCache optimization method. This method can provide a 50% inference speed up for multi-step inference (See https://livepeer-ai-spe.productlane.com/roadmap?id=899de8b9-812f-461c-9866-820af7e27438).