search

AI-Hypercomputer / jetstream-pytorch

PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"
Apache License 2.0
41 stars 15 forks source link

fix ray engine crashes on multihost #170

Closed sixiang-google closed 3 months ago

sixiang-google commented 3 months ago

Fix https://github.com/google/jetstream-pytorch/issues/150 without all_gather the kv cache