search

google / jetstream-pytorch

PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"
Apache License 2.0
33 stars 14 forks source link

fix ray engine crashes on multihost #170

Closed sixiang-google closed 3 weeks ago

sixiang-google commented 3 weeks ago

Fix https://github.com/google/jetstream-pytorch/issues/150 without all_gather the kv cache