search

google-ai-edge / ai-edge-torch

Supporting PyTorch models with the Google AI Edge TFLite runtime.
Apache License 2.0
327 stars 45 forks source link

External kv #90

Closed hheydary closed 3 months ago

hheydary commented 3 months ago

Add experimental support for external KV cache.

BUG=b/353293690