Closed avianion closed 2 weeks ago
This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 15 days."
This issue was closed because it has been stalled for 15 days with no activity.
https://arxiv.org/pdf/2407.02490
As according to this paper, 10x prefill rates can be achieved using different attention mechanisms.