-
### Proposal to improve performance
I have observed that TTFT increases linearly with a total number of batched tokens.
For example, given 100k batch
- TTFT is around 2min when an average prompt…
-
_This issue was automatically created by [Allstar](https://github.com/ossf/allstar/)._
**Security Policy Violation**
Security policy not enabled.
A SECURITY.md file can give users information about w…
-
_This issue was automatically created by [Allstar](https://github.com/ossf/allstar/)._
**Security Policy Violation**
Security policy not enabled.
A SECURITY.md file can give users information about w…
-
_This issue was automatically created by [Allstar](https://github.com/ossf/allstar/)._
**Security Policy Violation**
Security policy not enabled.
A SECURITY.md file can give users information about w…
-
_This issue was automatically created by [Allstar](https://github.com/ossf/allstar/)._
**Security Policy Violation**
Security policy not enabled.
A SECURITY.md file can give users information about w…
-
_This issue was automatically created by [Allstar](https://github.com/ossf/allstar/)._
**Security Policy Violation**
Security policy not enabled.
A SECURITY.md file can give users information about w…
-
_This issue was automatically created by [Allstar](https://github.com/ossf/allstar/)._
**Security Policy Violation**
Security policy not enabled.
A SECURITY.md file can give users information about w…
-
_This issue was automatically created by [Allstar](https://github.com/ossf/allstar/)._
**Security Policy Violation**
Security policy not enabled.
A SECURITY.md file can give users information about w…
-
_This issue was automatically created by [Allstar](https://github.com/ossf/allstar/)._
**Security Policy Violation**
Security policy not enabled.
A SECURITY.md file can give users information about w…
-
_This issue was automatically created by [Allstar](https://github.com/ossf/allstar/)._
**Security Policy Violation**
Security policy not enabled.
A SECURITY.md file can give users information about w…