Problem Statement: Currently, the v1/models API does not expose the context size of the models, making it difficult for API consumers to understand the limitations or capabilities of the models in terms of context size.
Proposed Solution: It would be highly beneficial to add an additional field to the v1/models API response that specifies the context size of each model. This field could be named contextSize and should provide an integer value representing the number of tokens or characters the model can handle.
Expected Outcome: By including the contextSize field in the API response, developers and users will have a clearer understanding of model capabilities, enabling better decision-making when choosing models for specific tasks.
Additional context: Context size is a critical parameter for many NLP tasks and applications, and making this information readily available via the API could significantly enhance user experience and application efficiency.
Thank you for considering this enhancement to the v1/models API.
Feature Request
Problem Statement: Currently, the
v1/modelsAPI does not expose the context size of the models, making it difficult for API consumers to understand the limitations or capabilities of the models in terms of context size.Proposed Solution: It would be highly beneficial to add an additional field to the
v1/modelsAPI response that specifies the context size of each model. This field could be namedcontextSizeand should provide an integer value representing the number of tokens or characters the model can handle.Expected Outcome: By including the
contextSizefield in the API response, developers and users will have a clearer understanding of model capabilities, enabling better decision-making when choosing models for specific tasks.Additional context: Context size is a critical parameter for many NLP tasks and applications, and making this information readily available via the API could significantly enhance user experience and application efficiency.
Thank you for considering this enhancement to the
v1/modelsAPI.