search

mlc-ai / mlc-llm

Universal LLM Deployment Engine with ML Compilation
https://llm.mlc.ai/
Apache License 2.0
19.23k stars 1.58k forks source link

[Feature Request] openai v1/completion api support #2998

Closed lpb1 closed 3 weeks ago

lpb1 commented 3 weeks ago

🚀 Feature

openai v1/completion api

from openai import OpenAI

client = OpenAI(
    base_url='https://xxxx/v1',
    api_key='sk-xxxx'
)

response = client.completions.create(
    model="gpt4o",
    prompt="I am a ",
)

print(response)

Motivation

For IDE code assistant plugins, these use the v1/completion endpoint for code autocompletion.

lpb1 commented 3 weeks ago

my bad, I find the implementation in code...