Closed Chi-Kai closed 1 week ago
Support 阶跃星辰(stepfun) model, api documentation (https://platform.stepfun.com)
fixes #958
envoy.yaml:
# File generated by hgctl. Modify as required. admin: address: socket_address: protocol: TCP address: 0.0.0.0 port_value: 9901 static_resources: listeners: - name: listener_0 address: socket_address: protocol: TCP address: 0.0.0.0 port_value: 10000 filter_chains: - filters: - name: envoy.filters.network.http_connection_manager typed_config: "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager scheme_header_transformation: scheme_to_overwrite: https stat_prefix: ingress_http # Output envoy logs to stdout access_log: - name: envoy.access_loggers.stdout typed_config: "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog # Modify as required route_config: name: local_route virtual_hosts: - name: local_service domains: [ "*" ] routes: - match: prefix: "/" route: cluster: stepfun timeout: 300s http_filters: - name: wasmtest typed_config: "@type": type.googleapis.com/udpa.type.v1.TypedStruct type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm value: config: name: wasmtest vm_config: runtime: envoy.wasm.runtime.v8 code: local: filename: /etc/envoy/plugin.wasm configuration: "@type": "type.googleapis.com/google.protobuf.StringValue" value: | { "provider": { "type": "stepfun", "domain": "api.stepfun.com", "apiTokens": [ "api_token" ] } } - name: envoy.filters.http.router clusters: - name: httpbin connect_timeout: 30s type: LOGICAL_DNS # Comment out the following line to test on v6 networks dns_lookup_family: V4_ONLY lb_policy: ROUND_ROBIN load_assignment: cluster_name: httpbin endpoints: - lb_endpoints: - endpoint: address: socket_address: address: httpbin port_value: 80 - name: stepfun connect_timeout: 30s type: LOGICAL_DNS dns_lookup_family: V4_ONLY lb_policy: ROUND_ROBIN load_assignment: cluster_name: stepfun endpoints: - lb_endpoints: - endpoint: address: socket_address: address: api.stepfun.com port_value: 443 transport_socket: name: envoy.transport_sockets.tls typed_config: "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext "sni": "api.stepfun.com"
Request:
curl "http://localhost:10000/v1/chat/completions" -H "Content-Type: application/json" -d '{ "model": "step-1-8k", "messages": [ { "role": "system", "content": "你是由阶跃星辰提供的AI聊天助手,你擅长中文,英文,以及多种其他语言的对话。在保证用户数据安全的前提下,你能对用户的问题和请求,作出快速和精准 的回答。同时,你的回答和建议应该拒绝黄赌毒,暴力恐怖主义的内容。" }, { "role": "user", "content": "你好,请介绍一下阶跃星辰的人工智能!" } ] }'
Response
{ "id": "c0ab86c4213e81665861b1ef792377ef.018fc2def96479d5ab2211389dc00bbc", "object": "chat.completion", "created": 1716961343, "model": "step-1-8k", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "阶跃星辰是一家致力于提供人工智能技术的公司。他们的人工智能技术旨在为用户提供高效、精准的回答和建议。这种技术可以理解和处理多种语言,包括中文、英文以及其他一些语言。\n\n阶跃星辰的AI聊天助手能够快速理解用户的问题和请求,并提供相关的回答。它还能够保护用户的数据安全,确保用户的个人信息得到妥善处理。此外,这个AI聊天助手还能够拒绝黄赌毒、暴力恐怖主义等不良内容,为用户提供一个健康、积极的交流环境。\n\n总之,阶跃星辰的人工智能技术旨在为用户提供更好的交流体验,帮助他们解决问题、获取信息,并在这个过程中保护他们的数据安全。" }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 85, "completion_tokens": 141, "total_tokens": 226 } }
@Chi-Kai Please resolve the conflicts
Ⅰ. Describe what this PR did
Support 阶跃星辰(stepfun) model, api documentation (https://platform.stepfun.com)
Ⅱ. Does this pull request fix one issue?
fixes #958
Ⅲ. Why don't you add test cases (unit test/integration test)?
Ⅳ. Describe how to verify it
envoy.yaml:
Request:
Response
Ⅴ. Special notes for reviews