kubeagi / arcadia

A diverse, simple, and secure one-stop LLMOps platform
http://www.kubeagi.com/
Apache License 2.0
64 stars 21 forks source link

Support to use ak/sk to invoke chat/OpenAI API through ai-gateway #935

Open nkwangleiGIT opened 3 months ago

nkwangleiGIT commented 3 months ago

for now we only support K8s bearer token to invoke K8s API, we can also use kubectl using kubeconfig file but we also need to invoke chat(application) API and OpenAI API, we can use the same ak/sk for OpenAI API through ai-gateway, as chat API will normally invoke OpenAI under the hood.

nkwangleiGIT commented 2 months ago

API gateway support to generate ak/sk to protect the upstreaming AI API, here is the basic steps:

## 1.1 Create consumer 

Applications can correspond a Consumer to a user in the actual application.
```bash
kubectl create -f - <<EOF
apiVersion: gw.t7d.io/v1
kind: Consumer
metadata:
  annotations:
    tamp/clusterId: host-cluster
    tamp/displayName: example
  name: consumer-example
  namespace: test
spec:
  conName: example
  description: "consumer example"

EOF

1.2 Create AK/SK

kubectl create -f - <<EOF
apiVersion: v1
data:
  kongCredType: YmFzaWMtYXV0aA==  #base64 encoded,the value is fixed,decode is basic-auth
  password: M0B4R28zQW5O    #base64 encoded
  username: emhpcHVhaQ==    #base64 encoded
kind: Secret
metadata:
  labels:
    tamp/consumernanme: consumer-example #the cr name of consumer
    tamp/credentialtype: basic-auth
  name: consumer-example-key
  namespace: test
type: Opaque
EOF

1.3 Grant api

kubectl create -f - <<EOF
apiVersion: gw.t7d.io/v1
kind: Authorize
metadata:
  annotations:
    kubernetes.io/ingress.class: kong
    tamp/clusterId: host-cluster
    tamp/consumerdisnanme: example
    tamp/groupdisname: openai service
    tamp/publishapidisname: openai api
  labels:
    tamp/apins: test
    tamp/bindingStatus: binding
    tamp/consumernanme: consumer-example
    tamp/consumerns: test
    tamp/env: kong
    tamp/groupname: apigroups-openai
    tamp/publishapiname: snapshotapi-openai-api
    tamp/snapshot: snapshotapi-openai-api
  name: auth-example
  namespace: test
spec:
  apiName: snapshotapi-ejayynv9
  bindingStatus: binding
  bindingType: allow
  conName: consumer-fccdce2l
  env: kong
EOF