sayakpaul / ml-deployment-k8s-fastapi

This project shows how to serve an ONNX-optimized image classification model as a web service with FastAPI, Docker, and Kubernetes.
https://medium.com/google-developer-experts/load-testing-tensorflow-serving-and-fastapi-on-gke-411bc14d96b2
Apache License 2.0
198 stars 36 forks source link

Deploy the Docker image on GKE #6

Closed sayakpaul closed 2 years ago

deep-diver commented 2 years ago

cloud build scripts to push docker image to GCR

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/{PROJECT_ID}/{IMAGE_ID}', '.']
- name: 'gcr.io/cloud-builders/docker'
  args: ['push', 'gcr.io/{PROJECT_ID}/{IMAGE_ID}']

then run gcloud builds submit --config {PATH}/script.yaml

sayakpaul commented 2 years ago

Would configurations do script.yaml specify in this case?

deep-diver commented 2 years ago

@sayakpaul yup. the file name is just an example.

here are the scripts for GKE. I have seen Welcome! message successfully from the fastAPI server.

Both k8s objects are created by the following commands respectively.

$ kubectl create deployment fastapi-server \
                            --image=gcr.io/gcp-ml-172005/k8s-fastapi \
                            --dry-run=client \
                            --output=yaml > deployment.yaml

$ kubectl expose deployment fastapi-server \
                            --type LoadBalancer \
                            --port 80 \
                            --target-port 8080 \
                            --dry-run=client \
                            --output=yaml > service.yaml

Deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: fastapi-server
  name: fastapi-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: fastapi-server
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: fastapi-server
    spec:
      containers:
      - image: gcr.io/{PROJECT_ID}/{IMAGE_ID}
        name: {IMAGE_ID}
        ports:
        - containerPort: 80          
        resources: {}
status: {}

Service.yaml

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: null
  labels:
    app: fastapi-server
  name: fastapi-server
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: fastapi-server
  type: LoadBalancer
status:
  loadBalancer: {}
deep-diver commented 2 years ago

@sayakpaul

if you don't mind, let me put these basic yaml spec files.

sayakpaul commented 2 years ago

Sure. So the next steps would be to create a workflow action file that:

Does this sound good?

sayakpaul commented 2 years ago

@deep-diver it'd be also cool to add config spec for horizontal pod autoscaling (HPA).

sayakpaul commented 2 years ago

have seen Welcome! message successfully from the fastAPI server.

Do you mean you get this after locally running Docker? Also, did you create a k8s cluster on GKE beforehand? I guess that's the only thing that needed to be done beforehand. Rests can be automated using GitHub Actions.

deep-diver commented 2 years ago

@sayakpaul

Yes. I will provide the script file for the Cloud Build as well, so you can use them in the following-up steps (GitHub Action).

About HPA, I will include it. For any further modifications, please feel free to make them since these are the basic stuffs.

Do you mean you get this after locally running Docker? Also, did you create a k8s cluster on GKE beforehand? I guess that's the only thing that needed to be done beforehand. Rests can be automated using GitHub Actions.

I have created GKE cluster manually via GCP console, then deploy the pods and service. Then I have seen the Welcome! message from the FastAPI server running on GKE cluster (so no locally I guess).

sayakpaul commented 2 years ago

Yes. I will provide the script file for the Cloud Build as well, so you can use them in the following-up steps (GitHub Action).

I don't think Cloud Build is needed. We might wanna use GitHub Actions for that part, no? You can refer to the sample configuration I had sent over to you via chat.

deep-diver commented 2 years ago

@sayakpaul yeah I am aware of using docker build/push simply solves the problem. OK, let me push the k8s resources only then.

I thought yaml script is much more handy and manageable, so I just wanted to keep it. But it is a good point since lots of audience expects not to use Cloud Build but as much as GitHub Action possible.

sayakpaul commented 2 years ago

Yes. I am a big fan of wrapping all the steps into a YAML build configuration as well. But since we are relying on GitHub Actions heavily for this project, I thought it'd be a cool idea to showcase what we can do with GitHub Actions. WDYT?

Also, if this sounds good then do you think these steps would suffice?

deep-diver commented 2 years ago

yeah it sounds good, and those steps look sufficient :)