search

sassoftware / sas-container-recipes

A collection of recipes and other resources for building containers that include SAS Viya software.
https://www.sas.com
Apache License 2.0
89 stars 42 forks source link

Kubernete Pods are evicted #12

Closed onion1984 closed 5 years ago

onion1984 commented 5 years ago

Hi,

I have followed the instructions to built multi-container visual analytics on CentOS and I deployed it into Azure Kubernetes Service (AKS). The cluster only consist of 1 worker node, with 32 core and 256G RAM.

After the deployment, the administrativeservices pod status shows error after it ran for a while, and the rest of the Pods' status start to show error too. Eventually all Pods were Evicted.

screenshot

g8sman commented 5 years ago

Hi...I have never seen where one pod was replicated so many times. My first guess is that the amount of memory needed for the pods is greater than the one node can handle. I have some questions.

Cheers!

onion1984 commented 5 years ago

To answer your questions

  1. This is not the only pod. I deployed all the pods in the visual mpp directory. The rest of the pods all look like this
  2. I did not change the replica number, and I also didn't try to scale the pods. Once a pod is being evicted, Kubernetes will start another pod, again and again. That's why you see so many evicted pods in the screenshot
  3. I will try to decribe the pod and post the information here later
TanJicong commented 5 years ago

Hi, I read the build instruction again, I noticed that the wording "Only Red Hat Enterprise Linux 7 based images are supported for recipes" for multi container build. Does it matter if my containers are built using CentOS as the base image?

TanJicong commented 5 years ago

Issue closed. The root cause is because AKS is not able to pull all the docker images due to my Docker Hub setting

g8sman commented 5 years ago

Thanks @TanJicong. Will close the issue. Are you using DockerHub as your registry?

onion1984 commented 5 years ago

yes, i was using docker hub initially with private repo. I have removed all the docker images after that