kubernetes 2020

[uniquejava]

IBM Cloud Kubernetes Free Edition

Free 版本
1 个 working node,
2C 4G，
磁盘： 2G
Docker Image 空间： 1G

不能互相访问不同account下的 docker image， 暂时是给他们的 docker image 设置了不同的 namespace.

1. Development: cn.icr.io/awesome_lab/awesome-app:1.0.1
2. Production: cn.icr.io/awesome_app/awesome-app:1.0.1

Development 环境

java: http://1.2.3.4:32439/awesome_app/swagger-ui.html

mysql: 1.2.3.4:32439, user1/secret1

维护 Development 上的 k8s 集群

#  k8s切换到development环境
ibmcloud login -a cloud.ibm.com -r cn -g AwesomeApp --sso
ibmcloud ks cluster config --cluster bxxxxxxxfo3623sfd
ibmcloud get po
# 查看端口号
kubectl get svc

# 查看public ip
ic ks worker ls --cluster mycluster

# 最终的访问地址是 `https://Public_Ip:<NodePort>`

# 查看spring boot启动的日志文件 (网络原因有点卡), 两个pod实例的日志交织在一起
# 10.47.84.77 是node的ip地址
../kubernetes/development/nsenter-node.sh 10.47.84.77
=> If you dont see a command prompt, try pressing enter.
=> root@kube-xxx:~#
=> root@kube-xxx:~# tail -f /mnt/awesome/logs/sa.log (查看后台日志文件)
=> root@kube-xxx:~# exit
（按Ctrl + D)
=> logout
=> pod "cyper-nsenter-10.47.84.37" deleted

# 详见： https://alexei-led.github.io/post/k8s_node_shell/

# mysql和日志文件挂在node的/mnt/aa目录下
$ ls /mnt/aa
=> mysql-data
=> logs

名词解释

IBM Cloud CLI 下的：
ic => alias for ibmcloud
cr => container registry
ks => kubernetes

K8s 中的 （以下缩略词，单数和复数形式通用）
no => node，nodes
po => pod，pods
deploy => deployment，deployments
svc => service，services

另外
resource_type name 和 resource_type/name 等价

如
kubectl get deploy/aa -o wide
<==>
kubectl get deploy aa -o wide

通过 IBM Cloud CLI 登录并切换 kubernetes context

安装 IBM Cloud CLI
curl -sL https://ibm.biz/idt-installer | bash

登录
ibmcloud login -a cloud.ibm.com -r cn-tok -g default --sso

输入one time passcode

选择 Cyper's Account

ibmcloud target

ibmcloud ks cluster config --cluster bps2xxxxxxxxs8g

export KUBECONFIG=/Users/\$USER/.bluemix/plugins/container-service/clusters/bps2xxxxxxxxs8g/kube-config-hou02-mycluster.yml

kubectl version --short

创建 namespace, 远程构建 docker image

在IBM Cloud Container Registry中必须创建namespace才能保存image
ic cr namespace-add awesomelab

查看image列表
ic cr images

显示当前的region名称
ic cr region
You are targeting region 'ap-north', the registry is 'cn.icr.io'.

进入到spring boot src目录下
cd awesome_app

下面是打包image， 会用到region和namespace， 注意提前写好.dockerignore文件
ic cr build -t cn.icr.io/awesomelab/awesome-app:1.0.1 .

查看image列表
ic cr images

Alternative: 本地构建 docker image 并 push 到 ibm cloud

cd awesome_app
docker build -t awesome-app .
dockers images|grep awesome-app
docker tag awesome-app cn.icr.io/aales_support_app/awesome-app:1.0.1
docker push cn.icr.io/aales_support_app/awesome-app:1.0.1

部署 redis/mysql/spring boot

cd awesome_app/kubernetes
kubectl apply -f k8s-redis.yaml
kubectl apply -f k8s-pv.yaml
kubectl apply -f k8s-mysql.yaml
kubectl apply -f k8s-aa.yaml

查看部署状态
kubectl get pod【-w】

Development 环境查看日志

pod 的名字: kubectl get pods

实时查看日志： kubectl logs + <pod 的名字>

1. app 的日志：kubectl logs deploy/aa --tail 5 -f
2. mysql 的日志：kubectl logs mysql-68b7479b44-smj4v
3. redis 的日志: kubectl logs redis-65fd448c9b-2lwnq

查看日志文件

kubectl exec -it <pod 的名字> bash
vi /logs/aa.log

初期通过命令行而非 yaml 的方式部署 (备份)

创建 redis 的 yaml 文件

kubectl create deployment redis --image=redis
# 默认的service type是ClusterIP (让集群内部可以通过别名访问pod)
kubectl expose deployment redis --port 6379

直接好用！

查看redis的yaml
kubectl get deploy/redis -o yaml

创建 MySQL 的 yaml 文件

kubectl create deployment mysql --image=mysql
kubectl expose deploy/mysql --type=NodePort --port 3306
kubectl get services

会报错，需要把yaml导出，然后配置MYSQL_ROOT_PASSWORD

查看mysql的yaml
kubectl get deploy/mysql -o yaml

删除mysql
kubectl delete deployment,svc mysql

手动修改k8s-mysql.yaml

最后 apply

创建 app 的 yaml 文件

kubectl create deployment aa --image=cn.icr.io/awesomelab/awesome-app:1.0.0
kubectl expose deploy/aa --name=aa --type=NodePort --port 9090
kubectl describe svc aa

查看app的yaml
kubectl get deploy/aa -o yaml

References

1. https://cloud.ibm.com/docs/containers?topic=containers-cs_cluster_tutorial#cs_cluster_tutorial
2. https://github.com/IBM/java-spring-app
3. https://kubernetes.io/docs/tasks/run-application/run-single-instance-stateful-application/

[uniquejava]

问题排查

1. pod 一直显示pending状态

kubectl exec kubia-zgxn9 -- curl -s http://10.47.252.17 Error from server (BadRequest): pod kubia-zgxn9 does not have a host assigned

见： https://stackoverflow.com/a/50831348/2497876

关键句： perhaps it's asking for resources that no node can satisfy at the moment. 方案： kubectl describe pod kubia-zgxn9

使用descrbie命令发现是我的pod 找不到pvc， 瞬间明白。

进入pod的另一种方式

# run shpod
kubectl apply -f https://bret.run/shpod.yml
kubectl attach --namespace=shpod -ti shpod

#  手动抓取分配给svc的clusterIP
IP=$(kubectl get svc httpenv -o go-template --template '{{ .spec.clusterIP }}')
curl http://$IP:8888/

# 使用jq过滤 (brew install jq)
curl -s http://$IP:8888/ | jq .HOSTNAME

shpod.yml

# from https://github.com/jpetazzo/shpod

apiVersion: v1
kind: Namespace
metadata:
  name: shpod
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: shpod
  namespace: shpod
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: shpod
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: shpod
  namespace: shpod
---
apiVersion: v1
kind: Pod
metadata:
  name: shpod
  namespace: shpod
spec:
  serviceAccountName: shpod
  containers:
  - name: shpod
    image: bretfisher/shpod
    imagePullPolicy: Always
    stdin: true
    tty: true
    env:
    - name: HOSTIP
      valueFrom:
        fieldRef:
          fieldPath: status.hostIP