12. Webサイトにアクセスできない

kitakou0313 commented 3 years ago

https://contest.ictsc.net/#/problems/5077a5fa-d0c3-4851-8cc3-97e579403a25

kitakou0313 commented 3 years ago

初期状態踏み台からcurl実行

user@team35-bastion:~$ curl -k https://192.168.13.101
<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx</center>
</body>
</html>
user@team35-bastion:~$

502なのでこれがちゃんと返るようにする

kitakou0313 commented 3 years ago

nginx deploymrnyのyaml

user@team01-prob13-master:~$ cat nginx.yaml
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-svc
  labels:
    app: nginx
spec:
  selector:
    app: nginx
  ports:
  - name: https
    protocol: TCP
    port: 443
    targetPort: 443
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 443
        - containerPort: 80
        volumeMounts:
        - name: nginx-conf
          mountPath: /etc/nginx/conf.d
        - name: tls-cert
          mountPath: /etc/nginx/tls
        - name: index-html
          mountPath: /usr/share/nginx/html
      volumes:
      - name: nginx-conf
        configMap:
          name: nginx-conf
      - name: tls-cert
        secret:
          secretName: tls-secret
      - name: index-html
        configMap:
          name: index-html

user@team01-prob13-master:~$

kitakou0313 commented 3 years ago

デプロイリソース一覧

user@team01-prob13-master:~$ kubectl get deployment
NAME               READY   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   3/3     3            3           20d
user@team01-prob13-master:~$ kubectl get services
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
kubernetes   ClusterIP   10.96.0.1      <none>        443/TCP          20d
nginx-svc    ClusterIP   10.100.167.0   <none>        443/TCP,80/TCP   20d
user@team01-prob13-master:~$ kubectl get ingresses
NAME            CLASS    HOSTS   ADDRESS         PORTS     AGE
nginx-ingress   <none>   *       192.168.13.11   80, 443   20d

kitakou0313 commented 3 years ago

ingress.yamlの設定

user@team01-prob13-master:~$ cat ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/proxy-buffering: "on"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "8"
    kubernetes.io/ingress.allow-http: "false"
  name: nginx-ingress
spec:
  tls:
  - secretName: tls-secret
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-svc
            port:
              number:  443

kitakou0313 commented 3 years ago

Cluster IP経由では接続される -> ingressを頑張ればいけそう？

user@team01-prob13-master:~$ kubectl get services
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
kubernetes   ClusterIP   10.96.0.1      <none>        443/TCP          20d
nginx-svc    ClusterIP   10.100.167.0   <none>        443/TCP,80/TCP   20d
user@team01-prob13-master:~$ curl -k https://10.100.167.0
welcome to ictsc2021 summer

user@team01-prob13-master:~$ curl -k https://10.100.167.0

kitakou0313 commented 3 years ago

user@team01-prob13-master:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 9c:a3:ba:32:14:61 brd ff:ff:ff:ff:ff:ff
    inet 192.168.13.1/24 brd 192.168.13.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::9ea3:baff:fe32:1461/64 scope link
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:df:1f:46:4a brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
4: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default
    link/ether 26:b8:8e:e6:52:1e brd ff:ff:ff:ff:ff:ff
    inet 10.244.0.0/32 brd 10.244.0.0 scope global flannel.1
       valid_lft forever preferred_lft forever
    inet6 fe80::24b8:8eff:fee6:521e/64 scope link
       valid_lft forever preferred_lft forever
5: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
    link/ether 4a:f3:de:0d:f7:3e brd ff:ff:ff:ff:ff:ff
    inet 10.244.0.1/24 brd 10.244.0.255 scope global cni0
       valid_lft forever preferred_lft forever
    inet6 fe80::48f3:deff:fe0d:f73e/64 scope link
       valid_lft forever preferred_lft forever
6: veth29baf4ec@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default
    link/ether 56:52:4f:ca:a8:04 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::5452:4fff:feca:a804/64 scope link
       valid_lft forever preferred_lft forever
7: vethf70bc5fc@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default
    link/ether 56:1c:b8:15:1a:2b brd ff:ff:ff:ff:ff:ff link-netnsid 1
    inet6 fe80::541c:b8ff:fe15:1a2b/64 scope link
       valid_lft forever preferred_lft forever

kitakou0313 commented 3 years ago

nginx.conf

user@team01-prob13-master:~$ cat nginx.conf
server {
    listen 443 ssl http2;
    server_name nginx.ictsc2021sum.local;

    server_tokens off;
    access_log /var/log/nginx/ucwork_ssl_access.log;
    error_log /var/log/nginx/ucwork_ssl_error.log;

    ssl_certificate      /etc/nginx/tls/tls.crt;
    ssl_certificate_key  /etc/nginx/tls/tls.key;

    location / {
       root   /usr/share/nginx/html;
       index  index.html index.htm;
    }
}

kitakou0313 commented 3 years ago

user@team01-prob13-master:~$ cat ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/proxy-buffering: "on"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "8"
    kubernetes.io/ingress.allow-http: "true"
  name: nginx-ingress
spec:
  tls:
  - secretName: tls-secret
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-svc
            port:
              number:  443

kitakou0313 commented 3 years ago

https://kubernetes.github.io/ingress-nginx/troubleshooting/ 最初からこれを見るべきでしたね…

kitakou0313 commented 3 years ago

user@team01-prob13-master:~$ kubectl logs ingress-nginx-controller-5cb8d9c6dd-pdttj -n ingress-nginx

2021/08/28 07:30:17 [error] 32#32: *1300630 upstream sent too big header while reading response header from upstream, client: 10.244.1.1, server: _, request: "GET / HTTP/2.0", upstream: "http://10.244.1.15:443/", host: "192.168.13.101"

https://fits.hatenablog.com/entry/2015/08/28/112250 なるほどね

kitakou0313 commented 3 years ago

user@team01-prob13-master:~$ cat ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/proxy-buffering: "on"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "800"
    kubernetes.io/ingress.allow-http: "false"
  name: nginx-ingress
spec:
  tls:
  - secretName: tls-secret
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-svc
            port:
              number:  443

bufferが小さすぎてこけてたみたいなので大きくする

user@team35-bastion:~$ curl -k https://192.168.13.101
<html>
<head><title>400 The plain HTTP request was sent to HTTPS port</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<center>The plain HTTP request was sent to HTTPS port</center>
<hr><center>nginx</center>
</body>
</html>

エラーが変わった！

kitakou0313 commented 3 years ago

user@team01-prob13-master:~$ cat nginx.conf
server {
    listen 443 ssl http2;
    server_name nginx.ictsc2021sum.local;

    server_tokens off;
    access_log /var/log/nginx/ucwork_ssl_access.log;
    error_log /var/log/nginx/ucwork_ssl_error.log;

    ssl_certificate      /etc/nginx/tls/tls.crt;
    ssl_certificate_key  /etc/nginx/tls/tls.key;

    location / {
       root   /usr/share/nginx/html;
       index  index.html index.htm;
    }
}

kitakou0313 commented 3 years ago

masterサーバー上だと普通に通る

user@team01-prob13-master:~$ curl -k https://10.100.167.0
welcome to ictsc2021 summer

-k外したときのerrorも自己署名証明書関連なのでingressが悪そう

kitakou0313 commented 3 years ago

user@team01-prob13-master:~$ cat ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/proxy-buffering: "on"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "800"
    kubernetes.io/ingress.allow-http: "true"
  name: nginx-ingress
spec:
  tls:
  - secretName: tls-secret
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-svc
            port:
              number:  443

kitakou0313 commented 3 years ago

まとめ

nginx側ではport 443へのHTTPSしか受け付けていない
igressのnginx側でhttpsで接続できないとBad requestになる

ingressを挟むか挟まないかでエラーが変わる

user@team01-prob13-master:~$ curl https://10.100.167.0
curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

user@team01-prob13-master:~$ curl https:///192.168.13.101
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

ingress抜きの方は自己署名証明書なので-kで無理やりhttpsにできる

kitakou0313 commented 3 years ago

user@team01-prob13-master:~$ cat ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/proxy-buffering: "on"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "800"
    kubernetes.io/ingress.allow-http: "false"
  name: nginx-ingress
spec:
  tls:
  - secretName: tls-secret
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-svc
            port:
              number:  443

kitakou0313 commented 3 years ago

user@team35-bastion:~$ curl -vk https://192.168.13.101
*   Trying 192.168.13.101:443...
* TCP_NODELAY set
* Connected to 192.168.13.101 (192.168.13.101) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: O=Acme Co; CN=Kubernetes Ingress Controller Fake Certificate
*  start date: Aug 26 11:23:09 2021 GMT
*  expire date: Aug 26 11:23:09 2022 GMT
*  issuer: O=Acme Co; CN=Kubernetes Ingress Controller Fake Certificate
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x5652b27a6db0)
> GET / HTTP/2
> Host: 192.168.13.101
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 400
< date: Sun, 29 Aug 2021 03:52:57 GMT
< content-type: text/html
< content-length: 248
< strict-transport-security: max-age=15724800; includeSubDomains
<
<html>
<head><title>400 The plain HTTP request was sent to HTTPS port</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<center>The plain HTTP request was sent to HTTPS port</center>
<hr><center>nginx</center>
</body>
</html>
* Connection #0 to host 192.168.13.101 left intact

CAがKubernetes Ingress Controller Fake Certificateになってるせいでうまくいってなさそう　ここでnginx側のやつ使ってくれたら自己署名証明書にできるので-kオプション付与でtlsできるかな…？ ingress介さない場合はちゃんと自己署名つかってくれてるのでなんかingressの設定がわるいやつかな…？

user@team01-prob13-master:~$ curl -vk https://10.100.167.0
*   Trying 10.100.167.0:443...
* TCP_NODELAY set
* Connected to 10.100.167.0 (10.100.167.0) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=JP; ST=Tokyo; CN=*.ictsc2021sum.local
*  start date: Aug  5 03:32:50 2021 GMT
*  expire date: Aug  5 03:32:50 2022 GMT
*  issuer: C=JP; ST=Tokyo; CN=*.ictsc2021sum.local
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55e6d24bee10)
> GET / HTTP/2
> Host: 10.100.167.0
> user-agent: curl/7.68.0
> accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 200
< server: nginx
< date: Sun, 29 Aug 2021 04:01:15 GMT
< content-type: text/html
< content-length: 30
< last-modified: Sat, 07 Aug 2021 13:47:23 GMT
< etag: "610e8eeb-1e"
< accept-ranges: bytes
<
welcome to ictsc2021 summer

* Connection #0 to host 10.100.167.0 left intact
user@team01-prob13-master:~$

kitakou0313 commented 3 years ago

https://www.ibm.com/docs/en/cloud-paks/cp-management/1.3.0?topic=troubleshooting-key-management-service#fake_cert これ見ながらnginx-ingress-controllerのデフォルト証明書を変更　自己署名証明書に切り替えられた

user@team35-bastion:~$ curl -vk https://192.168.13.101
*   Trying 192.168.13.101:443...
* TCP_NODELAY set
* Connected to 192.168.13.101 (192.168.13.101) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=JP; ST=Tokyo; CN=*.ictsc2021sum.local
*  start date: Aug  5 03:32:50 2021 GMT
*  expire date: Aug  5 03:32:50 2022 GMT
*  issuer: C=JP; ST=Tokyo; CN=*.ictsc2021sum.local
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55910a225db0)
> GET / HTTP/2
> Host: 192.168.13.101
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 400
< date: Sun, 29 Aug 2021 04:43:21 GMT
< content-type: text/html
< content-length: 248
< strict-transport-security: max-age=15724800; includeSubDomains
<
<html>
<head><title>400 The plain HTTP request was sent to HTTPS port</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<center>The plain HTTP request was sent to HTTPS port</center>
<hr><center>nginx</center>
</body>
</html>
* Connection #0 to host 192.168.13.101 left intact

もう倒れてくれ…

kitakou0313 commented 3 years ago

多分LB => nginxでHTTP2使ってるのがダメっぽい　これを切り替えれば動くはず

kitakou0313 commented 3 years ago

httpsを強制

user@team01-prob13-master:~$ cat ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/proxy-buffering: "on"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "800"
    kubernetes.io/ingress.allow-http: "false"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
  name: nginx-ingress
spec:
  tls:
  - secretName: tls-secret
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-svc
            port:
              number:  443

user@team35-bastion:~$ curl -k https://192.168.13.101
welcome to ictsc2021 summer

NKT....

kitakou0313 commented 3 years ago

お世話になっております。チームkstmです。

本事象の原因としては、以下の要因が考えられます。

ingress側のproxy-buffer-sizeによる制限

nginx.ingress.kubernetes.io/proxy-buffer-sizeで指定されたサイズがnginxサーバーのレスポンスのヘッダーよりも小さいため、Bad Gatewayが返されていました。

ser@team01-prob13-master:~$ kubectl logs ingress-nginx-controller-5cb8d9c6dd-pdttj -n ingress-nginx

2021/08/28 07:30:17 [error] 32#32: *1300630 upstream sent too big header while reading response header from upstream, client: 10.244.1.1, server: _, request: "GET / HTTP/2.0", upstream: "http://10.244.1.15:443/", host: "192.168.13.101"

解消手順

こちらは、ingress.yaml内のnginx.ingress.kubernetes.io/proxy-buffer-sizeの値に大きな値を設定し、適用することで解消しました。

nginx ingressがデフォルトで使用する証明書のCN名

nginx ingressにIPアドレスでアクセスした際、デフォルトの設定で使用される証明書のCNがKubernetes Ingress Controller Fake Certificateとなっていました。この場合、curlコマンドの-kオプションでは証明書の検証に失敗しHTTPS通信ができません、

解消手順

nginx ingressのデフォルト証明書をデプロイされていたdefault/tls-secretに変更することで自己署名証明書を使用する設定にし、curlコマンドの-kオプションでnginx ingressとHTTPSで通信できるように設定しました。

user@team01-prob13-master:~$ cat nginx-ingress/deploy.yaml | grep -i default
            - --default-ssl-certificate=default/tls-secret
user@team01-prob13-master:~$

nginx ingrress -> nginixサーバー間でのHTTPS通信を使用するよう強制

デプロイされているnginx deploymantはポート443へのHTTPS通信のみ受信する設定となっていましたが、ingress -> nginx間はHTTPが使用されるため、Bad requestが返されていました。

user@team35-bastion:~$ curl -k https://192.168.13.101
<html>
<head><title>400 The plain HTTP request was sent to HTTPS port</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<center>The plain HTTP request was sent to HTTPS port</center>
<hr><center>nginx</center>
</body>
</html>

解消手順

ingress.yamlを編集し、nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"を追記して適用することでingress -> nginx間でHTTPS接続が行われるよう設定しました。

user@team01-prob13-master:~$ cat ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/proxy-buffering: "on"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "800"
    kubernetes.io/ingress.allow-http: "false"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
  name: nginx-ingress
spec:
  tls:
  - secretName: tls-secret
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-svc
            port:
              number:  443

上記の解消作業を行い、終了状態を満たしていることを確認しました。

user@team35-bastion:~$ curl -k https://192.168.13.101
welcome to ictsc2021 summer

user@team35-bastion:~$

お手数をおかけしますが、ご確認をお願いいたします。

kstm-su / isctsc2021Summer