Closed williamsun-hha closed 3 weeks ago
I do see two issues:
ingressClassName
and that seems to be set to a hostname. However, an ingress class name should refer to an ingress class.Most issues should be visible when you describe the ingress resource...
Thank you very much for your help!
webapprouting.kubernetes.azure.com
is correct ingress class for Azure web app routing ingress. All of other applications are working in this ingress, include the bitnami minio helm chart. Name: myminio
Labels: app.kubernetes.io/managed-by=Helm
Namespace: tenant-ns
Address: 17.17.9.11
Ingress Class: webapprouting.kubernetes.azure.com
Default backend: <default>
TLS:
minio-api-tls-cert terminates
Rules:
Host Path Backends
---- ---- --------
minioapi.xyz.abc.com
/ minio:https-minio (10.244.2.198:9000,10.244.3.176:9000,10.244.0.149:9000 + 1 more...)
Annotations: meta.helm.sh/release-name: tenant
meta.helm.sh/release-namespace: tenant-ns
Events: <none>
). The console ingress has no event either. Since Service Load balancer is working on both api and console, so the backend services are working fine. Bitnami MinIO Console and API ingress is working perfectly on the same AKS cluster as well all of the other cluster.
The ingress looks fine at first sight. Can you post the output of curl -v https://minioapi.xyz.abc.com
? I see that you are using the https-minio
endpoint so the NGINX controller connects to MinIO using TLS too. I can't tell which ingress controller you are using, but you may want to check that too...
Thank you very much for your help, Ramon! I got Client sent an HTTP request to an HTTPS server Response
Sent from Yahoo Mail for iPhone
On Tuesday, October 22, 2024, 12:11 PM, Ramon de Klein @.***> wrote:
The ingress looks fine at first sight. Can you post the output of curl -v https://minioapi.xyz.abc.com? I see that you are using the https-minio endpoint so the NGINX controller connects to MinIO using TLS too. I can't tell which ingress controller you are using, but you may want to check that too...
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
You didn't specify the correct arguments to curl
. It looks like you used http
and https
. Also the returned response seems to be incomplete. Please reply via Github instead of e-mail. Please invoke it as curl -v https://<hostname>
and post the output.
Thank you very much, Ramon! Here is the full message:
GET /? HTTP/2 Host: minioapi.xyz.abc.com User-Agent: curl/8.7.1 Accept: /
GET /? HTTP/2 Host: minioapi.xyz.abc.com User-Agent: curl/8.7.1 Accept: /
It looks like your connection to the Ingress server is fine, but the problem seems to be the connection from the ingress controller to the downstream MinIO server. Your MinIO server seems to be configured to use HTTPS (by default it is), but the ingress controller seems to connect using HTTP. That's probably why it returns Client sent an HTTP request to an HTTPS server.
.
If you are using NGINX, then you need to add the nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
annotation to your ingress controller to ensure that NGINX connects using HTTPS (docs). I'm not sure if NGINX controller will trust the Kubernetes CA by default (I think it will), so you may run into certificate issues, because NGINX controller doesn't trust the MinIO certificate. If you are using another ingress controller, then please check the documentation of the ingress controller on how to connect to the downstream server using HTTPS.
You can add this annotation in the tenant's Helm value as .ingress.api.annotations
, so your values.yaml
should probably read something like this:
ingress:
api:
enabled: true
ingressClassName: "webapprouting.kubernetes.azure.com"
labels: { }
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
tls:
- secretName: minio-api-tls-cert # Reference to the TLS secret for the S3 API
host: minioapi.xyz.abc.com
path: /
pathType: Prefix
console:
enabled: true
ingressClassName: "webapprouting.kubernetes.azure.com"
labels: { }
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
tls:
- secretName: minio-api-tls-cert # Reference to the TLS secret for the S3 API
host: minioconsole.xyz.abc.com
path: /
pathType: Prefix
If you can live without HTTPS within the cluster, then you could also leave it as HTTP and disable HTTPS in MinIO by setting adding the following to the values.yaml
:
tenant:
certificate:
requestAutoCert: false
This will disable certificates (if you don't specify any other certificates) and should allow NGINX to connect via HTTP.
PS: Please reply via Github instead of via email. Replies via email are not formatted and hard to read.
Thank you very much for your help, Ramon! I thought the same thing. As soon as I turn off the Minio TLS, console comes right up. I will check the annotations and test TLS. Really appreciate the help and support! You have a great day!
@cesnietor @cniackz We may want to fix this in our Helm script. The default setting is that MinIO will generate certificates and it looks like this break ingress. We may want to add the proper annotations if the MinIO server uses HTTPS.
We also may want to raise warnings instead of generating the annotations. The annotations are ingress controller specific, so we could assume NGINX, but some may use Traefik instead.
Thank you very much for your help, Ramon! Really appreciate the support!
I did exactly as below: It works.
tenant:
certificate:
requestAutoCert: false
My goal is to have cert-manager to automatically renew the cert on the ingress. I can try to do that from ingress configuration perspective. Do you have a sample config on helm chart values.yaml to share? If not, I will try to get it implemented from ingress perspective.
Thanks again and have a nice day!
Thank you very much for your help, Ramon!
Kubernetes ingress part is configured as following
Expected Behavior
However
The S3 API is working through the service port.
Current Behavior
Possible Solution
Steps to Reproduce (for bugs)
1. 2. 3. 4.
Context
Regression
Your Environment
minio-operator
):uname -a
):