Yatai customize DNS domain for internal components

[mts-dyt]

Hello bentoml team 👋

My goal is to have a production ready BentoML platform on AZURE, I managed to have these features working:

• external PG database on AZURE (PostGres)
• external docker registry
• external NGINX controller (delete the current one installed by default)
• expose Yatai to custom URL with custom SSL

I still have one painful point concerning the storage. For AZURE, I can't find a suitable option for storing data outside of the cluster (MinIO gateway for blob storage is end of support: https://blog.min.io/deprecation-of-the-minio-gateway/).

If I used the default MinIO installation there are settings which are incompatible with our security policies:

1. MinIO standard installation in yatai-components is installed and exposed on URL like *.apps.yatai.dev
2. Ingress has no SSL

Even if I customize the NGINX setting after the installation, there are still some reference to the old URL MinIO (when we push local bento to Yatai) that lead to error.

Is it possible to customize the domain name (through an env vars for example) for NGINX exposition of internal components and add custom annotation in NGINX exposition?

[yetone]

@mts-dyt Thanks for the feedback!

The default minio component is intended to be used in a test environment, you can specify external S3 during installation so that minio will not be installed.

It is designed to work out of the box, and we automatically generate the domain name and certificate to save users valuable time from having to manually configure it themselves.

If you really want to use our minio installation, you can do so by modifying the yatai-minio ingress resource under the yatai-components namespace, but of course, yatai will remove the logic to automatically install minio and docker-registry later

[mts-dyt]

@yetone , when customizing the Yatai ingress with my own domain without SSL, the bento push command still refers to https:

Error: [cli] `push` failed: request failed with status code 400: {"error":"pre sign s3 upload url: get bucket yatai exist: Get \"https://minio-yatai-infra-cluster.<domain>.io/yatai/?location=\": x509: certificate is valid for ingress.local, not minio-yatai-infra-cluster.lychee.<domain>.io"}

update: This is fixed in v0.3.11 with the annotation yatai.ai/enable-ssl: false

[mts-dyt]

I finally managed to install a MinIO gateway to Azure Blob Storage. As this feature is deprecated by MinIO, here is the last usable version for Azure Gateway:

HELM REPO:    https://charts.min.io/
HELM VERSION: 11.2.15
APP_VERSION:  2022.4.29

Helm custom values should look like:

persistence:
  enabled: false

gateway:
  enabled: true
  type: azure
  replicaCount: 1
  auth:
    azure:
      accessKey: <YATAI_ACCESS_KEY>
      secretKey: <YATAI_SECRET_KEY>
      serviceEndpoint: <STORAGE_ACCOUNT_ENDPOINT>
      storageAccountName: <STORAGE_ACCOUNT_NAME>
      storageAccountKey: <STORAGE_ACCOUNT_ACCESS_KEY>

On my side, this issue can be closed as the new nginx annotation is available: yatai.ai/enable-ssl