Kuby Ingress Error: Could not apply https://raw.githubusercontent.com/kubernetes/ingress-nginx....

getkuby / kuby-core

A convention over configuration approach for deploying Rails apps. https://getkuby.io

MIT License

580 stars 26 forks source link

Kuby Ingress Error: Could not apply https://raw.githubusercontent.com/kubernetes/ingress-nginx.... #94

Closed scart88 closed 2 years ago

scart88 commented 2 years ago

I'm getting this error when we are trying to setup the provider on Digital Ocean.

Kuby config

require 'active_support/core_ext'
require 'active_support/encrypted_configuration'

# Define a production Kuby deploy environment
Kuby.define('KubyApp') do
  environment(:production) do
    # Because the Rails environment isn't always loaded when
    # your Kuby config is loaded, provide access to Rails
    # credentials manually.
    app_creds = ActiveSupport::EncryptedConfiguration.new(
      config_path: File.join('config', 'credentials', 'production.yml.enc'),
      key_path: File.join('config', 'credentials', 'production.key'),
      env_key: 'RAILS_MASTER_KEY',
      raise_if_missing_key: true
    )

    docker do
      # Configure your Docker registry credentials here. Add them to your
      # Rails credentials file by running `bundle exec rake credentials:edit`.
      credentials do
        username app_creds[:KUBY_DOCKER_USERNAME]
        password app_creds[:KUBY_DOCKER_PASSWORD]
        email app_creds[:KUBY_DOCKER_EMAIL]
      end

      # distro :alpine

      # Configure the URL to your Docker image here, eg:
      # image_url 'foo.bar.com/me/myproject'
      #
      # If you're using Gitlab's Docker registry, try something like this:
      image_url 'registry.gitlab.com/user/repo'
    end

    kubernetes do

      provider :digitalocean do
        access_token app_creds[:DIGITALOCEAN_ACCESS_TOKEN]
        cluster_id app_creds[:CLUSTER_ID]
      end

      # Add a plugin that facilitates deploying a Rails app.
      add_plugin :rails_app do
        hostname 'app2.domain.com'
        manage_database false

        env do
          data do
            add "DATABASE_URL", app_creds[:DATABASE_URL]
          end
        end
        # configure database credentials
        # database do
        #   user app_creds[:KUBY_DB_USER]
        #   password app_creds[:KUBY_DB_PASSWORD]
        # end
      end

      # Use Docker Desktop as the provider.
      # See: https://www.docker.com/products/docker-desktop
      #
      # Note: you will likely want to use a different provider when deploying
      # your application into a production environment. To configure a different
      # provider, add the corresponding gem to your gemfile and update the
      # following line according to the provider gem's README.

    end
  end
end

Kuby terminal error on kuby setup

bundle exec kuby -e production setup
Refreshing kubeconfig...
Successfully refreshed kubeconfig!
Deploying nginx ingress resources
Error from server (NotFound): namespaces "ingress-nginx" not found
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
deployment.apps/nginx-ingress-controller created
limitrange/ingress-nginx created
unable to recognize "https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.27.1/deploy/static/mandatory.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.27.1/deploy/static/mandatory.yaml": no matches for kind "Role" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.27.1/deploy/static/mandatory.yaml": no matches for kind "RoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.27.1/deploy/static/mandatory.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
Could not apply https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.27.1/deploy/static/mandatory.yaml: kubectl exited with status code 1
error: Could not apply https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.27.1/deploy/static/mandatory.yaml: kubectl exited with status code 1

Kuby terminal error on kuby deploy

bundle exec kuby -e production deploy
Error from server (NotFound): namespaces "app-production" not found
Validating global resource, namespace 'app-production'
namespace/app-production created (dry run)
Deploying namespace 'app-production'
namespace/app-production created
[INFO][2022-03-11 17:41:57 +0200]
[INFO][2022-03-11 17:41:57 +0200]   ------------------------------------Phase 1: Initializing deploy------------------------------------
[INFO][2022-03-11 17:42:00 +0200]   All required parameters and files are present
[INFO][2022-03-11 17:42:00 +0200]   Discovering resources:
[INFO][2022-03-11 17:42:03 +0200]     - Service/app-assets-svc
[INFO][2022-03-11 17:42:03 +0200]     - Ingress/app-ingress
[INFO][2022-03-11 17:42:03 +0200]     - Deployment/app-assets
[INFO][2022-03-11 17:42:03 +0200]     - ConfigMap/app-config
[INFO][2022-03-11 17:42:03 +0200]     - ConfigMap/app-assets-nginx-config
[INFO][2022-03-11 17:42:03 +0200]     - ServiceAccount/app-assets-sa
[INFO][2022-03-11 17:42:03 +0200]     - Service/app-svc
[INFO][2022-03-11 17:42:03 +0200]     - ClusterIssuer/letsencrypt-production
[INFO][2022-03-11 17:42:03 +0200]     - ServiceAccount/app-sa
[INFO][2022-03-11 17:42:03 +0200]     - Deployment/app-web
[INFO][2022-03-11 17:42:03 +0200]     - Secret/app-secrets
[INFO][2022-03-11 17:42:03 +0200]     - Secret/app-registry-secret
[INFO][2022-03-11 17:42:07 +0200]
[INFO][2022-03-11 17:42:07 +0200]   ------------------------------------------Result: FAILURE-------------------------------------------
[FATAL][2022-03-11 17:42:07 +0200]  Template validation failed
[FATAL][2022-03-11 17:42:07 +0200]
[FATAL][2022-03-11 17:42:07 +0200]  Invalid template: ClusterIssuer-letsencrypt-production20220311-21350-rc7rha.yml
[FATAL][2022-03-11 17:42:07 +0200]  > Error message:
[FATAL][2022-03-11 17:42:07 +0200]      W0311 17:42:03.089014   21380 helpers.go:557] --dry-run is deprecated and can be replaced with --dry-run=client.
[FATAL][2022-03-11 17:42:07 +0200]      error: unable to recognize "/var/folders/hh/z_vjqk3j3dl5vw0whd7h7bp80000gn/T/ClusterIssuer-letsencrypt-production20220311-21350-rc7rha.yml": no matches for kind "ClusterIssuer" in version "cert-manager.io/v1alpha2"
[FATAL][2022-03-11 17:42:07 +0200]  > Template content:
[FATAL][2022-03-11 17:42:07 +0200]      ---
[FATAL][2022-03-11 17:42:07 +0200]      apiVersion: cert-manager.io/v1alpha2
[FATAL][2022-03-11 17:42:07 +0200]      kind: ClusterIssuer
[FATAL][2022-03-11 17:42:07 +0200]      metadata:
[FATAL][2022-03-11 17:42:07 +0200]        name: letsencrypt-production
[FATAL][2022-03-11 17:42:07 +0200]        namespace: cert-manager
[FATAL][2022-03-11 17:42:07 +0200]      spec:
[FATAL][2022-03-11 17:42:07 +0200]        acme:
[FATAL][2022-03-11 17:42:07 +0200]          server: https://acme-v02.api.letsencrypt.org/directory
[FATAL][2022-03-11 17:42:07 +0200]          email: shopandfy@xmail8.com
[FATAL][2022-03-11 17:42:07 +0200]          privateKeySecretRef:
[FATAL][2022-03-11 17:42:07 +0200]            name: letsencrypt-production
[FATAL][2022-03-11 17:42:07 +0200]          solvers:
[FATAL][2022-03-11 17:42:07 +0200]          - http01:
[FATAL][2022-03-11 17:42:07 +0200]              ingress:
[FATAL][2022-03-11 17:42:07 +0200]                class: nginx
[FATAL][2022-03-11 17:42:07 +0200]
[FATAL][2022-03-11 17:42:07 +0200]
[FATAL][2022-03-11 17:42:07 +0200]  Invalid template: Ingress-app-ingress20220311-21350-ht7nke.yml
[FATAL][2022-03-11 17:42:07 +0200]  > Error message:
[FATAL][2022-03-11 17:42:07 +0200]      W0311 17:42:05.241076   21398 helpers.go:557] --dry-run is deprecated and can be replaced with --dry-run=client.
[FATAL][2022-03-11 17:42:07 +0200]      error: unable to recognize "/var/folders/hh/z_vjqk3j3dl5vw0whd7h7bp80000gn/T/Ingress-app-ingress20220311-21350-ht7nke.yml": no matches for kind "Ingress" in version "extensions/v1beta1"
[FATAL][2022-03-11 17:42:07 +0200]  > Template content:
[FATAL][2022-03-11 17:42:07 +0200]      ---
[FATAL][2022-03-11 17:42:07 +0200]      apiVersion: extensions/v1beta1
[FATAL][2022-03-11 17:42:07 +0200]      kind: Ingress
[FATAL][2022-03-11 17:42:07 +0200]      metadata:
[FATAL][2022-03-11 17:42:07 +0200]        annotations:
[FATAL][2022-03-11 17:42:07 +0200]          kubernetes.io/ingress.class: nginx
[FATAL][2022-03-11 17:42:07 +0200]          cert-manager.io/cluster-issuer: letsencrypt-production
[FATAL][2022-03-11 17:42:07 +0200]        name: app-ingress
[FATAL][2022-03-11 17:42:07 +0200]        namespace: app-production
[FATAL][2022-03-11 17:42:07 +0200]      spec:
[FATAL][2022-03-11 17:42:07 +0200]        rules:
[FATAL][2022-03-11 17:42:07 +0200]        - host: app2.domain.com
[FATAL][2022-03-11 17:42:07 +0200]          http:
[FATAL][2022-03-11 17:42:07 +0200]            paths:
[FATAL][2022-03-11 17:42:07 +0200]            - backend:
[FATAL][2022-03-11 17:42:07 +0200]                serviceName: app-svc
[FATAL][2022-03-11 17:42:07 +0200]                servicePort: 8080
[FATAL][2022-03-11 17:42:07 +0200]              path: "/"
[FATAL][2022-03-11 17:42:07 +0200]        - host: app2.domain.com
[FATAL][2022-03-11 17:42:07 +0200]          http:
[FATAL][2022-03-11 17:42:07 +0200]            paths:
[FATAL][2022-03-11 17:42:07 +0200]            - backend:
[FATAL][2022-03-11 17:42:07 +0200]                serviceName: app-assets-svc
[FATAL][2022-03-11 17:42:07 +0200]                servicePort: 8082
[FATAL][2022-03-11 17:42:07 +0200]              path: "/assets"
[FATAL][2022-03-11 17:42:07 +0200]            - backend:
[FATAL][2022-03-11 17:42:07 +0200]                serviceName: app-assets-svc
[FATAL][2022-03-11 17:42:07 +0200]                servicePort: 8082
[FATAL][2022-03-11 17:42:07 +0200]              path: "/packs"
[FATAL][2022-03-11 17:42:07 +0200]        tls:
[FATAL][2022-03-11 17:42:07 +0200]        - hosts:
[FATAL][2022-03-11 17:42:07 +0200]          - app2.domain.com
[FATAL][2022-03-11 17:42:07 +0200]          secretName: app-tls
[FATAL][2022-03-11 17:42:07 +0200]
error: Template validation failed

Is there anything I can do to fix it?

camertron commented 2 years ago

Hey @scart88, thanks for reaching out. I'm 99% sure the problem you're running into is related to your Kubernetes version. At the moment, Kuby only supports Kubernetes v1.19 or earlier. I can't tell from the output alone, but the error message suggests your cluster doesn't support the rbac.authorization.k8s.io/v1beta1 API, which was removed in k8s v1.22. I'm working on supporting later versions, but it's proving to be quite a bit of work.

There is hope however. Because you've set manage_database false in your Kuby config, you can sidestep these problems by using kuby-core@master. In other words, put this in your Gemfile:

gem "kuby-core", github: "getkuby/kuby-core", ref: "9885015027e1ede7fd4f8e53fb7f10772f975845"

scart88 commented 2 years ago

Thank you very much for your fast reply and help. I've updated the gem master version.

I'm unable to select version v.1.19, is not showing in the DigitalOcean dropdown anymore. I can only select v.1.20, v1.21 and v1.22.

Right now, my Kubernetes version is v1.22.7. However, with this gem version, I'm getting the following errors

> bundle exec kuby -e production build
error: undefined method `map' for nil:NilClass

> bundle exec kuby -e production push
error: undefined method `map' for nil:NilClass

Setup seems to work better than before

bundle exec kuby -e production setup
Refreshing kubeconfig...
Successfully refreshed kubeconfig!
Deploying nginx ingress resources
Error from server (NotFound): namespaces "ingress-nginx" not found
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
Nginx ingress resources deployed!
Installing cert-manager...
namespace/cert-manager created
serviceaccount/cert-manager-cainjector created
serviceaccount/cert-manager created
serviceaccount/cert-manager-webhook created
clusterrole.rbac.authorization.k8s.io/cert-manager-webhook:webhook-requester created
clusterrole.rbac.authorization.k8s.io/cert-manager-view created
clusterrole.rbac.authorization.k8s.io/cert-manager-edit created
service/cert-manager created
service/cert-manager-webhook created
deployment.apps/cert-manager-cainjector created
deployment.apps/cert-manager created
deployment.apps/cert-manager-webhook created
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "Role" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "RoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "RoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "Role" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "RoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "MutatingWebhookConfiguration" in version "admissionregistration.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ValidatingWebhookConfiguration" in version "admissionregistration.k8s.io/v1beta1"
Could not apply https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml: kubectl exited with status code 1
error: Could not apply https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml: kubectl exited with status code 1

I will try with v.1.20. Let's see if it works.

denikus commented 2 years ago

Thank you very much for your fast reply and help. I've updated the gem master version.

I'm unable to select version v.1.19, is not showing in the DigitalOcean dropdown anymore. I can only select v.1.20, v1.21 and v1.22.

Right now, my Kubernetes version is v1.22.7. However, with this gem version, I'm getting the following errors

> bundle exec kuby -e production build
error: undefined method `map' for nil:NilClass

> bundle exec kuby -e production push
error: undefined method `map' for nil:NilClass

Setup seems to work better than before

bundle exec kuby -e production setup
Refreshing kubeconfig...
Successfully refreshed kubeconfig!
Deploying nginx ingress resources
Error from server (NotFound): namespaces "ingress-nginx" not found
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
Nginx ingress resources deployed!
Installing cert-manager...
namespace/cert-manager created
serviceaccount/cert-manager-cainjector created
serviceaccount/cert-manager created
serviceaccount/cert-manager-webhook created
clusterrole.rbac.authorization.k8s.io/cert-manager-webhook:webhook-requester created
clusterrole.rbac.authorization.k8s.io/cert-manager-view created
clusterrole.rbac.authorization.k8s.io/cert-manager-edit created
service/cert-manager created
service/cert-manager-webhook created
deployment.apps/cert-manager-cainjector created
deployment.apps/cert-manager created
deployment.apps/cert-manager-webhook created
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "Role" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "RoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "RoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "Role" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "RoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "MutatingWebhookConfiguration" in version "admissionregistration.k8s.io/v1beta1"
unable to recognize "https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml": no matches for kind "ValidatingWebhookConfiguration" in version "admissionregistration.k8s.io/v1beta1"
Could not apply https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml: kubectl exited with status code 1
error: Could not apply https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml: kubectl exited with status code 1

I will try with v.1.20. Let's see if it works.

Did you manage to make it work? Exactly same problem here. Stuck on cert-manager now.

denikus commented 2 years ago

I've downgraded Kubernetes cluster to version 1.20 and it helps.

scart88 commented 2 years ago

Thanks. Setup bundle exec kuby -e production setup command works on v1.20. However, the deploy command is getting stuck at -create-db on my end.

denikus commented 2 years ago

Thanks. Setup bundle exec kuby -e production setup command works on v1.20. However, the deploy command is getting stuck at -create-db on my end.

Exactly same problem, fighting it right now. Let me know if you solve it, please.

scart88 commented 2 years ago

Not yet, but I found this relevant issue #74. I'm kind of stuck, so I took a pause on trying k8s. Please let me know if you find a way to make it work.

camertron commented 2 years ago

Hey @denikus and @scart88, sorry for all the trouble. Getting all of these things to work together is quite a challenge.

Yes, it looks like v1.20 should work with kuby-core master as well 😄

@scart88 it sounds like Rails isn't able to connect to the database specified in your database.yml. Even though manage_database false may be set in your Kuby config, Kuby will still try to create and/or migrate your database automatically. Try running the following command to get a list of running pods:

bundle exec kuby -e production remote status

There should be one pod in the Init:CrashLoopBackOff state. Copy the name of the pod and run the following command to print out the logs from the create-db init container:

bundle exec kuby -e production kubectl -N -- logs -c app-create-db <pod name here>

(This should be a lot easier, I'll file an issue)

scart88 commented 2 years ago

Unfortunately it didn't worked.

NAME                                           READY   STATUS                  RESTARTS   AGE
cm-acme-http-solver-stpkj                      1/1     Running                 0          50s
hotwiresample-assets-77cbcb497c-kq6pg   1/1     Running                 0          4m1s
hotwiresample-web-74df7f7d9b-ngkgh      0/1     Init:CrashLoopBackOff   3          4m1s
❯ bundle exec kuby -e production kubectl -N -- logs -c app-create-db hotwiresample-web-74df7f7d9b-ngkgh

rake aborted!
ActiveRecord::ConnectionNotEstablished: could not connect to server: Connection timed out
        Is the server running on host "app-psql-do-user-------0.b.db.ondigitalocean.com" (0.0.0.0) and accepting
        TCP/IP connections on port 25060?
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:83:in `rescue in new_client'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:77:in `new_client'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:37:in `postgresql_connection'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:882:in `public_send'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:882:in `new_connection'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:926:in `checkout_new_connection'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:905:in `try_to_checkout_new_connection'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:866:in `acquire_connection'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:588:in `checkout'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:428:in `connection'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:1128:in `retrieve_connection'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_handling.rb:327:in `retrieve_connection'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_handling.rb:283:in `connection'
/usr/local/bundle/bundler/gems/kuby-core-9885015027e1/lib/kuby/plugins/rails_app/tasks.rake:23:in `block (4 levels) in <main>'
/usr/local/bundle/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/cli/exec.rb:58:in `load'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/cli/exec.rb:58:in `kernel_load'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/cli/exec.rb:23:in `run'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/cli.rb:483:in `exec'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/cli.rb:31:in `dispatch'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/cli.rb:25:in `start'
/usr/local/bundle/gems/bundler-2.3.9/exe/bundle:48:in `block in <top (required)>'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/friendly_errors.rb:103:in `with_friendly_errors'
/usr/local/bundle/gems/bundler-2.3.9/exe/bundle:36:in `<top (required)>'
/usr/src/app/bin/bundle:113:in `load'
/usr/src/app/bin/bundle:113:in `<main>'

Caused by:
PG::ConnectionBad: could not connect to server: Connection timed out
        Is the server running on host "app-psql-do-user-------0.b.db.ondigitalocean.com" (0.0.0.0) and accepting
        TCP/IP connections on port 25060?
/usr/local/bundle/gems/pg-1.2.3/lib/pg.rb:58:in `initialize'
/usr/local/bundle/gems/pg-1.2.3/lib/pg.rb:58:in `new'
/usr/local/bundle/gems/pg-1.2.3/lib/pg.rb:58:in `connect'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:78:in `new_client'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/postgresql_adapter.rb:37:in `postgresql_connection'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:882:in `public_send'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:882:in `new_connection'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:926:in `checkout_new_connection'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:905:in `try_to_checkout_new_connection'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:866:in `acquire_connection'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:588:in `checkout'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:428:in `connection'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:1128:in `retrieve_connection'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_handling.rb:327:in `retrieve_connection'
/usr/local/bundle/gems/activerecord-6.1.5/lib/active_record/connection_handling.rb:283:in `connection'
/usr/local/bundle/bundler/gems/kuby-core-9885015027e1/lib/kuby/plugins/rails_app/tasks.rake:23:in `block (4 levels) in <main>'
/usr/local/bundle/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/cli/exec.rb:58:in `load'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/cli/exec.rb:58:in `kernel_load'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/cli/exec.rb:23:in `run'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/cli.rb:483:in `exec'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/cli.rb:31:in `dispatch'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/cli.rb:25:in `start'
/usr/local/bundle/gems/bundler-2.3.9/exe/bundle:48:in `block in <top (required)>'
/usr/local/bundle/gems/bundler-2.3.9/lib/bundler/friendly_errors.rb:103:in `with_friendly_errors'
/usr/local/bundle/gems/bundler-2.3.9/exe/bundle:36:in `<top (required)>'
/usr/src/app/bin/bundle:113:in `load'
/usr/src/app/bin/bundle:113:in `<main>'
Tasks: TOP => kuby:rails_app:db:create_unless_exists
(See full trace by running task with --trace)

Update 1

I added the cluster to verified sources to the DO managed database as well and it worked.

❯ bundle exec kuby -e production remote status

NAME                                           READY   STATUS    RESTARTS   AGE
cm-acme-http-solver-stpkj                      1/1     Running   0          16m
hotwiresample-assets-77cbcb497c-kq6pg   1/1     Running   0          20m
hotwiresample-web-74df7f7d9b-ngkgh      1/1     Running   0          20m

I pointed the Load Balancer to the A records on subdomain and the only issue now is the SSL certificate. Your connection is not private

Update 2

It took some time, and finally the SSL certificate did his thing and now it works.

Considering it took so much time to figure out how to make this to work. I'm a little afraid to go in production with this. I don't want to abuse the issues and your time every time I get an error.

Do you have any recommendations on what would be my next steps in learning more? Do I have to worry about rollbacks, log rotation, backups (I already have backups on the DO managed db), cron jobs ?

camertron commented 2 years ago

Hey @scart88, glad you were able to get things working!

It took some time, and finally the SSL certificate did his thing and now it works.

Yeah, sometimes certs are slow to issue because of DNS propagation lag. Unfortunately I don't know of any workarounds for that, but 1) usually it's just a matter of waiting for a bit, and 2) you should only have to worry about it once.

Considering it took so much time to figure out how to make this to work. I'm a little afraid to go in production with this. I don't want to abuse the issues and your time every time I get an error.

Yes, I totally understand. Kuby is pre-1.0 software, and I'm still actively working out the kinks. I really appreciate you taking the time to give it a try and report your experience back to me. It's super helpful to see where folks are getting tripped up.

Do you have any recommendations on what would be my next steps in learning more?

That's a good question. What specifically are you interested in learning more about? Kuby uses Docker and Kubernetes under the hood. I'd be happy to point you at some resources for either or both of those things. A good place to start for now might be to run kuby dockerfiles and kuby resources. Those will print out the Dockerfiles and Kubernetes resource manifests Kuby will use to build and deploy your app. They might not make sense right away, but it's good to know what parts of Docker and k8s Kuby is using so you can narrow your focus.

Do I have to worry about rollbacks, log rotation, backups (I already have backups on the DO managed db), cron jobs?

Rollbacks are baked into Kuby. You should be able to run kuby rollback to deploy the previous release.

In Kubernetes, logs are automatically sent to STDOUT. I believe k8s will keep a certain number of lines around if you want to see log output later, but other than that logs are simply discarded. So no log rotation is necessary as far as I know. If you're interested in looking at old logs, I would recommend a log aggregation service like SumoLogic, etc.

You don't have to worry about backups unless you're writing to the local disk for some reason. If you are, I believe DigitalOcean has a backup feature you could look into.

I'm not sure what you mean about cron jobs? Kubernetes has a cron feature that Kuby doesn't expose.

scart88 commented 2 years ago

Yeah, sometimes certs are slow to issue because of DNS propagation lag. Unfortunately I don't know of any workarounds for that, but 1) usually it's just a matter of waiting for a bit, and 2) you should only have to worry about it once.

I don't much about the Kubernetes ingress, but I found it so easy to run the caddy docker file and do a reverse proxy like in the example below. However, I guess is much more complicated in Kubernetes.

docker run -d --name p-caddy --restart always --network host -v /home/Caddyfile:/etc/caddy/Caddyfile -v /home/caddy-data:/data caddy

 Caddyfile

example-domain.com {
    reverse_proxy 0.0.0.0:3000
}

Yes, I totally understand. Kuby is pre-1.0 software, and I'm still actively working out the kinks. I really appreciate you taking the time to give it a try and report your experience back to me. It's super helpful to see where folks are getting tripped up.

I will try to help if I can, however my Rails / Ruby experience is pretty basic. I'm really glad you built this and I have the chance to play with it.

That's a good question. What specifically are you interested in learning more about? Kuby uses Docker and Kubernetes under the hood. I'd be happy to point you at some resources for either or both of those things. A good place to start for now might be to run kuby dockerfiles and kuby resources. Those will print out the Dockerfiles and Kubernetes resource manifests Kuby will use to build and deploy your app. They might not make sense right away, but it's good to know what parts of Docker and k8s Kuby is using so you can narrow your focus.

Thanks. I got a basic docker course from Nick Janetakis, however didn't had much time to go through everything.

Rollbacks are baked into Kuby. You should be able to run kuby rollback to deploy the previous release.

That handy. I notice on every kuby build, the registry on Gitlab is growing bigger. Should I remove old images?

I'm not sure what you mean about cron jobs? Kubernetes has a cron feature that Kuby doesn't expose.

I will have to play with the whenever gem and see if it works.

denikus commented 2 years ago

@camertron maybe you can help. Seems like Puma just crashing for some db reason. No details even in pods' logs.

[25] Early termination of worker [23] ! Unable to start worker [23] /usr/local/bundle/gems/activerecord-7.0.2.2/lib/active_record/enum.rb:301:inraise_conflict_error' [23] Early termination of worker `

What could it possibly be?

Database in separate db cluster and connection open.

denikus commented 2 years ago

Solved. Problem was in DATABASE_URL and how I send it to database.yml.

scart88 commented 2 years ago

Yeah, sometimes certs are slow to issue because of DNS propagation lag. Unfortunately I don't know of any workarounds for that, but 1) usually it's just a matter of waiting for a bit, and 2) you should only have to worry about it once.

I found that it works faster with kind: Issuer (about 1 or 2 minutes), however it must live in the same namespace with the App / Service / Deployment / Pod

This is on DO with a Load Balancer and ingress-nginx helm

---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: letsencrypt-nginx
  namespace: app-production-namespace <--- the issuer is is the same namespace
spec:
  # ACME issuer configuration
  # `email` - the email address to be associated with the ACME account (make sure it's a valid one)
  # `server` - the URL used to access the ACME server’s directory endpoint
  # `privateKeySecretRef` - Kubernetes Secret to store the automatically generated ACME account private key
  acme:
    email: email...
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-nginx-private-key
    solvers:
      # Use the HTTP-01 challenge provider
      - http01:
          ingress:
            class: nginx

apiVersion: v1
kind: Service
metadata:
  name: app-production-service
  namespace: app-production-namespace
spec:
  selector:
    app: app-production
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080   <--- the ingress is targeting this port

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-production-ingress
  namespace: app-production-namespace
  annotations:
    cert-manager.io/issuer: letsencrypt-nginx
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - example.com
    secretName: letsencrypt-nginx
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: app-production-service
            port:
              number: 8080

JorgenIvarsson commented 2 years ago

I have the same problem on Linode. Error when running bundle exec kuby -e production setup. I get the "services ingress-nginx not found"-error. I don't seem to be able to downgrade Kubernetes to a lower version than 1.22 on Linode. As I understand it, there's no workaround for this scenario?

camertron commented 2 years ago

Hey @JorgenIvarsson, thanks for your comment. I'm getting ready to publish a big Kuby release ahead of my talk at RailsConf that should address this issue (and a bunch of others). If you can wait a week or so, the upcoming release should unblock you.

JorgenIvarsson commented 2 years ago

Greate news @camertron! As a new Rails user I am struggling with how deploy my first app. Learning Rails, and at the same time having to learn Docker and Kubernetes is just too much. Just the amount of configuration files blows my mind! My last hope to deploy my app is getting Kuby to work. I will be waiting for the update :-)

camertron commented 2 years ago

@JorgenIvarsson yes, it's quite a lot to learn and manage. There are a number of other deployment solutions out there that might work for you - have you considered Heroku for example?

scart88 commented 2 years ago

@JorgenIvarsson, if Heroku is not an option, you can also check Hatchbox, developed by Chris Oliver from GoRails. It's far easier to get started as a new Rails developer than something like Kubernetes.

JorgenIvarsson commented 2 years ago

Thank you for your helpful suggestions @camertron and @scart88 . I am familiar with these services. I have tested Heroku, but not yet Hatchbox.

Heroku seems to be the suggested solution in 90% in all discussions I have read. The other 10% seem to be varying degrees of complicated setup of a VM and then using Capistrano with application servers such as Puma/Passenger/Unicorn together with Postgresql on that VM. So, I must choose between Heroku or a lot of tinkering (If I don’t want to pay Hatchbox to take care of the tinkering).

But what I want is a solution where I can follow along when DHH builds his famous demo blog, and where I can continue the demo with a deployment that is as easy as it was creating the blog.

Do you remember when you first saw Rail's scaffolding? Pure magic!

I want the deployment of an app to give the same feeling of magic (But using industry standard solutions like Docker and Kubernetes) In short, Active Deployment.

Do you think that Kuby could be that solution for a developer of small and simple rails apps (typically a monolithic rails app + redis + postgresql )? Or should I look elsewhere?

camertron commented 2 years ago

But what I want is a solution where I can follow along when DHH builds his famous demo blog, and where I can continue the demo with a deployment that is as easy as it was creating the blog.

Me too!

Do you think that Kuby could be that solution for a developer of small and simple rails apps (typically a monolithic rails app + redis + postgresql )? Or should I look elsewhere?

That's exactly what I want Kuby to be. It's amazing to me that a large number of Rails developers are still using Capistrano in 2022. Don't get me wrong, Capistrano is a great tool, but I believe there are better ways to deploy apps these days.

camertron commented 2 years ago

Hey @JorgenIvarsson, good news! At the end of last week I released the promised big update. Let me know if it works better for you :) Here's the changelog entry if you're curious.

JorgenIvarsson commented 2 years ago

Thanks for the info @camertron! But I am following this project closely and have already upgraded to the new version. I was (and still is) unsure of the best way to upgrade the gem files and the kuby core. I think that I got it right, but it would be nice to have an instruction on how to upgrade from one version to the next. Anyhow, when I continued the instruction I encountered another problem. When I ran bundle exec kuby -e production setup I got ..Refreshing kubeconfig... And then the error: 404 Not Found: NOT FOUND. I have no clue what this means. I know I have been follow along a guide on Kubernetes, using googles gcloud tool and kubectl. Maybe this has altered something in the kubeconf? Anyway I am going to start over with a completely new app and see if I succeed with a fresh project (and credentials). If that doesn't work I will look in to this kubeconfig-file and see if there is something wrong or missing in that file.

camertron commented 2 years ago

I am following this project closely and have already upgraded to the new version.

🎉 🎉

I was (and still is) unsure of the best way to upgrade the gem files and the kuby core.

That's good feedback, Kuby does need to have a better upgrade story. I'll have an answer for that before v1.0 (whenever that is).

When I ran bundle exec kuby -e production setup I got ..Refreshing kubeconfig... And then the error: 404 Not Found: NOT FOUND.

Hmm, did Kuby print any message saying it had successfully refreshed the kubeconfig? If not, then it sounds like a problem communicating with the cloud provider. Which cloud provider are you using, and are your credentials correct? Kuby doesn't support Google k8s yet. In any case, sounds like an opportunity for a better error message.

JorgenIvarsson commented 2 years ago

Thanks for the tip @camertron! Yes, it turns out when I left off I hadn't fully configured the kubernets section. But once I manage to deploy i got a 404 nginx error message. Tried both with Linode and DigitalOcean. My app did not use a database and maybe that caused some problems?

Anyhow, I created a new simple app from scratch, this time using a database (just in case using a database is a prerequisite). But when I run "bundle exec kuby -e production build" i get an error: Kuby does not support the 'postgresql' database adapter. This error message really surprised me. I thought that Postgresql was one of the most popular databases in the Rails community? Or is it that there is more than one database adapter for postgres, and I am using the wrong one?

camertron commented 2 years ago

But once I manage to deploy i got a 404 nginx error message. Tried both with Linode and DigitalOcean. My app did not use a database and maybe that caused some problems?

Hmm interesting. How were you attempting to make requests to the app? Was there TLS involved? Enabling/disabling the managed database feature shouldn't be able to cause a connectivity issue like that.

This error message really surprised me. I thought that Postgresql was one of the most popular databases in the Rails community? Or is it that there is more than one database adapter for postgres, and I am using the wrong one?

I can't believe this, but I forgot to document the database changes from the previous Kuby release 🤦 Kuby used to support both Postgres and MySQL, but now only supports CockroachDB. There are a lot of good reasons for this, one of the biggest being that Cockroach is wire-compatible with Postgres. In other words they're both almost 100% compatible with each other. More importantly however, Cockroach has been designed to run in the cloud. You'll need to add the cockroach-activerecord-adapter to your Gemfile and set the adapter to cockroachdb in your database.yml. I will update the docs right now.

JorgenIvarsson commented 2 years ago

Yes, that was strange and I have no explanation for not being able to connect to the app. I could l connect to the app through the command "bundle exec kuby -e production remote shell". I could even spin up the puma server with "Rails server". I did not configure a domain name. Instead I tried to connect via the public ip-adress provided by the load balancer. So on one hand the app was running via the shell and I could also access the log files. And on the other hand I could not connet via ip-adress in the browser or curl. All I got was a 404 error message from nginx. Here is the docker image I used: https://hub.docker.com/repository/docker/jorgenivarsson/kuby

So now I have a created a new fresh app that uses a database (Postgresql). If I want to use Kuby, do I have to switch to cockroach completely or can I use postgres as a development database and only use cockroach as a production database in my database.yml? Maybe load the gem file for cockroach under a production group? I'm not sure what happens when the app is dockerized? Is the database actually involved in the dockerization/kubernetization of the app or is only the database schema and migrations used? The reason I ask is that I already have mysql/mariadb and postgres installed on my computer. I rather not install (or learn) yet another database if I can avoid it. I understand the reasons for choosing Cockroach. But the demands for using a specific database complicates things if there is no simple way of switching from say Postgres to Cockroach. Or is Cockroach a new default for all cloud based applications so it is expected that you use it together with Kubernetes from the start when developing a new application? Anyway, thanks for your help and patience @camertron!

UPDATE: It seems that cockroach-activerecord-adapter don't work with Rails 7. The gem won't install in a Rails 7 project. An issue asking for Rails 7 support was published on Januari 1 and is still today not fixed: https://github.com/cockroachdb/activerecord-cockroachdb-adapter/issues/239

camertron commented 2 years ago

Hey @JorgenIvarsson, apologies for the delayed response.

If I want to use Kuby, do I have to switch to cockroach completely or can I use postgres as a development database and only use cockroach as a production database in my database.yml?

Yes, what you've described is absolutely possible. Kuby doesn't offer a solution for development environments at the moment.

Maybe load the gem file for cockroach under a production group?

Yes, that's what I'd suggest 😄

Is the database actually involved in the dockerization/kubernetization of the app or is only the database schema and migrations used?

The schema and migrations are part of your app and are copied into the Docker image, and the CockroachDB adapter gem is installed inside the image as well. Other than that, the choice of database isn't considered during the Docker build.

Kuby instructs Kubernetes to launch several CockroachDB pods (older versions launched Postgres or MySQL pods). You can tell Kuby to not create any database pods (for example, if you're using a managed database offering from your cloud provider) by setting manage_database false as documented here.

I rather not install (or learn) yet another database if I can avoid it.

Right, I understand. Cockroach is very similar to Postgres from what I understand, so most of your existing knowledge should transfer easily.

Or is Cockroach a new default for all cloud based applications so it is expected that you use it together with Kubernetes from the start when developing a new application?

It's not necessarily expected by the industry at large; most companies still use MySQL or Postgres I believe. I chose it for Kuby because it's specifically designed for the cloud. As the cool kids say, it's "cloud native" and much easier to upgrade in-place. That's actually one of the biggest reasons Kuby doesn't have a good upgrade story right now, because MySQL and Postgres are difficult to upgrade in-place and without incurring downtime.

It seems that cockroach-activerecord-adapter don't work with Rails 7.

Oh no, that's not good! Thanks for letting me know 😓