老版本k8s 创建crd失败

cyberchao commented 10 months ago

Ask your question here:

k8s版本：Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.12", GitCommit:"e2a822d9f3c2fdb5c9bfbe64313cf9f657f0a725", GitTreeState:"clean", BuildDate:"2020-05-06T05:09:48Z", GoVersion:"go1.12.17", Compiler:"gc", Platform:"linux/amd64"} loggie版本：v1.4.0

安装方式，没有使用helm install，用的生成yaml文件方式

helm pull https://github.com/loggie-io/installation/releases/download/v1.4.0/loggie-v1.4.0.tgz && tar xvzf loggie-v1.4.0.tgz
helm template loggie loggie --output-dir manifests

直接使用crds.yaml,报错如下

unable to recognize "crds.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1"
unable to recognize "crds.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1"
unable to recognize "crds.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1"
unable to recognize "crds.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1"

apiVersion: apiextensions.k8s.io/v1 修改为 apiVersion: apiextensions.k8s.io/v1beta1 后,报错如下

kubectl apply -f crd.yaml 发生如下报错：error: error validating "crds.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec.versions[0].additionalPrinterColumns[0]): unknown field "jsonPath" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.CustomResourceColumnDefinition, ValidationError(CustomResourceDefinition.spec.versions[0].additionalPrinterColumns[0]): missing required field "JSONPath" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.CustomResourceColumnDefinition, ValidationError(CustomResourceDefinition.spec.versions[0].additionalPrinterColumns[1]): unknown field "jsonPath" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.CustomResourceColumnDefinition, ValidationError(CustomResourceDefinition.spec.versions[0].additionalPrinterColumns[1]): missing required field "JSONPath" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.CustomResourceColumnDefinition, ValidationError(CustomResourceDefinition.spec.versions[0].additionalPrinterColumns[2]): unknown field "jsonPath" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.CustomResourceColumnDefinition, ValidationError(CustomResourceDefinition.spec.versions[0].additionalPrinterColumns[2]): missing required field "JSONPath" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.CustomResourceColumnDefinition, ValidationError(CustomResourceDefinition.spec.versions[0].additionalPrinterColumns[3]): unknown field "jsonPath" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.CustomResourceColumnDefinition, ValidationError(CustomResourceDefinition.spec.versions[0].additionalPrinterColumns[3]): missing required field "JSONPath" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.CustomResourceColumnDefinition]; if you choose to ignore these errors, turn validation off with --validate=false

请问，该如何解决呢

Auggie321 commented 3 months ago

也遇到了类似的问题。 k8s v1.15.6 helm2的环境部署 loggie v1.4.0. 宿主机系统

NAME="BigCloud Enterprise Linux"
VERSION="22.10U2 LTS"
ID="bclinux"
VERSION_ID="22.10U2"
PRETTY_NAME="BigCloud Enterprise Linux For Euler 22.10U2 LTS"
ANSI_COLOR="0;31"

这个和宿主机的系统应该没关系的。

https://loggie-io.github.io/docs/main/getting-started/install/kubernetes/#service 基于quick start的部署介绍，无论是否删除 rm loggie/crds/crds.yaml (实测好几次了) 在使用私有定义lgc文件的时候，也会出现 no match kind of logconfig的内容。

error: unable to recognize "logconfig1.yaml": no matches for kind "LogConfig" in version "loggie.io/v1beta1"

apiVersion: loggie.io/v1beta1
kind: LogConfig
metadata:
  name: nginx-ingress-ingress-nginx-controller
  namespace: loggie
spec:
  selector:
    type: pod
    labelSelector:
      app.kubernetes.io/component: controller
  pipeline:
    sources: |
      - type: file
        name: nginx-ingress-controller
        paths:
        - stdout
    sink: |
      type: dev
      printEvents: true
      codec:
        type: json
        pretty: true

根据之前部署 helm kube-prometheus-stack 的排错印象，这个还是要基于crd.yaml改造兼容。

Auggie321 commented 3 months ago

基于当前k8s 1.15.6适配的版本。调整修改了crd.yaml的配置。

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: clusterlogconfigs.loggie.io
spec:
  group: loggie.io
  names:
    kind: ClusterLogConfig
    listKind: ClusterLogConfigList
    plural: clusterlogconfigs
    shortNames:
      - clgc
    singular: clusterlogconfig
  scope: Cluster
  versions:
  - name: v1beta1
    served: true
    storage: true
  additionalPrinterColumns:
    - description: select pods
      JSONPath: .spec.selector.labelSelector
      name: Pod Selector
      type: string
    - description: select nodes
      JSONPath: .spec.selector.nodeSelector
      name: Node Selector
      type: string
    - description: select cluster
      JSONPath: .spec.selector.cluster
      name: Cluster
      type: string
    - JSONPath: .metadata.creationTimestamp
      name: Age
      type: date
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          description: 'APIVersion defines the versioned schema of this representation
            of an object. Servers should convert recognized schemas to the latest
            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
          type: string
        kind:
          description: 'Kind is a string value representing the REST resource this
            object represents. Servers may infer this from the endpoint the client
            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
          type: string
        metadata:
          type: object
        spec:
          properties:
            pipeline:
              properties:
                interceptorRef:
                  type: string
                interceptors:
                  type: string
                name:
                  type: string
                sink:
                  type: string
                sinkRef:
                  type: string
                sources:
                  type: string
              type: object
            selector:
              properties:
                cluster:
                  type: string
                labelSelector:
                  additionalProperties:
                    type: string
                  type: object
                nodeSelector:
                  additionalProperties:
                    type: string
                  type: object
                type:
                  type: string
              type: object
          type: object
        status:
          properties:
            message:
              properties:
                lastTransitionTime:
                  type: string
                observedGeneration:
                  format: int64
                  type: integer
                reason:
                  type: string
              type: object
          type: object
      required:
      - spec
      type: object
  subresources:
    status: {}

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: logconfigs.loggie.io
spec:
  group: loggie.io
  names:
    kind: LogConfig
    listKind: LogConfigList
    plural: logconfigs
    shortNames:
      - lgc
    singular: logconfig
  scope: Namespaced
  versions:
  - name: v1beta1
    served: true
    storage: true
  additionalPrinterColumns:
    - description: select pods
      JSONPath: .spec.selector.labelSelector
      name: Pod Selector
      type: string
    - JSONPath: .metadata.creationTimestamp
      name: Age
      type: date
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          description: 'APIVersion defines the versioned schema of this representation
            of an object. Servers should convert recognized schemas to the latest
            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
          type: string
        kind:
          description: 'Kind is a string value representing the REST resource this
            object represents. Servers may infer this from the endpoint the client
            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
          type: string
        metadata:
          type: object
        spec:
          properties:
            pipeline:
              properties:
                interceptorRef:
                  type: string
                interceptors:
                  type: string
                name:
                  type: string
                sink:
                  type: string
                sinkRef:
                  type: string
                sources:
                  type: string
              type: object
            selector:
              properties:
                cluster:
                  type: string
                labelSelector:
                  additionalProperties:
                    type: string
                  type: object
                nodeSelector:
                  additionalProperties:
                    type: string
                  type: object
                type:
                  type: string
              type: object
          type: object
        status:
          properties:
            message:
              properties:
                lastTransitionTime:
                  type: string
                observedGeneration:
                  format: int64
                  type: integer
                reason:
                  type: string
              type: object
          type: object
      required:
      - spec
      type: object
  subresources:
    status: {}

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: interceptors.loggie.io
spec:
  group: loggie.io
  names:
    kind: Interceptor
    listKind: InterceptorList
    plural: interceptors
    shortNames:
      - icp
    singular: interceptor
  scope: Cluster
  versions:
  - name: v1beta1
    served: true
    storage: true
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          description: 'APIVersion defines the versioned schema of this representation
            of an object. Servers should convert recognized schemas to the latest
            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
          type: string
        kind:
          description: 'Kind is a string value representing the REST resource this
            object represents. Servers may infer this from the endpoint the client
            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
          type: string
        metadata:
          type: object
        spec:
          properties:
            interceptors:
              type: string
          type: object
      required:
      - spec
      type: object
  conversion:
    strategy: None

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: sinks.loggie.io
spec:
  group: loggie.io
  names:
    kind: Sink
    listKind: SinkList
    plural: sinks
    singular: sink
  scope: Cluster
  versions:
  - name: v1beta1
    served: true
    storage: true
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          description: 'APIVersion defines the versioned schema of this representation
            of an object. Servers should convert recognized schemas to the latest
            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
          type: string
        kind:
          description: 'Kind is a string value representing the REST resource this
            object represents. Servers may infer this from the endpoint the client
            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
          type: string
        metadata:
          type: object
        spec:
          properties:
            sink:
              type: string
          type: object
      required:
      - spec
      type: object
  conversion:
    strategy: None

[root@bcha92 crds]# kubectl apply -f oldk8scrds.yaml 
customresourcedefinition.apiextensions.k8s.io/clusterlogconfigs.loggie.io created
customresourcedefinition.apiextensions.k8s.io/logconfigs.loggie.io created
customresourcedefinition.apiextensions.k8s.io/interceptors.loggie.io created
customresourcedefinition.apiextensions.k8s.io/sinks.loggie.io created

我的nginx-ingress部署在的是defalut,修改了下namespace.

apiVersion: loggie.io/v1beta1
kind: LogConfig
metadata:
  name: nginx-ingress-ingress-nginx-controller
  namespace: default
spec:
  selector:
    type: pod
    labelSelector:
      app.kubernetes.io/component: controller
  pipeline:
    sources: |
      - type: file
        name: nginx-ingress-controller
        paths:
        - stdout
    sink: |
      type: dev
      printEvents: true
      codec:
        type: json
        pretty: true

[root@bcha92 2test_loggie]# kubectl apply -f logconfig1.yaml 
logconfig.loggie.io/nginx-ingress-ingress-nginx-controller created

检查log，正常了

kubectl logs -f --tail=20 -n loggie loggie-k5b8s 
----
2024-07-10 11:23:31 INF pkg/sink/dev/sink.go:100 > event: {
    "fields": {
        "podname": "nginx-ingress-ingress-nginx-controller-65dcc86d88-vs75p",
        "containername": "controller",
        "logconfig": "nginx-ingress-ingress-nginx-controller",
        "namespace": "default",
        "nodename": "10.6.3.92"
    },
    "body": "{\"log\":\"10.6.3.7 - - [10/Jul/2024:03:23:30 +0000] \\\"POST /devicelog HTTP/1.1\\\" 200 7 \\\"-\\\" \\\"python-requests/2.32.3\\\" 7156 0.025 [adbos-adbos-filecollector-80] [] 172.20.0.61:80 7 0.025 200 c619a15608a226e1ae3d7ffa38c83fe6\\n\",\"stream\":\"stdout\",\"time\":\"2024-07-10T03:23:30.480049251Z\"}"
}

解决方案，供大家参考。原则上不管k8s的版本新还是旧，crds.yaml 是不能删除的，否则会影响 loggie 的接口功能。

loggie-io / loggie

老版本k8s 创建crd失败 #659

Ask your question here: