`step-certificates` with Inject Enabled Results in `error opening Badger database: manifest has unsupported version: 7 (we support 4).`

[BenB196]

When trying to deploy step-certificates helm with Inject enabled, I get the following error:

Error opening database of Type badger with source /home/step/db: error opening Badger database: manifest has unsupported version: 7 (we support 4).
Please see https://github.com/dgraph-io/badger/blob/master/README.md#i-see-manifest-has-unsupported-version-x-we-support-y-error on how to fix this.

If I leave Inject disabled I don't get the error.

Here is my values.yaml:

affinity: {}
autocert:
  enabled: false
bootstrap:
  configmaps: true
  enabled: true
  image:
    pullPolicy: IfNotPresent
    repository: cr.step.sm/smallstep/step-ca-bootstrap
    tag: latest
  secrets: true
ca:
  address: ':9000'
  bootstrap:
    postInitHook: ''
  db:
    accessModes:
      - ReadWriteOnce
    enabled: true
    persistent: true
    size: 10Gi
    storageClass: nfsv3
  dns: null
  name: Step Certificates
  password: null
  provisioner:
    name: admin
    password: null
  runAsRoot: false
  url: null
fullnameOverride: ''
image:
  initContainerRepository: 'busybox:latest'
  pullPolicy: IfNotPresent
  repository: cr.step.sm/smallstep/step-ca
  tag: ''
ingress:
  annotations: {}
  enabled: false
  hosts: []
  tls: []
inject:
  certificates:
    certificate_issuer: ''
    intermediate_ca: |
      <snipped>
    root_ca: |
      <snipped>
    ssh_host_ca: ''
    ssh_user_ca: ''
  config:
    files:
      ca.json:
        address: '0.0.0.0:9000'
        authority:
          claims:
            defaultHostSSHCertDuration: 720h
            defaultTLSCertDuration: 24h
            defaultUserSSHCertDuration: 24h
            disableRenewal: false
            maxHostSSHCertDuration: 1680h
            maxTLSCertDuration: 24h
            maxUserSSHCertDuration: 24h
            minHostSSHCertDuration: 5m
            minTLSCertDuration: 5m
            minUserSSHCertDuration: 5m
          provisioners:
            - claims: {}
              forceCN: false
              name: devtest
              type: ACME
        crt: /home/step/certs/intermediate_ca.crt
        db:
          dataSource: /home/step/db
          type: badger
          badgerFileLoadingMode: MemoryMap
        dnsNames:
          - step-ca.example.domain.com
          - step-ca-dev.example.domain.com
          - step-ca.step-ca.svc.cluster.local
          - 127.0.0.1
        federateRoots: []
        key: /home/step/secrets/intermediate_ca_key
        logger:
          format: json
        root: /home/step/certs/root_ca.crt
        tls:
          cipherSuites:
            - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
            - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
            - TLS_AES_128_GCM_SHA256
          maxVersion: 1.3
          minVersion: 1.2
          renegotiation: false
      defaults.json:
        ca-config: /home/step/config/ca.json
        ca-url: 'https://step-ca.example.domain.com:9000'
        fingerprint: <snipped>
        root: /home/step/certs/root_ca.crtt
    templates:
      ssh.tpl: |
        {
          "type": {{ toJson .Type }},
          "keyId": {{ toJson .KeyID }},
          "principals": {{ toJson .Principals }},
          "extensions": {{ toJson .Extensions }},
          "criticalOptions": {{ toJson .CriticalOptions }}
        }
      x509_leaf.tpl: |
        {
          "subject": {{ toJson .Subject }},
          "sans": {{ toJson .SANs }},
        {{- if typeIs "*rsa.PublicKey" .Insecure.CR.PublicKey }}
          "keyUsage": ["keyEncipherment", "digitalSignature"],
        {{- else }}
          "keyUsage": ["digitalSignature"],
        {{- end }}
          "extKeyUsage": ["serverAuth", "clientAuth"]
        }
  enabled: true
  secrets:
    ca_password: <snipped>
    certificate_issuer:
      enabled: false
      key: ''
      password: ''
    provisioner_password: <snipped>
    ssh:
      enabled: false
      host_ca_key: ''
      user_ca_key: ''
    x509:
      enabled: true
      intermediate_ca_key: |
        <snipped>
      root_ca_key: ''
kind: StatefulSet
linkedca:
  secretKeyRef:
    key: null
    name: null
  token: null
nameOverride: ''
nodeSelector: {}
replicaCount: 1
resources: {}
service:
  nodePort: ''
  port: 443
  targetPort: 9000
  type: ClusterIP
tolerations: []

Using chart versionv0.17.2

$kubectl version
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.1", GitCommit:"632ed300f2c34f6d6d15ca4cef3d3c7073412212", GitTreeState:"clean", BuildDate:"2021-08-19T15:45:37Z", GoVersion:"go1.16.7", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.4+rke2r2", GitCommit:"3cce4a82b44f032d0cd1a1790e6d2f5a55d20aae", GitTreeState:"clean", BuildDate:"2021-08-29T22:29:54Z", GoVersion:"go1.16.6b7", Compiler:"gc", Platform:"linux/amd64"}

The persistent volume is an NFSv3 volume provisioned by https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner

I've effectively run out of ideas, as running without Inject enabled, step-certificates starts as expected.

[maraino]

I believe this is because is using a badger v2, can you try to change type: badger to type: badgerV2?

[BenB196]

@maraino that seems to have fixed the issue. Before I close this issue, is this documented somewhere/is this a bug in the chart? When debugging, I didn't find any mention of setting type to badgerV2

[maraino]

@BenB196 Both badger (alias of badgerV1) and badgerV2 are valid options, is it possible that your database was configured before as V2?

[BenB196]

@maraino no, it was a fresh namespace, so nothing existed prior to running the chart.

[maraino]

It's weird because I cannot reproduce this error, but we've changed the default version to be v2, but we didn't do it if you use step ca init --helm we'll fix this on the next release.

[maraino]

Not sure how did this happen but the next version will use badgerv2 by default https://github.com/smallstep/certificates/pull/708