[FEATURE] Add initContainer to generate nVidia Models

[maxirus]

It would be great to have an init container option to auto-generate the models needed, upon startup, for the nVidia GPU detector.

Ideally, this would be configurable in values.yaml and models stored on a PVC. I suspect logic would be required to rebuild the models if the GPU changes (e.g. in a multi-node, multi-GPU cluster). Of course, if models already existed for the given GPU, the generation would be skipped at startup.

example values.yaml section:

gpu:
  nvidia:
    models:
      autoGenerate: false
      useFP16: true
      yoloModels: ""
      persistence:
        enabled: true
        accessMode: ReadWriteOnce
        size: 10Gi
        # existingClaim:

[torsteinelv]

Is this fixed in version 0.13 beta? Do you have a temporary work arraund to use kubernetes and frigate with nvidia detector?

[danotrampus]

As a workaround I did this:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: frigate-model-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: storage-mt
  resources:
    requests:
      storage: 100Mi
---
apiVersion: batch/v1
kind: Job
metadata:
  name: yolo-export-job
spec:
  template:
    spec:
      containers:
      - name: export-container
        image: nvcr.io/nvidia/tensorrt:22.07-py3
        command: ["/bin/bash", "-c"]
        args:
          - |
            wget -O /tensorrt_models.sh https://github.com/blakeblackshear/frigate/raw/master/docker/tensorrt_models.sh
            chmod +x /tensorrt_models.sh
            /tensorrt_models.sh  # Execute the downloaded script
        env:
        - name: YOLO_MODELS
          value: "yolov7-tiny-416"
        - name: USE_FP16
          value: "true"
        volumeMounts:
        - name: frigate-model-volume
          mountPath: /tensorrt_models
        resources:
            limits:
              nvidia.com/gpu: 1
      runtimeClassName: nvidia
      volumes:
      - name: frigate-model-volume
        persistentVolumeClaim:
          claimName: frigate-model-pvc
      restartPolicy: Never

Don't forget to mount the PVC to frigate deployment:

extraVolumes:
- name: frigate-model-volume
  persistentVolumeClaim:
    claimName: frigate-model-pvc
extraVolumeMounts:
- name: frigate-model-volume