vesoft-inc / nebula-operator

Operation utilities for Nebula Graph
https://vesoft-inc.github.io/nebula-operator
Apache License 2.0
81 stars 30 forks source link

backup not support `concurrency` value #456

Closed jinyingsunny closed 9 months ago

jinyingsunny commented 9 months ago

Please check the FAQ documentation before raising an issue on AWS when do backup with sf300(3replica-24part,61亿点+变,单副本大小270G),backup cost nearly 75min. The most time-consuming is the storage data upload, since backup default 5 concurrency,so i try to increase it, however, it report error.

image image

image

with backup file:

apiVersion: apps.nebula-graph.io/v1alpha1
kind: NebulaBackup
metadata:
  name: nb202402282f300-2
  namespace: harris
spec:
  concurrency: "12"
  image: 527027779790.dkr.ecr.us-east-2.amazonaws.com/cloud-dev/br-ent
  version: v3.7.0
  resources:
    limits:
      cpu: "200m"
      memory: 300Mi
    requests:
      cpu: 100m
      memory: 200Mi
  autoRemoveFinished: false
  cleanBackupData: false
  config:
    #concurrency: "12"
    clusterName: nebula
    s3:
      region: "us-west-2"
      bucket: "nebula-br-test2"
      endpoint: "https://s3.us-west-2.amazonaws.com"
      secretName: "aws-secret-ql3"

Your Environments (required)

operator:reg.vesoft-inc.com/cloud-dev/nebula-operator:snap-1.37 br-ent: reg.vesoft-inc.com/cloud-dev/br-ent:v3.7.0 agent: reg.vesoft-inc.com/cloud-dev/nebula-agent:snap nebula: reg.vesoft-inc.com/qa/nebula-graphd-ent:snap-hotfix

Expected behavior

can accept the param concurrency

MegaByte875 commented 9 months ago

https://github.com/vesoft-inc/nebula-operator/pull/457

jinyingsunny commented 9 months ago

snap:1.39, has supported concurrency can be set under config, eg:

apiVersion: apps.nebula-graph.io/v1alpha1
kind: NebulaBackup
metadata:
  name: nb20240229f100-2
  namespace: harris
spec:
  image: 527027779790.dkr.ecr.us-east-2.amazonaws.com/cloud-dev/br-ent
  version: v3.7.0
  resources:
    limits:
      cpu: "200m"
      memory: 300Mi
    requests:
      cpu: 100m
      memory: 200Mi
  autoRemoveFinished: false
  cleanBackupData: false
  config:
    concurrency: 12
    clusterName: nebulasf100
    s3:
      region: "us-west-2"
      bucket: "nebula-br-test2"
      endpoint: "https://s3.us-west-2.amazonaws.com"
      secretName: "aws-secret-ql3"