Velero Syncs to Buckets in Regions not specified

[kdnash82]

What steps did you take and what happened: Velero is being used to backup multiple clusters. Cyber reports that they noticed traffic coming from a few of those clusters with a destination to a s3 bucket in other countries. Team member reports that the traffic is coming from velero.

Command Used to Identify network traffic conntrack -E | grep dst=<IP_REPORTED_BY_CYBER>

The output showed syn-ack messages being transmitted between one of the internal cluster IPs and IPs in the /16 CIDR range. Internal Cluster IP was mapped to the Velero pod by running kubectl get pods --all-namespaces --output wide in the cluster.

What did you expect to happen: Expectation was that velero would only backup to the S3 bucket within the region identified in the config. BackupStorageLocation

spec:
  default: true
  objectStorage:
    bucket: <bucket-name>
    prefix: ""
  provider: aws

VolumeSnapshotLocation

spec:
  provider: aws
  config:
    region: us-gov-west-1

The following information will help us better understand what's going on:

Anything else you would like to add: We have multiple clusters with velero, but only certain clusters have this behavior. All are using the same configs with the exception of the bucket name.

Environment:

• Velero version (use velero version): v1.7.0
• Velero features (use velero client config get features): features: <NOT SET>
• Kubernetes version (use kubectl version): v1.21.0
• Kubernetes installer & version:
• Cloud provider or hardware configuration: AWS
• OS (e.g. from /etc/os-release):

  NAME="Ubuntu"
  VERSION="18.04.6 LTS (Bionic Beaver)"
  ID=ubuntu
  ID_LIKE=debian
  PRETTY_NAME="Ubuntu 18.04.6 LTS"
  VERSION_ID="18.04"

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

• :+1: for "I would like to see this bug fixed as soon as possible"
• :-1: for "There are more important bugs to focus on right now"

[kdnash82]

bundle-2022-06-21-15-21-56.tar.gz

[qiuming-best]

@kdnash82 traffic coming from a few of those clusters with a destination to a s3 bucket in other countries

basically, velero will backup files or snapshot volumes and store them in the region configured in BSL / VSL.

do you mean for some specific cluster, the velero stored backups that are not in the region configured in BSL / VSL?

[kdnash82]

@kdnash82 traffic coming from a few of those clusters with a destination to a s3 bucket in other countries

basically, velero will backup files or snapshot volumes and store them in the region configured in BSL / VSL.

do you mean for some specific cluster, the velero stored backups that are not in the region configured in BSL / VSL?

That is correct... We have specified us-gov-west-1 for all Velero backups, but are receiving reports that certain clusters are backing up to that region as well as regions in other countries. This seems to only apply to certain clusters. Not all clusters are behaving this way.