bareos / bareos

Bareos is a cross-network Open Source backup solution (licensed under AGPLv3) which preserves, archives, and recovers data from all major operating systems.
http://www.bareos.org
Other
969 stars 262 forks source link

Consolidation job ignores Next/Full Backup Pool #1928

Open EBCEEB opened 1 month ago

EBCEEB commented 1 month ago

Bareos component version

23.0.3

Steps to reproduce

I have two storages for copy jobs, and two pools on each storage for consolidation jobs.

``` Storage { Name = storage-1 Address = 10.x.x.11 Password = *** Device = FileStorage1 Device = FileStorage2 Device = FileStorage3 Device = FileStorage4 Media Type = storage-1 Maximum Concurrent Jobs = 4 } Pool { Name = pool-1-1 Next Pool = pool-2-1 Storage = storage-1 } Pool { Name = pool-1-2 Next Pool = pool-2-2 Storage = storage-1 ``` ``` Storage { Name = storage-2 Address = 10.x.x.21 Password = *** Device = FileStorage1 Device = FileStorage2 Device = FileStorage3 Device = FileStorage4 Media Type = storage-2 Maximum Concurrent Jobs = 4 } Pool { Name = pool-2-1 Next Pool = pool-1-1 Storage = storage-2 } Pool { Name = pool-2-2 Next Pool = pool-1-2 Storage = storage-2 ```

Next pools are defined as shown above for copy jobs, so I'm trying to define pools for consolidation job explicitly:

Job {
  Name = "Artifactory Backup"
  Type = Backup
  Client = atrifactory
  File Set = Artifactory

  Pool = pool-1-1
  Next Pool = pool-1-2
  Full Backup Pool =  pool-1-2

  Schedule = Daily
  Always Incremental = yes
  Accurate = yes
  Messages = Standard
}
Job {
  Name = "Artifactory-Consolidate"
  Type = "Consolidate"
  Client = atrifactory
  File Set = Artifactory

  Pool = pool-1-1
  Next Pool = pool-1-2
  Full Backup Pool = pool-1-2

  Schedule = Daily
  Priority = 100
  Accurate = yes
  Messages = Standard
}

Expected results

Successful consolidation + virtual full jobs reading from pool-1-1 and writing to pool-1-2

Actual results

Vitrual full job fails with error:

Director JobId 999: Fatal error:
Storage daemon didn't accept Device "FileStorage4" because:
3924 Device "FileStorage4" not in SD Device resources or no matching Media Type.

Job summary has line

Pool: "pool-2-1" (From Job Pool's NextPool resource)

Looks like Next Pool and Full Backup Pool defined in jobs are ignored.

Environment

- OS: `linux`
- component: Bareos dir, sd, fd = 23.0.3

Relevant log output

No response

Relevant traces output

No response

Anything else?

Manually configured virtual full job having

  Pool = pool-1-1
  Next Pool = pool-1-2

works fine

sebsura commented 1 month ago

Consolidation jobs are just administration jobs. The pool related configuration is taken from the job that is run as a virtualfull job.

You need to set the VirtualFullBackupPool configuration option in Artifactory Backup to achieve what you want to achieve.

If this does not work for you, feel free to reopen this issue.

EBCEEB commented 1 month ago

I've added Virtual Full Backup Pool = pool-1-2 to job Artifactory Backup, and got the same error.

I didn't new full/incremental backups (just updated job Artifactory Backup, reloaded config, and ran Artifactory-Consolidate) - is it matters?

EBCEEB commented 1 month ago

Tried one more time only having Next Pool = pool-1-2 in Artifactory Backup and it works.

I think you may close the issue. Looks like I missed something.

Sorry.

sebsura commented 1 month ago

ill leave this open as we actually found a bug: Currently it seems that the VirtuafFullBackupPool option of the Job resources aer getting ignored. Ill push a pr later today that will fix this and then this issue will get auto closed once the pr is merged

EBCEEB commented 1 month ago

Looks like Next Pool should work always:

https://github.com/bareos/bareos/blob/bdb86e83b4736033956f9d6ef1d1498cf43c0a40/core/src/dird/vbackup.cc#L171

EBCEEB commented 1 month ago

BTW it works even if Pool and Next Pool are the same. So there is no need to define an additional pool for consolidation jobs.