search

vmware-tanzu / velero

Backup and migrate Kubernetes applications and their persistent volumes
https://velero.io
Apache License 2.0
8.42k stars 1.37k forks source link

The log of repo maintain job is duplicated #7741

Open ywk253100 opened 2 months ago

ywk253100 commented 2 months ago 
kubectl -n velero logs default-default-kopia-lc27t-maintain-job-1714114635627-jt5z4
time="2024-04-26T06:57:17Z" level=warning msg="active indexes [xn0_12dfc3e75424a01cab1b6ec26878f577-s57f085408cc12f06128-c1 xn0_416b668dc4a94bb14424fb7bc8ec8bbf-sfdf1fbb94d9d45c5128-c1 xn0_889199a6d9ea6fec3d7226e3b7ee0701-s604d39cd255d8794128-c1 xn0_8ed7a31b860375390e311ce5702055d1-sa776db43884d05a8128-c1 xn0_ee310d060268fed439a131f5eee4baeb-sbaf9b9e2accb51ec128-c1] deletion watermark 0001-01-01 00:00:00 +0000 UTC" logModule=kopia/kopia/format logSource="pkg/kopia/kopia_log.go:101" logger name="[index-blob-manager]" sublevel=error
time="2024-04-26T06:57:17Z" level=info msg="Start to open repo for maintenance, allow index write on load" logSource="pkg/repository/udmrepo/kopialib/lib_repo.go:165"
time="2024-04-26T06:57:17Z" level=warning msg="active indexes [xn0_12dfc3e75424a01cab1b6ec26878f577-s57f085408cc12f06128-c1 xn0_416b668dc4a94bb14424fb7bc8ec8bbf-sfdf1fbb94d9d45c5128-c1 xn0_889199a6d9ea6fec3d7226e3b7ee0701-s604d39cd255d8794128-c1 xn0_8ed7a31b860375390e311ce5702055d1-sa776db43884d05a8128-c1 xn0_ee310d060268fed439a131f5eee4baeb-sbaf9b9e2accb51ec128-c1] deletion watermark 0001-01-01 00:00:00 +0000 UTC" logModule=kopia/kopia/format logSource="pkg/kopia/kopia_log.go:101" logger name="[index-blob-manager]" sublevel=error
time="2024-04-26T06:57:17Z" level=info msg="Succeeded to open repo for maintenance" logSource="pkg/repository/udmrepo/kopialib/lib_repo.go:172"
time="2024-04-26T06:57:17Z" level=info msg="Running full maintenance..." logModule=kopia/maintenance logSource="pkg/kopia/kopia_log.go:94" logger name="[shared-manager]"
time="2024-04-26T06:57:17Z" level=info msg="Running full maintenance..." logModule=kopia/kopia/format logSource="pkg/kopia/kopia_log.go:94" logger name="[shared-manager]"
time="2024-04-26T06:57:17Z" level=warning msg="active indexes [xn0_12dfc3e75424a01cab1b6ec26878f577-s57f085408cc12f06128-c1 xn0_416b668dc4a94bb14424fb7bc8ec8bbf-sfdf1fbb94d9d45c5128-c1 xn0_889199a6d9ea6fec3d7226e3b7ee0701-s604d39cd255d8794128-c1 xn0_8ed7a31b860375390e311ce5702055d1-sa776db43884d05a8128-c1 xn0_ee310d060268fed439a131f5eee4baeb-sbaf9b9e2accb51ec128-c1] deletion watermark 0001-01-01 00:00:00 +0000 UTC" logModule=kopia/kopia/format logSource="pkg/kopia/kopia_log.go:101" logger name="[index-blob-manager]" sublevel=error
time="2024-04-26T06:57:17Z" level=info msg="Looking for active contents..." logModule=kopia/snapshotgc logSource="pkg/kopia/kopia_log.go:94" logger name="[shared-manager]"
time="2024-04-26T06:57:17Z" level=info msg="Looking for active contents..." logModule=kopia/kopia/format logSource="pkg/kopia/kopia_log.go:94" logger name="[shared-manager]"
time="2024-04-26T06:57:17Z" level=info msg="Looking for unreferenced contents..." logModule=kopia/snapshotgc logSource="pkg/kopia/kopia_log.go:94" logger name="[shared-manager]"
time="2024-04-26T06:57:17Z" level=info msg="Looking for unreferenced contents..." logModule=kopia/kopia/format logSource="pkg/kopia/kopia_log.go:94" logger name="[shared-manager]"
time="2024-04-26T06:57:17Z" level=info msg="GC found 0 unused contents (0 B)" logModule=kopia/snapshotgc logSource="pkg/kopia/kopia_log.go:94" logger name="[shared-manager]"
time="2024-04-26T06:57:17Z" level=info msg="GC found 0 unused contents (0 B)" logModule=kopia/kopia/format logSource="pkg/kopia/kopia_log.go:94" logger name="[shared-manager]"
time="2024-04-26T06:57:17Z" level=info msg="GC found 0 unused contents that are too recent to delete (0 B)" logModule=kopia/snapshotgc logSource="pkg/kopia/kopia_log.go:94" logger name="[shared-manager]"
time="2024-04-26T06:57:17Z" level=info msg="GC found 0 unused contents that are too recent to delete (0 B)" logModule=kopia/kopia/format logSource="pkg/kopia/kopia_log.go:94" logger name="[shared-manager]"
time="2024-04-26T06:57:17Z" level=info msg="GC found 17 in-use contents (7.4 MB)" logModule=kopia/snapshotgc logSource="pkg/kopia/kopia_log.go:94" logger name="[shared-manager]"
time="2024-04-26T06:57:17Z" level=info msg="GC found 17 in-use contents (7.4 MB)" logModule=kopia/kopia/format logSource="pkg/kopia/kopia_log.go:94" logger name="[shared-manager]"
...

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

Lyndon-Li commented 1 month ago

Kopia arrange logs by modules, logs from different modules could be written to different target. Velero redirect Kopia logs from all modules to one destination. There is a special case, logs for "Maintenance" module are also written to the the content module ("kopia/Format") after calling AlsoLogToCotentLog inside the maintenance operation. But the "Maintenance" logs before the call only exists in the "Maintenance" log. As a result, in order to include all the logs for maintenance, we have to redirect all the "Maintenance" logs; in order to redirect logs for content, we have to redirect all the logs for "kopia/Format". Since there is an overlap after calling AlsoLogToCotentLog, there will be some duplicated logs for the maintenance job.