GC enhancement engineering story

[wy65701436]

Since Harbor introduces the online GC in v2.1, there are several issues that reported from community about online GC. To provide better use experience, it should do some enhancements on the garbage collection.

• [x] Failure tolerance: let GC continue executing on any blob & manifest deletion (P0)

1. backends #15469
2. GC fails when manifest not found #15822

• [ ] Performance: parallel deletion of blobs can be introduced(P0); And for further action, we can modify Harbor not to land tag into backend storage, so that no need to call v2 API to remove tag from the backend, which is the root cause of performanche problem , especially for large mount of tags associated with one single manifest(P2).

1. GC fails after deleting many tags from same repository #15807
2. [GC performance] The performance of v2 manifest deletion is not good in S3 environment #12948

• [ ] Upstream Distribution Driver issues

1. GC fails with "invalid checksum digest format" from registry when deleting manifests #15970

• [x] Show more items in the GC history.

1. the size was released
2. artifact count was removed.

[wy65701436]

As the failure skip has already merged in, and we should keep parallel deletion PR since we should carafully modify GC code.

For the performance issue, we can move it to 2.6 to discuss more details.

[Grounz]

Hi,

If i've well understood, we have to wait a 2.6.0 harbor release for fix all above issue ?

[wy65701436]

thanks @Grounz for the reminding, we actually did some enhancement on GC. And as for the parallel deletion, I'd like to gather more user data, then we can decide what to do next.

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[dkulchinsky]

still relevant

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[dkulchinsky]

still relevant

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[sidewinder12s]

Not stale

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[dkulchinsky]

definitely not stale

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[dkulchinsky]

not stale

[dkulchinsky]

@wy65701436 I believe https://github.com/goharbor/harbor/issues/18381 should be added to the list above.

Current behaviour when GC encounters an unknown manifest or blob (404 from registry) is to keep retrying until retry timeout reached, the manifest/blob is also not removed from artifact_thrash so it will retry again and again to delete non-existent objects every time GC runs.

Since the goal is to delete the blobs, if they are already gone - then GC should just remove it from artifact_thrash and proceed.

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[sidewinder12s]

Not stale

[wy65701436]

Multiple deletion has been merged, and it will be in the v2.9.

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[dkulchinsky]

not stale

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[dkulchinsky]

not stale

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[dkulchinsky]

not stale

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[dkulchinsky]

not stale

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[sidewinder12s]

no stale

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[dkulchinsky]

not stale

[github-actions[bot]]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[dkulchinsky]

not stale