search

goharbor / harbor

An open source trusted cloud native registry project that stores, signs, and scans content.
https://goharbor.io
Apache License 2.0
23.6k stars 4.71k forks source link

duplicate key value violates unique constraint "scan_report_digest_registration_uuid_mime_type_key" #20644

Open flbla opened 2 months ago

flbla commented 2 months ago

If you are reporting a problem, please make sure the following information are provided:

Expected behavior and actual behavior:

When a scan is launched (manually or via a push) we randomly get this error :

2024-06-13T14:23:13Z [ERROR] [/controller/event/handler/internal/artifact.go:259]: scan artifact ************************* failed, error: ERROR: duplicate key value violates unique constraint "scan_report_digest_registration_uuid_mime_type_key" (SQLSTATE 23505)

In my postgresql logs I can see :

2024-06-13 14:23:13 UTC-666aff97.4a66-LOG:  QUERY STATISTICS","detail": "! system usage stats:\n!\t0.000146 s user, 0.000000 s system, 0.005479 s elapsed\n!\t[0.315484 s user, 0.050385 s system total]\n!\t20940 kB max resident size\n!\t0/48 [672/864] filesystem blocks in/out\n!\t0/0 [0/1906] page faults/reclaims, 0 [0] swaps\n!\t0 [0] signals rcvd, 0/0 [0/0] messages rcvd/sent\n!\t2/0 [1487/2322] voluntary/involuntary context switches","statement": "commit
2024-06-13 14:23:13 UTC-666aff7a.49cc-ERROR:  duplicate key value violates unique constraint \"scan_report_digest_registration_uuid_mime_type_key\"","detail": "Key (digest, registration_uuid, mime_type)=(sha256:62b692583d8dada8d68dbc996c5e98fd397a5f3263f4149fde95cad749ba416b, ed4df02c-6216-11eb-980a-e2227a954221, application/vnd.security.vulnerability.report; version=1.1) already exists.","statement": "INSERT INTO \"scan_report\" (\"uuid\", \"digest\", \"registration_uuid\", \"mime_type\", \"report\", \"critical_cnt\", \"high_cnt\", \"medium_cnt\", \"low_cnt\", \"unknown_cnt\", \"none_cnt\", \"fixable_cnt\") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12) RETURNING \"id\"","schema_name": "public","table_name": "scan_report","constraint_name": "scan_report_digest_registration_uuid_mime_type_key

harbor core log :

2024-06-13T14:23:25Z [ERROR] [/controller/event/handler/internal/artifact.go:259]: scan artifact *************************  failed, error: a previous scan process is Running

it looks like harbor try to scan the same artifact more than once, but I don't know why.

Steps to reproduce the problem: it's random

Versions: Please specify the versions of following systems.

stonezdj commented 2 months ago

Because there is an interval between the reportMgr.Get and the reportMgr.Create https://github.com/goharbor/harbor/blob/970bdab936d0755467123878a63e8cfb002d8e6b/src/controller/scan/base_controller.go#L547

https://github.com/goharbor/harbor/blob/970bdab936d0755467123878a63e8cfb002d8e6b/src/controller/scan/base_controller.go#L580

If some other user trigger the scan of same digest at this interval, it will cause this issue. To fix this issue, we need to change the reportMgr.Create into InsertOrUpdate

This issue is low impact because user could retrigger the scan.

github-actions[bot] commented 1 day ago

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.