search

goharbor / harbor

An open source trusted cloud native registry project that stores, signs, and scans content.
https://goharbor.io
Apache License 2.0
23.82k stars 4.74k forks source link

Immutability rules issue with character "/" #20102

Open gaglimax opened 7 months ago

gaglimax commented 7 months ago

Expected behavior and actual behavior: When I define an immutability rule excluding several repos containing the charactere "/", the rule should work (this is the only rule enabled) : image However, the rule is not working, no tag is immutable.

Steps to reproduce the problem: In a project, push next images (just the name is important) :

Then, create the rule above and see that you can remove all artifacts whereas redis:latest should be immutable.

Versions:

MinerYang commented 6 months ago

Hi @gaglimax , Could you provide the specific details of your immutable rule?

docker exec -it <harbor-database-container> /bin/bash
psql -d registry
select * from  immutable_tag_rule where project_id=<replace_with_real_project_id>;
gaglimax commented 6 months ago

Hi, Here is the result (the first rule is disabled) :

id   | project_id |                                                                                                                                                                        tag_filter                                                                                                                                                                         | disabled |       creation_time        
-------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+----------------------------
 45625 |       3196 | {"id":0,"project_id":3196,"disabled":false,"priority":0,"action":"immutable","template":"immutable_template","tag_selectors":[{"kind":"doublestar","decoration":"matches","pattern":"**"}],"scope_selectors":{"repository":[{"kind":"doublestar","decoration":"repoMatches","pattern":"**"}]}}                                                            | t        | 2024-01-15 15:19:13.791467
 49873 |       3196 | {"id":49873,"project_id":3196,"disabled":false,"priority":0,"action":"immutable","template":"immutable_template","tag_selectors":[{"kind":"doublestar","decoration":"matches","pattern":"**"}],"scope_selectors":{"repository":[{"kind":"doublestar","decoration":"repoExcludes","pattern":"{argocd,crossplane/crossplane,cloudnative-pg/postgresql}"}]}} | f        | 2024-03-07 10:40:06.002116
(2 rows)
MinerYang commented 6 months ago

Emmm.. it should work as expected.

  1. Could you also get the result of tags table 
    
psql -d registry

\d tag

select * from tag where artifact_id=(select id from artifact where repository_name='repo/redis');

And the result of api call

curl -k -u 'admin:' https:///api/v2.0/projects//repositories/redis/artifacts/latest?with_tag=true&with_immutable_status=true

MinerYang commented 6 months ago

Did you create the rule via API? Could you try to create one through UI?

gaglimax commented 6 months ago

Here it is :

breghr1=> \d tag
                                          Table "public.tag"
    Column     |            Type             | Collation | Nullable |             Default             
---------------+-----------------------------+-----------+----------+---------------------------------
 id            | integer                     |           | not null | nextval('tag_id_seq'::regclass)
 repository_id | integer                     |           | not null | 
 artifact_id   | integer                     |           | not null | 
 name          | character varying(255)      |           | not null | 
 push_time     | timestamp without time zone |           |          | now()
 pull_time     | timestamp without time zone |           |          | 
Indexes:
    "tag_pkey" PRIMARY KEY, btree (id)
    "idx_tag_artifact_id" btree (artifact_id)
    "idx_tag_push_time" btree (push_time)
    "unique_tag" UNIQUE CONSTRAINT, btree (repository_id, name)
Foreign-key constraints:
    "tag_artifact_id_fkey" FOREIGN KEY (artifact_id) REFERENCES artifact(id)
breghr1=> select * from tag where artifact_id=(select id from artifact where repository_name='repo/redis');
   id   | repository_id | artifact_id |     name     |         push_time          | pull_time 
--------+---------------+-------------+--------------+----------------------------+-----------
 257881 |         13330 |      650845 | 7.2.4-alpine | 2024-03-06 10:53:54.485468 | 
(1 row)
$ curl -k -u 'admin' https://<hostname>/api/v2.0/projects/repo/repositories/redis/artifacts/7.2.4-alpine?with_tag=true&with_immutable_status=true
{
    "accessories": null,
    "addition_links": {
        "build_history": {
            "absolute": false,
            "href": "/api/v2.0/projects/repo/repositories/redis/artifacts/sha256:edbb71d676ba3572ce1258255a8ad2971dff7a400fadf0c635cb8d2ebfaa643a/additions/build_history"
        },
        "vulnerabilities": {
            "absolute": false,
            "href": "/api/v2.0/projects/repo/repositories/redis/artifacts/sha256:edbb71d676ba3572ce1258255a8ad2971dff7a400fadf0c635cb8d2ebfaa643a/additions/vulnerabilities"
        }
    },
    "annotations": {
        "org.opencontainers.image.created": "2024-01-09T16:09:57Z",
        "org.opencontainers.image.revision": "b77450d34ae54354f41970fc44bf840353f59ef4",
        "org.opencontainers.image.source": "https://github.com/docker-library/redis.git#b77450d34ae54354f41970fc44bf840353f59ef4:7.2/alpine",
        "org.opencontainers.image.url": "https://hub.docker.com/_/redis",
        "org.opencontainers.image.version": "7.2.4-alpine"
    },
    "digest": "sha256:edbb71d676ba3572ce1258255a8ad2971dff7a400fadf0c635cb8d2ebfaa643a",
    "extra_attrs": {
        "architecture": "amd64",
        "author": "",
        "config": {
            "ArgsEscaped": true,
            "Cmd": [
                "redis-server"
            ],
            "Entrypoint": [
                "docker-entrypoint.sh"
            ],
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "GOSU_VERSION=1.17",
                "REDIS_VERSION=7.2.4",
                "REDIS_DOWNLOAD_URL=http://download.redis.io/releases/redis-7.2.4.tar.gz",
                "REDIS_DOWNLOAD_SHA=8d104c26a154b29fd67d6568b4f375212212ad41e0c2caa3d66480e78dbd3b59"
            ],
            "ExposedPorts": {
                "6379/tcp": {}
            },
            "Volumes": {
                "/data": {}
            },
            "WorkingDir": "/data"
        },
        "created": "2024-01-09T16:09:57Z",
        "os": "linux"
    },
    "icon": "sha256:0048162a053eef4d4ce3fe7518615bef084403614f8bca43b40ae2e762e11e06",
    "id": 650845,
    "labels": null,
    "manifest_media_type": "application/vnd.oci.image.manifest.v1+json",
    "media_type": "application/vnd.oci.image.config.v1+json",
    "project_id": 3196,
    "pull_time": "0001-01-01T00:00:00.000Z",
    "push_time": "2024-03-06T10:53:54.308Z",
    "references": null,
    "repository_id": 13330,
    "size": 17272379,
    "tags": [
        {
            "artifact_id": 650845,
            "id": 257881,
            "immutable": false,
            "name": "7.2.4-alpine",
            "pull_time": "0001-01-01T00:00:00.000Z",
            "push_time": "2024-03-06T10:53:54.485Z",
            "repository_id": 13330
        }
    ],
    "type": "IMAGE"
}

The rule was indeed created by an API call. I just tried to delete it and recreate it though UI, and the result is the same : image

gaglimax commented 5 months ago

Hi @MinerYang,

Any news about this issue ?

MinerYang commented 5 months ago

Hi @gaglimax ,

I can not reproduce the issue accordingly. We may need more contexts to triage.

gaglimax commented 5 months ago

Hi,

I tried to reproduce the issue on a staging Harbor instance and I have the same behaviour. However, I just noticed the issue occurs when there are multiple repos with "/" :

I tried to tailing logs of all pods while creating/activating the rule, but nothing is logged.

MinerYang commented 4 months ago

Any errors message in the harbor-core while you apply this rule? like syntax error in pattern

gaglimax commented 4 months ago

No nothing regarding this issue in harbor-core...

github-actions[bot] commented 2 months ago

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

absynth76 commented 1 month ago

We do have the same issue, following the error in the core logs:

2024-08-21T12:16:44Z [ERROR] [/lib/http/error.go:56]: {"errors":[{"code":"UNKNOWN","message":"launcher: syntax error in pattern"}]}
/harbor/src/pkg/retention/launcher.go:263, github.com/goharbor/harbor/src/pkg/retention.launcherError
/harbor/src/pkg/retention/launcher.go:167, github.com/goharbor/harbor/src/pkg/retention.(*launcher).Launch
/harbor/src/controller/retention/controller.go:235, github.com/goharbor/harbor/src/controller/retention.(*defaultController).TriggerRetentionExec
/harbor/src/server/v2.0/handler/retention.go:262, github.com/goharbor/harbor/src/server/v2.0/handler.(*retentionAPI).TriggerRetentionExecution
/harbor/src/server/v2.0/restapi/configure_harbor.go:3467, github.com/goharbor/harbor/src/server/v2.0/restapi.HandlerAPI.func187
/harbor/src/server/v2.0/restapi/operations/retention/trigger_retention_execution.go:21, github.com/goharbor/harbor/src/server/v2.0/restapi/operations/retention.TriggerRetentionExecutionHandlerFunc.Handle
/harbor/src/server/v2.0/restapi/operations/retention/trigger_retention_execution.go:71, github.com/goharbor/harbor/src/server/v2.0/restapi/operations/retention.(*TriggerRetentionExecution).ServeHTTP
/harbor/src/server/v2.0/restapi/operations/harbor_api.go:1954, github.com/goharbor/harbor/src/server/v2.0/restapi/operations.(*HarborAPI).handler.func1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/middleware/apiversion/api_version.go:29, github.com/goharbor/harbor/src/server/middleware/apiversion.Middleware.func1.1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/router/router.go:92, github.com/goharbor/harbor/src/server/router.(*Route).Handler.func1
/harbor/src/vendor/github.com/beego/beego/v2/server/web/router.go:1149, github.com/beego/beego/v2/server/web.(*ControllerRegister).serveHttp
/harbor/src/vendor/github.com/beego/beego/v2/server/web/filter.go:83, github.com/beego/beego/v2/server/web.(*FilterRouter).filter
/harbor/src/vendor/github.com/beego/beego/v2/server/web/router.go:1002, github.com/beego/beego/v2/server/web.(*ControllerRegister).ServeHTTP
/harbor/src/server/middleware/readonly/readonly.go:77, github.com/goharbor/harbor/src/server/middleware/readonly.MiddlewareWithConfig.func1
/harbor/src/server/middleware/middleware.go:57, github.com/goharbor/harbor/src/server/middleware.New.func1.1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/middleware/security/security.go:75, github.com/goharbor/harbor/src/server/middleware/security.UnauthorizedMiddleware.func1
/harbor/src/server/middleware/middleware.go:57, github.com/goharbor/harbor/src/server/middleware.New.func1.1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/middleware/security/security.go:62, github.com/goharbor/harbor/src/server/middleware/security.Middleware.func1
/harbor/src/server/middleware/middleware.go:57, github.com/goharbor/harbor/src/server/middleware.New.func1.1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/middleware/artifactinfo/artifact_info.go:62, github.com/goharbor/harbor/src/server/middleware/artifactinfo.Middleware.func1.1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/middleware/transaction/transaction.go:60, github.com/goharbor/harbor/src/server/middleware/transaction.Middleware.func1.1
/harbor/src/lib/orm/orm.go:157, github.com/goharbor/harbor/src/lib/orm.WithTransaction.func1
/harbor/src/server/middleware/transaction/transaction.go:69, github.com/goharbor/harbor/src/server/middleware/transaction.Middleware.func1
/harbor/src/server/middleware/middleware.go:57, github.com/goharbor/harbor/src/server/middleware.New.func1.1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/middleware/notification/notification.go:31, github.com/goharbor/harbor/src/server/middleware/notification.Middleware.func1
/harbor/src/server/middleware/middleware.go:57, github.com/goharbor/harbor/src/server/middleware.New.func1.1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/middleware/orm/orm.go:54, github.com/goharbor/harbor/src/server/middleware/orm.MiddlewareWithConfig.func1
/harbor/src/server/middleware/middleware.go:57, github.com/goharbor/harbor/src/server/middleware.New.func1.1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/server/middleware/csrf/csrf.go:48, github.com/goharbor/harbor/src/server/middleware/csrf.attach.func1
/usr/local/go/src/net/http/server.go:2122, net/http.HandlerFunc.ServeHTTP
/harbor/src/vendor/github.com/gorilla/csrf/csrf.go:297, github.com/gorilla/csrf.(*csrf).ServeHTTP
/harbor/src/server/middleware/csrf/csrf.go:68, github.com/goharbor/harbor/src/server/middleware/csrf.Middleware.func2
/harbor/src/server/middleware/middleware.go:57, github.com/goharbor/harbor/src/server/middleware.New.func1.1

I tried by escaping the "/" using "\" w/o success (no error reported but the corresponding repository is not excluded as expected)

absynth76 commented 1 month ago

looks similar issue like mentioned in https://github.com/goharbor/harbor/issues/14259 but we are on 2.8.1 and the ticket stipulates fix is on 2.2