search

moooofly / harborctl

A CLI tool for the Docker Registry Harbor. This project offer a command-line interface to the Harbor API, you can use it to manager your users, projects, repositories, etc. (This project aims to refactor harbor-go-client based on Harbor v1.6.0)
MIT License
28 stars 6 forks source link

[NOTE] Notes on "POST /policies/replication" API #30

Open moooofly opened 5 years ago

moooofly commented 5 years ago

This is a very complicated API.

by swagger file

image image

The content body of this API consist of so many parameters. But which of them is required?

by Harbor UI

image

It seems that only three of them are really required

moooofly commented 5 years ago

Case 1

image

{
  "name": "sync-to-somewhere",
  "description": "",
  "trigger": {
    "kind": "Manual"
  },
  "replicate_existing_image_now": true,
  "replicate_deletion": false,
  "filters": [],
  "projects": [
    {
      "project_id": 10,
      "owner_id": 1,
      "name": "temp_9",
      "creation_time": "2018-11-02T06:58:29Z",
      "update_time": "2018-11-02T06:58:29Z",
      "deleted": false,
      "owner_name": "",
      "togglable": true,
      "current_user_role_id": 1,
      "repo_count": 1,
      "metadata": {
        "public": "true"
      }
    }
  ],
  "targets": [
    {
      "id": 1,
      "endpoint": "http://11.11.11.22",
      "name": "fake endpoint",
      "username": "admin",
      "password": "",
      "type": 0,
      "insecure": true,
      "creation_time": "2018-11-07T14:42:31.876842Z",
      "update_time": "2018-11-07T14:42:31.876842Z"
    }
  ]
}
moooofly commented 5 years ago

Case 2

image

{
  "name": "sync-again",
  "description": "",
  "trigger": {
    "kind": "Immediate"
  },
  "replicate_existing_image_now": true,
  "replicate_deletion": false,
  "filters": [],
  "projects": [
    {
      "project_id": 9,
      "owner_id": 1,
      "name": "temp_8",
      "creation_time": "2018-11-02T06:58:25Z",
      "update_time": "2018-11-02T06:58:25Z",
      "deleted": false,
      "owner_name": "",
      "togglable": true,
      "current_user_role_id": 1,
      "repo_count": 1,
      "metadata": {
        "public": "true"
      }
    }
  ],
  "targets": [
    {
      "id": 2,
      "endpoint": "https://11.11.11.33",
      "name": "fake again",
      "username": "admin",
      "password": "",
      "type": 0,
      "insecure": false,
      "creation_time": "2018-11-07T14:46:34.752062Z",
      "update_time": "2018-11-07T14:46:34.752062Z"
    }
  ]
}
moooofly commented 5 years ago

Case 3

image

{
  "name": "sync-third",
  "description": "",
  "trigger": {
    "kind": "Scheduled",
    "schedule_param": {
      "type": "Daily",
      "offtime": 68400
    }
  },
  "replicate_existing_image_now": true,
  "replicate_deletion": false,
  "filters": [],
  "projects": [
    {
      "project_id": 8,
      "owner_id": 1,
      "name": "temp_7",
      "creation_time": "2018-11-02T06:58:22Z",
      "update_time": "2018-11-02T06:58:22Z",
      "deleted": false,
      "owner_name": "",
      "togglable": true,
      "current_user_role_id": 1,
      "repo_count": 1,
      "metadata": {
        "public": "true"
      }
    }
  ],
  "targets": [
    {
      "id": 3,
      "endpoint": "http://11.11.11.44",
      "name": "fake third",
      "username": "moooofly",
      "password": "",
      "type": 0,
      "insecure": true,
      "creation_time": "2018-11-07T15:15:35.87736Z",
      "update_time": "2018-11-07T15:15:35.87736Z"
    }
  ]
}
moooofly commented 5 years ago

Case 4

image

{
  "name": "sync-fourth",
  "description": "444",
  "trigger": {
    "kind": "Manual"
  },
  "replicate_existing_image_now": true,
  "replicate_deletion": false,
  "filters": [
    {
      "kind": "repository",
      "value": "repo"
    },
    {
      "kind": "tag",
      "value": "v10086"
    },
    {
      "kind": "label",
      "value": 33
    },
    {
      "kind": "label",
      "value": 30
    },
    {
      "kind": "label",
      "value": 27
    }
  ],
  "projects": [
    {
      "project_id": 7,
      "owner_id": 1,
      "name": "temp_6",
      "creation_time": "2018-11-02T06:58:22Z",
      "update_time": "2018-11-02T06:58:22Z",
      "deleted": false,
      "owner_name": "",
      "togglable": true,
      "current_user_role_id": 1,
      "repo_count": 0,
      "metadata": {
        "public": "true"
      }
    }
  ],
  "targets": [
    {
      "id": 1,
      "endpoint": "http://11.11.11.22",
      "name": "fake endpoint",
      "username": "admin",
      "password": "",
      "type": 0,
      "insecure": true,
      "creation_time": "2018-11-07T14:42:31.876842Z",
      "update_time": "2018-11-07T14:42:31.876842Z"
    }
  ]
}
moooofly commented 5 years ago

Case 5

image

{
  "name": "sync-fifth",
  "description": "fifth",
  "trigger": {
    "kind": "Scheduled",
    "schedule_param": {
      "type": "Weekly",
      "weekday": 1,
      "offtime": 0
    }
  },
  "replicate_existing_image_now": true,
  "replicate_deletion": false,
  "filters": [],
  "projects": [
    {
      "project_id": 6,
      "owner_id": 1,
      "name": "temp_5",
      "creation_time": "2018-11-02T06:58:11Z",
      "update_time": "2018-11-02T06:58:11Z",
      "deleted": false,
      "owner_name": "",
      "togglable": true,
      "current_user_role_id": 1,
      "repo_count": 1,
      "metadata": {
        "public": "true"
      }
    }
  ],
  "targets": [
    {
      "id": 1,
      "endpoint": "http://11.11.11.22",
      "name": "fake endpoint",
      "username": "admin",
      "password": "",
      "type": 0,
      "insecure": true,
      "creation_time": "2018-11-07T14:42:31.876842Z",
      "update_time": "2018-11-07T14:42:31.876842Z"
    }
  ]
}
moooofly commented 5 years ago

Conclusion

Here are the major steps that Harbor UI does

  1. By "GET /api/policies/replication?name=<xxx>" to check if the replication rule with name already exists.
  2. By "GET /api/projects?name=<yyy>" to get source project info to be replicated.
  3. By "GET /api/labels?scope=g" to get all global labels.
  4. By "GET /api/labels?scope=p&project_id=" to get labels from source project.
  5. By "GET /api/targets?name=<zzz>" to get the endpoint info to replicate to.
  6. By "POST /api/policies/replication" to create the replication rule based on above info and some other info.

As you can see when the kind of trigger is "Scheduled":

as per swagger file: The time offset with the UTC 00:00 in seconds.

The policy ID (parameter id defined in policyinfo) should not be included.

moooofly commented 5 years ago

by https://github.com/moooofly/harborctl/issues/30#issuecomment-436667550, you can see that 

"filters": [
    {
      "kind": "repository",
      "value": "repo"
    },
    {
      "kind": "tag",
      "value": "v10086"
    },
    {
      "kind": "label",
      "value": 33
    },
    {
      "kind": "label",
      "value": 30
    },
    {
      "kind": "label",
      "value": 27
    }
  ],

the type of "value" can be of either string type or int type. This makes codes for json.Marshal a little bit inconvenient.

Currently, when setting label IDs with harborctl, you will get 

$./harborctl replication policy create -e fake-remote -r sync-test -s test-1 -d "from cli" --label_ids_as_filter "1,2,3"
==> GET (with struct) https://localhost/api/projects?name=test-1
==> GET (with struct) https://localhost/api/targets?name=fake-remote

----- just for debug -----

===> {"name":"sync-test","description":"from cli","projects":[{"project_id":4,"owner_id":1,"name":"test-1","creation_time":"2019-02-10T04:48:17Z","update_time":"2019-02-10T04:48:17Z","deleted":false,"owner_name":"","togglable":true,"current_user_role_id":1,"repo_count":0,"chart_count":0,"metadata":{"public":"true","enable_content_trust":"","prevent_vul":"","severity":"","auto_scan":""}}],"targets":[{"id":1,"endpoint":"https://11.11.11.22","name":"fake-remote","username":"admin","password":"","type":0,"insecure":true,"creation_time":"2019-02-10T04:49:41.041147Z","update_time":"2019-02-10T04:49:41.041147Z"}],"trigger":{"kind":"Manual","schedule_param":{"type":"","weekday":0,"offtime":0}},"filters":[{"kind":"label","value":"1"},{"kind":"label","value":"2"},{"kind":"label","value":"3"}],"replicate_existing_image_now":true,"replicate_deletion":false}

----- just for debug -----

==> POST https://localhost/api/policies/replication
<==
<== Rsp Status: 400 Bad Request
<== Rsp Body: value the type of value should be integer for label filter
value the type of value should be integer for label filter
value the type of value should be integer for label filter

[#225#root@ubuntu-1604 /go/src/github.com/moooofly/harborctl]$

I think, making some adjustments from server side will be more reasonable.

@steven-zou PTAL