This tool is designed to ease the transition from 3rd party V2 Docker registries to JFrog Artifactory.
git clone https://github.com/JFrogDev/docker2artifactory.gitpython DockerMigrator.py ......docker run -it --rm --name my-running-script -v "$PWD":/usr/src/myapp -w /usr/src/myapp python:2.7.14 python DockerMigrator.py .....The generic registry migrator works against token based registries. This includes DockerHub, Artifactory registries, and many others. The this tool has two modes of operation, auto-discovery and guided migrations. In auto-discovery mode, the tool will query the source registry (using standard Docker registry apis) to get a full listing of all the available image names and tags. It will then migrate all of the images it finds. In the guided mode, you provide a list of image names and tags you wish to migrate (see Image file format)
usage: python DockerMigrator.py generic [-h]
[--source-username SOURCE_USERNAME]
[--source-password SOURCE_PASSWORD]
[--ignore-certs] [--overwrite]
[--num-of-workers WORKERS] [-v]
[--image-file IMAGE_FILE]
source artifactory username password
repo
optional arguments:
-h, --help show this help message and exit
--ignore-certs Ignore any certificate errors from both source and
destination
--overwrite Overwrite existing image/tag on the destination
--num-of-workers WORKERS
Number of worker threads. Defaults to 2.
-v, --verbose Make the operation more talkative
--image-file IMAGE_FILE
Limit the import to a set of images in the provided
file. Format of new line separated file: '<image-
name>:<tag>' OR '<image-name>' to import all tags of
that repository.
source:
source The source registry URL
--source-username SOURCE_USERNAME
The username to use for authentication to the source
--source-password SOURCE_PASSWORD
The password to use for authentication to the source
artifactory:
artifactory The destination Artifactory URL
username The username to use for authentication to Artifactory
password The password to use for authentication to Artifactory
repo The docker repository in Artifactory to store the imagesDockerHub does not implement the catalog API. To migrate images from DockerHub, follow these guide lines:
library/. For centos, this would look be library/centos or library/centos:latest for the latest tag.The DTR registry migrator works against the Docker Trusted Registry (DTR). The generic migrator can also be used for DTR but will rely on standard APIs (which have known issues in DTR). This version will use DTR proprietary APIs (api/v0) for the auto-discovery mechanism. The this tool has two modes of operation, auto-discovery and guided migrations. In auto-discovery mode, the tool will query DTR APIs to get a full listing of all the available image names and tags. It will then migrate all of the images it finds. In the guided mode, you provide a list of image names and tags you wish to migrate (see Image file format)
usage: python DockerMigrator.py dtr [-h] [--ignore-certs] [--overwrite]
[--num-of-workers WORKERS] [-v]
[--image-file IMAGE_FILE]
source dtr_username dtr_password
artifactory username password repo
optional arguments:
-h, --help show this help message and exit
--ignore-certs Ignore any certificate errors from both source and
destination
--overwrite Overwrite existing image/tag on the destination
--num-of-workers WORKERS
Number of worker threads. Defaults to 2.
-v, --verbose Make the operation more talkative
--image-file IMAGE_FILE
Limit the import to a set of images in the provided
file. Format of new line separated file: '<image-
name>:<tag>' OR '<image-name>' to import all tags of
that repository.
source:
source The DTR registry URL
dtr_username The username of a DTR admin
dtr_password The DTR admin password or token
artifactory:
artifactory The destination Artifactory URL
username The username to use for authentication to Artifactory
password The password to use for authentication to Artifactory
repo The docker repository in Artifactory to store the
imagesThe Quay registry migrator works against Quay's SaaS offering. It works against both public and private registries. The this tool has two modes of operation, auto-discovery and guided migrations. In auto-discovery mode, the tool will query the source registry (using Quay specifc apis) to get a full listing of all the available image names and tags. It will then migrate all of the images it finds. In the guided mode, you provide a list of image names and tags you wish to migrate (see Image file format)
To use this tool, you need to generate a token for internal application use.
usage: python DockerMigrator.py quay [-h] [--ignore-certs] [--overwrite]
[--num-of-workers WORKERS] [-v]
[--image-file IMAGE_FILE]
namespace token artifactory username
password repo
optional arguments:
-h, --help show this help message and exit
--ignore-certs Ignore any certificate errors from both source and
destination
--overwrite Overwrite existing image/tag on the destination
--num-of-workers WORKERS
Number of worker threads. Defaults to 2.
-v, --verbose Make the operation more talkative
--image-file IMAGE_FILE
Limit the import to a set of images in the provided
file. Format of new line separated file: '<image-
name>:<tag>' OR '<image-name>' to import all tags of
that repository.
quay:
namespace The username or organization to import repositories
from
token The OAuth2 Access Token
artifactory:
artifactory The destination Artifactory URL
username The username to use for authentication to Artifactory
password The password to use for authentication to Artifactory
repo The docker repository in Artifactory to store the imagesThe Quay registry migrator works against Quay's Enterprise offering. It works against both public and private registries. The this tool has two modes of operation, auto-discovery and guided migrations. In auto-discovery mode, the tool will query the source registry to get a full listing of all the available image names and tags. It will then migrate all of the images it finds. In the guided mode, you provide a list of image names and tags you wish to migrate (see Image file format)
To use this tool, you need to provide the super user's credentials or an oauth token with all permissions.
Quay does not have the concept of an all seeing user. The super user cannot see other users' repositories. To properly import all repositories, all users/organizations should grant the super user being used READ access to their repositories.
usage: python DockerMigrator.py quayee [-h]
[--source-username SOURCE_USERNAME]
[--source-password SOURCE_PASSWORD]
[--token TOKEN] [--ignore-certs]
[--overwrite]
[--num-of-workers WORKERS] [-v]
[--image-file IMAGE_FILE]
source artifactory username password
repo
optional arguments:
-h, --help show this help message and exit
--ignore-certs Ignore any certificate errors from both source and
destination
--overwrite Overwrite existing image/tag on the destination
--num-of-workers WORKERS
Number of worker threads. Defaults to 2.
-v, --verbose Make the operation more talkative
--image-file IMAGE_FILE
Limit the import to a set of images in the provided
file. Format of new line separated file: '<image-
name>:<tag>' OR '<image-name>' to import all tags of
that repository.
source:
source The source registry URL
--source-username SOURCE_USERNAME
The super user username
--source-password SOURCE_PASSWORD
The super user password
--token TOKEN The OAuth2 Access Token
artifactory:
artifactory The destination Artifactory URL
username The username to use for authentication to Artifactory
password The password to use for authentication to Artifactory
repo The docker repository in Artifactory to store the images
The Amazon ECR registry migrator works against Amazon's ECR service. The this tool has two modes of operation, auto-discovery and guided migrations. In auto-discovery mode, the tool will query the source registry (using standard Docker registry apis) to get a full listing of all the available image names and tags. It will then migrate all of the images it finds. In the guided mode, you provide a list of image names and tags you wish to migrate (see Image file format)
Amazon does not support token authentication like Docker token based registries expect them. Instead, you need to generate an Amazon token/password (which will be valid for 12 hours) to use with this tool. To do so, you can perform the following call with the aws tool (see Registry Authentication):
aws ecr get-login --no-include-emailThis will produce an output like this:
docker login -u AWS -p password https://aws_account_id.dkr.ecr.us-east-1.amazonaws.comThe token you need to supply to this tool is the password from above, the source will be the URL provided above (in this example the token would be password and the source https://aws_account_id.dkr.ecr.us-east-1.amazonaws.com).
usage: python DockerMigrator.py ecr [-h] [--ignore-certs] [--overwrite]
[--num-of-workers WORKERS] [-v]
[--image-file IMAGE_FILE]
source token artifactory username password
repo
optional arguments:
-h, --help show this help message and exit
--ignore-certs Ignore any certificate errors from both source and
destination
--overwrite Overwrite existing image/tag on the destination
--num-of-workers WORKERS
Number of worker threads. Defaults to 2.
-v, --verbose Make the operation more talkative
--image-file IMAGE_FILE
Limit the import to a set of images in the provided
file. Format of new line separated file: '<image-
name>:<tag>' OR '<image-name>' to import all tags of
that repository.
source:
source The source registry URL
token The token generated by the aws tool
artifactory:
artifactory The destination Artifactory URL
username The username to use for authentication to Artifactory
password The password to use for authentication to Artifactory
repo The docker repository in Artifactory to store the imagesThe GCR migrator works against Google's GCR service. The this tool has two modes of operation, auto-discovery and guided migrations. In auto-discovery mode, the tool will query the source registry (using standard Docker registry apis) to get a full listing of all the available image names and tags. It will then migrate all of the images it finds. In the guided mode, you provide a list of image names and tags you wish to migrate (see Image file format)
You will need a JSON key file with the correct permissions to perform the migration. If you do not already have one, follow Google's Advanced Authentication document.
usage: python DockerMigrator.py gcr [-h] [--source SOURCE] [--ignore-certs]
[--overwrite] [--num-of-workers WORKERS]
[-v] [--image-file IMAGE_FILE]
keyfile artifactory username password repo
optional arguments:
-h, --help show this help message and exit
--ignore-certs Ignore any certificate errors from both source and
destination
--overwrite Overwrite existing image/tag on the destination
--num-of-workers WORKERS
Number of worker threads. Defaults to 2.
-v, --verbose Make the operation more talkative
--image-file IMAGE_FILE
Limit the import to a set of images in the provided
file. Format of new line separated file: '<image-
name>:<tag>' OR '<image-name>' to import all tags of
that repository.
source:
--source SOURCE The source registry URL (defaults to https://gcr.io)
keyfile The Google JSON key file
artifactory:
artifactory The destination Artifactory URL
username The username to use for authentication to Artifactory
password The password to use for authentication to Artifactory
repo The docker repository in Artifactory to store the imagesThe image file format accepts two types of entries. The first is specifying an image and optionally a namespace. The second is specifying an image (and optional a namespace) and a tag. When the first option is used (no tag is specified), the tool will migrate ALL the tags for that particular image name.
For example:
busybox
jfrog/artifactory-pro
jfrog/mission-control:1.0This would result in all tags of busybox and jfrog/artifactory-pro being migrated but only the 1.0 tag for jfrog/mission-control.
To migrate security information from the Docker registry to Artifactory, follow the steps described at the Data Migration session but use the SecurityMigrator.py script instead of DockerMigrator.py. Examples:
python SecurityMigrator.py ......docker run -it --rm --name my-running-script -v "$PWD":/usr/src/myapp -w /usr/src/myapp python:2.7.14 python SecurityMigrator.py .....Using this option, the tool reads Users, Organizations and Teams data from Docker Universal Control Plane (UCP) and Permissions information from Docker Trusted Registry (DTR) and creates the appropriate Users, Groups and Permissions in Artifactory. This feature has been tested with UCP 2.2.5 and DTR 2.4.1.
usage: python SecurityMigrator.py dockeree [-h] [--ignore-certs] [--overwrite]
[-v]
ucp dtr dockeree-username
dockeree-password artifactory
username password repo
initial-password email-suffix
Docker EE security data to Artifactory migrator. Migrates users, teams and
permissions from Docker EE UCP and DTR to Artifactory.
positional arguments:
initial-password The password to be assigned to migrated users in
Artifactory
email-suffix The email suffix to be assigned to migrated users in
Artifactory
optional arguments:
-h, --help show this help message and exit
--ignore-certs Ignore any certificate errors from both source and
destination
--overwrite Overwrite existing users, groups or permissions on the
destination
-v, --verbose Make the operation more talkative
dockeree:
ucp The Docker Universal Control Plane (UCP) URL
dtr The Docker Trusted Registry (DTR) URL
dockeree-username The username to use for authentication to the Docker EE
tools
dockeree-password The password to use for authentication to the Docker EE
tools
artifactory:
artifactory The destination Artifactory URL
username The username to use for authentication to Artifactory
password The password to use for authentication to Artifactory
repo The docker repository in Artifactory to store the imagesUsing this option, the tool reads users, organization robots, organizations, teams, and permissions. The tools imports these elements (with some adjustments and mappings). See Limitations and Notes.
usage: python SecurityMigrator.py quayee [-h] [--ignore-certs] [--overwrite]
[-v]
source token artifactory username
password repo initial-password
Quay EE security data to Artifactory migrator. Migrates users, teams and
permissions from Quay EE to Artifactory.
positional arguments:
initial-password The password to be assigned to migrated users in
Artifactory
optional arguments:
-h, --help show this help message and exit
--ignore-certs Ignore any certificate errors from both source and
destination
--overwrite Overwrite existing users, groups or permissions on the
destination
-v, --verbose Make the operation more talkative
source:
source The source registry URL
token The OAuth2 Access Token of the super user
artifactory:
artifactory The destination Artifactory URL
username The username to use for authentication to Artifactory
password The password to use for authentication to Artifactory
repo The docker repository in Artifactory to store the imagesThis project uses Python's unittest framework. To run the tests, you can use your favorite IDE, or run them from the command line.
To run the unit tests from the command line (requires Python 2.7):
cd tests
python -m unittest discover -v -p "*Test.py"