search

openshift / vagrant-openshift

Other
83 stars 117 forks source link

Allow for users to specify Golang and Docker versions and RPM repos #459

Closed stevekuznetsov closed 8 years ago

stevekuznetsov commented 8 years ago

In order to allow for better configuration of our AMI builds, four new flags are added to the vagrant build-origin-base command: --repofile: to specify repo files to install --repo: to specify repositories to enable for Golang/Docker --golang-version: to specify Golang version --docker-version: to specify Docker version

Any repositories installed from files provided with --repofile will be enabled only for the installation of Golang and Docker, and will be enabled for those installs by default, unless the user provides repositories with --repo to override the default.

The --{golang,docker}-version flags are used to construct package names for the core package as well as for peripheral packages, for instance golang and golang-src. Provide these versions with this behavior in mind.

Signed-off-by: Steve Kuznetsov skuznets@redhat.com

@danmcp PTAL impl, will test when we agree on it

stevekuznetsov commented 8 years ago

I changed the behavior slightly. The flags are now:

$ vagrant build-origin-base --help
Usage: vagrant build-origin-base [vm-name]

        --golang.repofile [name]     Path to repository file to use when installing Golang.
        --golang.repo [name]         Name of repository to enable when installing Golang.
        --golang.version [version]   Install the specified Golang version. Leave empty to install latest available.
        --docker.repofile [name]     Path to repository file to use when installing Docker.
        --docker.repo [name]         Name of repository to enable when installing Docker.
        --docker.version [version]   Install the specified Docker version. Leave empty to install latest available.
    -h, --help                       Print this help

The behavior is to use the 'default' repos (which we only have for RHEL7) if no repofiles or reponames are given, or to use only the names of repos given in repofiles if they're given, or to use only the names given in reponames if they are given and no repofiles are given. Giving repofiles and reponames is fobidden.

We install latest from specified repos unless version is given.

To test we want to see:

@danmcp thoughts?

stevekuznetsov commented 8 years ago

Without any flags, I've had the default setting (which is the same as what we had before) go until flake in e2e somewhere, so I'm fairly confident the resource file refactor is solid. Now just to test the logic.

danmcp commented 8 years ago

If you specify a golang and docker repo (or repofile) that are the same, will you dedup or add 2 of them?

vagrant build-origin-base installs Golang and Docker latest versions from origin-deps-rhel7 and dockerextra

We should change this case to pull in nothing extra. These should really be special cases already.

stevekuznetsov commented 8 years ago

If you specify a golang and docker repo (or repofile) that are the same, will you dedup or add 2 of them?

I'm not particularly interested in adding deduping logic -- what's the long-term support going to be like for this plugin? Once we move away from it as an organization, who's going to be using it? Do they need good UX?

. These should really be special cases already.

I like it. Makes the logic simpler.

stevekuznetsov commented 8 years ago

For posterity, now that default is special-case, previous default is achieved with:

$ vagrant build-origin-base --golang.repofile="${vagrant-openshift}/lib/vagrant-openshift/resources/origin-deps-rhel7.repo" \
                            --golang.version=1.6.2                                                                          \
                            --docker.repofile="${vagrant-openshift}/lib/vagrant-openshift/resources/dockerextra.repo"       \
                            --docker.repo=oso-rhui-rhel-server-releases                                                     \
                            --docker.version=1.10.3
stevekuznetsov commented 8 years ago

@danmcp testing looks good:

origin/master

$ GOLANG_REPO='http://mirror.openshift.com/pub/openshift-origin/nightly/rhel-7/dependencies/x86_64/'
$ vagrant build-origin-base --golang.version=1.6.2            \
                            --golang.repourl="${GOLANG_REPO}" \
                            --docker.version=1.10.3
...
Jul 06 14:12:08   Installing : golang-1.6.2-1.el7.x86_64                                    3/3
...
Jul 06 14:12:32   Installing : docker-1.10.3-44.el7.x86_64                                  9/9

ose/master

$ GOLANG_REPO='http://mirror.openshift.com/pub/openshift-origin/nightly/rhel-7/dependencies/x86_64/'
$ vagrant build-origin-base --golang.version=1.6.2            \
                            --golang.repourl="${GOLANG_REPO}" \
                            --docker.version=1.10.3
...
Jul 06 14:16:07   Installing : golang-1.6.2-1.el7.x86_64                                    3/3
...
Jul 06 14:16:23   Installing : docker-1.10.3-44.el7.x86_64                                  9/9

ose/enterprise-3.0

$ vagrant build-origin-base --golang.version=1.4.2 \
                            --docker.version=1.8.2
...
Jul 06 14:17:22   Installing : golang-1.4.2-9.el7.x86_64                                    3/4
...
Jul 06 14:17:48   Installing : docker-1.8.2-10.el7.x86_64                                   2/2

ose/enterprise-3.1

$ vagrant build-origin-base --golang.version=1.4.2 \
                            --docker.version=1.8.2
...
Jul 06 14:20:12   Installing : golang-1.4.2-9.el7.x86_64                                    3/4
...
Jul 06 14:20:37   Installing : docker-1.8.2-10.el7.x86_64                                   2/2

ose/enterprise-3.2

$ GOLANG_REPO='http://mirror.openshift.com/pub/openshift-origin/nightly/rhel-7/dependencies/x86_64/'
$ vagrant build-origin-base --golang.version=1.6.2            \
                            --golang.repourl="${GOLANG_REPO}" \
                            --docker.version=1.9.1
...
Jul 06 14:21:50   Installing : golang-1.6.2-1.el7.x86_64                                    3/3
...
Jul 06 14:22:27   Installing : docker-1.9.1-40.el7.x86_64                                   4/4
$ GOLANG_REPO='http://mirror.openshift.com/pub/openshift-origin/nightly/rhel-7/dependencies/x86_64/'
$ vagrant build-origin-base --golang.version=1.6.2            \
                            --golang.repourl="${GOLANG_REPO}" \
                            --docker.version=1.10.3
...
Jul 06 14:23:42   Installing : golang-1.6.2-1.el7.x86_64                                    3/3
...
Jul 06 14:24:06   Installing : docker-1.10.3-44.el7.x86_64                                  9/9
stevekuznetsov commented 8 years ago

docker.repourl and docker.repo also work

$ GOLANG_REPO='http://mirror.openshift.com/pub/openshift-origin/nightly/rhel-7/dependencies/x86_64/'
$ DOCKER_REPO='https://mirror.openshift.com/enterprise/rhel/dockerextra/x86_64/os/'
$ vagrant build-origin-base --golang.version=1.6.2            \
                            --golang.repourl="${GOLANG_REPO}" \
                            --docker.version=1.10.3           \
                            --docker.repourl="${DOCKER_REPO}" \
                            --docker.repo=oso-rhui-rhel-server-releases
...
Jul 06 14:14:18   Installing : golang-1.6.2-1.el7.x86_64                                    3/3
...
Jul 06 14:14:59   Installing : docker-1.10.3-42.el7.x86_64                                  9/9
stevekuznetsov commented 8 years ago

@danmcp this is quickly snowballing into fragile nightmare territory. I think it might be less work for us to configure the base_ami not to include Docker or Golang and the next level to be responsible for installing them, and point the non-standard Jenkins jobs to start from base_ami and install their own.

danmcp commented 8 years ago

@stevekuznetsov The next level is devenv_ami. Which would still have the problem of creating a lot of combinations.

stevekuznetsov commented 8 years ago

@danmcp then you might have to live with my regexp hacks... there's no nice way out of this

stevekuznetsov commented 8 years ago

@danmcp lets chat tomorrow about the difficulties I ran into and why a "smarter" solution isn't going to be less fragile or more useful than my hacks

stevekuznetsov commented 8 years ago

@danmcp ready for a first-pass on the uninstall stuff

stevekuznetsov commented 8 years ago

[test]

stevekuznetsov commented 8 years ago

:confetti_ball: :star2: :tada: @danmcp I think this is ready

stevekuznetsov commented 8 years ago

@danmcp just noticed -- bot never updated the CI status for this even though the job succeeded. @dobbymoodge I know we put the SUCCESS comment from the job, do we also update the status from the job? Did we forget to do that / did I manage to break that when I edited the job?

danmcp commented 8 years ago

@stevekuznetsov Status is updated as part of mark test success. Failures to update it are ignored. It's possible it just failed. Could be bug too. Also mark test success is not a requirement. tpr will pick it up on the next run if the job misses it. The only reason we do it from the job as well is for faster feedback and so it won't be missed on merge pretest success.

stevekuznetsov commented 8 years ago

[test] me

stevekuznetsov commented 8 years ago

@danmcp I've taken the Docker and Golang installation completely out of build-origin-base since we can't do a re-install afterwords. Tests seem to be fine. I'll squash and make this pretty tomorrow, and we can get it installed Monday morning.

openshift-bot commented 8 years ago

Evaluated for vagrant openshift test up to 96a625d179fd377af7479ba1b106f938c27adb25

openshift-bot commented 8 years ago

Vagrant OpenShift Test Results: SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pull_requests_vagrant_openshift/83/) (Base Commit: f06be13f3f68dfbdf4f893e257b0994fafd57544)

stevekuznetsov commented 8 years ago

Green tests, I'll squash and merge. @danmcp FYI