Separate lago ovirt to a different repository and different packaging

[nvgoldin]

The proposed repository is ../lago-project/lago-ost-plugin.git, benefits:

1. This is required for python packaging(via pip)
2. Will create good logic separation between 'ovirt' verb related tests and Lago tests.

So the installation process with yum/dnf will be:

1. yum install python-lago and for OST plugin:
2. yum install python-lago-ost-plugin

blocks: https://github.com/lago-project/lago/issues/309

[ifireball]

It may be better to just move the code to the OST repo itself, then we get CI for changes to the plugin for free (The OST repo already has a "check_patch.sh" that runs tests against a "known-good" oVirt version)

[nvgoldin]

But then we would also need to do the packaging inside OST repo, no?

[eedri]

@nvgoldin no, if we didn't package it till now, why we now? The question should be, would it mean OST will have an install script or requirement to define something other than cloning the git repo and running tests, if it will then its a problem and it will make running OST more complicated for users and will require changes in CI to make sure this is done (instead of installing lago-ovirt plugin)

[ifireball]

@nvgoldin we can probably just do "pip install -e .../foo/..." from the OST repo during the test suit run, so we may end up having to do less packaging...

[nvgoldin]

@eedri - what I meant is that currently python-lago-ovirt is packed inside lago.spec.in, if we move the code itself(i.e. everything in /ovirt' directory and a new setup.py/cfg) into OST, we will need to create a new spec file there. So OST will include a combination of the current OST and the ost-plugin(the packaging and new spec file, will include include the ost-plugin only).

If we don't pack ost-plugin at all and just move it to the OST repo, we will need to add into OST CI scripts(at least):

1. Some sort of step of python setup.py/pip install as @ifireball noted, because the ost-plugin is not independent, among other things, it rely on the entry points and stevedore.
2. We will loose proper versioning of the ost-plugin.

Another option would be to do python packaging only for ost-plugin.

[eedri]

Another consideration is ease for adding new projects to run on Lago. By moving the code to OST we're moving the integration of lago outside of the project and into the project that uses Lago, in this case an automation testing framework.

Won't that affect Lago ability to support other projects? i.e will make the new project support it instead of supporting it via a Lago plugin? what if the new project can't/won't add code to its repo to be able to run Lago, that's a barrier we're putting for new projects to use Lago, no?

If we'll create the ost-plugin, then we are creating an example project that other projects can follow.

[ifireball]

@eedri new projects should not be using the lago-ovirt plugin. That plugin is just for OST, Lago is certainly usable without it.

[eedri]

@ifireball I'm aware of that, I just gave it as an example for a possible plugin. My point was how do we support new projects to run? for e.g a new testing framework to test project X? If it can be supported without writing a special plugin for it ( by supported I mean getting all the functionality OST gets today ) then it shouldn't be a problem. but if we need to write 'support/integration' in lago for it, then having an ost plugin in Lago can help as an example of such integration.

[ifireball]

@eedri IMO most projects (and even oVirt) can do just fine without something like ovirt-lago. Lago core has enough stuff in it IMO to be useful for many projects without any integration plugins.

[eedri]

Can you specify the 'features' lago-ovirt has? I know of reposync, what else?

[ifireball]

$ lago ovirt -h
usage: lago ovirt [-h] VERB ...

positional arguments:
  VERB
    runtest     Run unit tests from a specified file
    engine-setup
                Run engine-setup command on the engine machine
    deploy
    serve
    reposetup   Create a local rpm repository with rpms provided by external
                repository and rpms build from engine/vdsm sources if
                provided.
    revert      Revert to a previously created snapshot. This command
                activates storage domains and hosts after booting up.
    stop
    collect     Collect logs from running VMs
    snapshot    Create snapshots for all deployed resources. This command puts
                the storages, domains and hosts into maintenance before taking
                snapshot.
    start

Out of the above:

• runtest, engine-setup, snapshot, revert and collect are oVirt-specific (and Lago core has generic equivalents for most)
• deploy - Was included in lago core
• reposetup, serve - Repo management code that probably needs to be redisigned and generally gives us quite a bit of trouble... And not necessary for everyone

[nvgoldin]

disclaimer: this discussion is mostly related to OST(http://ovirt-system-tests.readthedocs.io/en/latest/)

Update after yesterday we(me, @gbenhaim , @eedri) discussed this, would be happy get your input on this:

First and foremost, it was widely agreed that reposync should be a core lago feature, this is tracked on https://github.com/lago-project/lago/issues/363, after that is done, I would divide what that would be left in lago-ovirt into 2 parts:

1. the testlib(which OST /test-scenarios use).
2. metadata defining VMs, and some wrappers around lago's command.

For the testlib, I think it is also widely agreed that it should go to to the ovirt-system-tests repository, as it is closely related to the tests(and might in the future be extended or replaced by ansible).

So the discussion is where we put (2). Two options were considered:

1. Into a new GitHub/(or gerrit) project called 'lago-ost-plugin').
2. Merging it into ovirt-system-tests repo in a new directory(i.e. /ovirtlago).

Just for clarification, the current situation is:

1. lago-ovirt is written as a lago-plugin, but its packaging is done in the same spec file as Lago.
2. lago-ovirt has the same versioning as lago.
3. Lago's CI, includes lago-ovirt as well.

Regardless of where we move the plugin to, a question that holds is if we would still want to publish RPMs(or upload it to pip).

Now for the advantages/disadvantages of creating a new repo vs merging into OST:

1. CI - will be easier to run CI tests if merging to OST(update check-merged.sh to track changes and run the stable suites against the plugin), on GitHub we will need to create a new CI flow.
2. If we still want to publish RPMs, we will need to build a new flow, doesn't matter if its on GitHub or inside OST.
3. Decoupling - most stuff in ovirtlago are consumed by OST, so it would be easier to make changes in the plugin and avoiding the round-trip of merging a change to a different project, and then testing it in OST. On the other hand, this might cause new failures in OST because of merges to the plugin.
4. Chance of contribution - people who send patches to OST, are not necessarily Lago developers and are more familiar with the gerrit/ovirt environment, so there is a potential to get more contributions/maintenance to the plugin.

My personal opinion:

1. I think it should be merged to OST, the plugin without reposync doesn't have enough justification for maintaining a different flow.
2. I don't think we should keep versioning of it. It can be installed inside OST. Two options I can think of to do so:
   • add to 'run_suite.sh' something like: cd ovirtlago && pip install . Possibly in virtualenv to ensure latest is installed.
   • Check if it is possible to dynamically load the entry points without running pip. As far as I understand, for that to happen we would need to load the stevedore entry points and update the PYTHONPATH to include ../ovirtlago. Though this needs more investigating.

[ifireball]

On 28 November 2016 at 09:54, Nadav Goldin notifications@github.com wrote:

disclaimer: this discussion is mostly related to OST(http://ovirt-system-tests.readthedocs.io/en/latest/)

Update after yesterday we(me, @gbenhaim , @eedri) discussed this, would be happy get your input on this:

First and foremost, it was widely agreed that reposync should be a core lago feature, this is tracked on #363, after that is done, I would divide what that would be left in lago-ovirt into 2 parts:

I don't think this is widely agreed. I've repeatedly stated that in its current state reposync does not belong in Lago, I'll restate the reasons:

1. Reposync only supports RPMs while Lago core aims to be distro neutral
2. Reposync requires repoman which is currently an oVirt-only project.
3. Reposync only maintains a single repo - useless for projects with more complex configuration where you don't want to make all packages available to all hosts
4. Reposync forces one to use a very simplistic yum configuration. Reproducing complex yum behaviour like, for example, in the Katello repos where the yum *.repo files contain pinning and version-lock information, with how reposync works now would be very cumbersome.

I don't think reposync is a mature enough feature as it is to make it into core Lago, IMO it should remain outside of it for now.

1. the testlib(which OST /test-scenarios use)

Do you mean the "lago ovirt runtest" command?

2. metadata defining VMs, and some wrappers around lago's command.

For the testlib, I think it is also widely agreed that it should go to to the ovirt-system-tests repository, as it is closely related to the tests(and might in the future be extended or replaced by ansible).

So the discussion is where we put (2). Two options were considered:

Into a new GitHub/(or gerrit) project called 'lago-ost-plugin'). Merging it into ovirt-system-tests repo in a new directory(i.e. /ovirtlago).

Just for clarification, the current situation is:

lago-ovirt is written as a lago-plugin, but its packaging is done in the same spec file as Lago. lago-ovirt has the same versioning as lago. Lago's CI, includes lago-ovirt as well.

Regardless of where we move the plugin to, a question that holds is if we would still want to publish RPMs(or upload it to pip).

Now for the advantages/disadvantages of creating a new repo vs merging into OST:

1. CI - will be easier to run CI tests if merging to OST(update check-merged.sh to track changes and run the stable suites against the plugin), on GitHub we will need to create a new CI flow.
2. If we still want to publish RPMs, we will need to build a new flow, doesn't matter if its on GitHub or inside OST.
3. Decoupling - most stuff in ovirtlago are consumed by OST, so it would be easier to make changes in the plugin and avoiding the round-trip of merging a change to a different project, and then testing it in OST. On the other hand, this might cause new failures in OST because of merges to the plugin.
4. Chance of contribution - people who send patches to OST, are not necessarily Lago developers and are more familiar with the gerrit/ovirt environment, so there is a potential to get more contributions/maintenance to the plugin.

My personal opinion:

1. I think it should be merged to OST, the plugin without reposync doesn't have enough justification for maintaining a different flow.

I agree.

2. I don't think we should keep versioning of it. It can be installed inside OST. Two options I can think of to do so:

• add to 'run_suite.sh' something like: cd ovirtlago && pip install . Possibly in virtualenv to ensure latest is installed.

Since we're running inside mock anyway we can probably refrain from using the virtual env. Also not sure if Lago's plugin detection code will know how to look inside a virtualenv.

• Check if it is possible to dynamically load the entry points without running pip. As far as I understand, for that to happen we would need to load the stevedore entry points and update the PYTHONPATH to include ../ovirtlago. Though this needs more investigating.

Probably not worth the effort.

-- Barak Korren bkorren@redhat.com RHCE, RHCi, RHV-DevOps Team https://ifireball.wordpress.com/

[gbenhaim]

https://github.com/lago-project/lago/issues/39

[nvgoldin]

For the record, went for #537 , which means:

1. Moving ovirtlago to https://github.com/lago-project/lago-ost-plugin
2. Keeping RPM packaging - same as today.
3. Using independent CI Flow.

I'd note that in the past months several things have changed:

1. There is more logic/features in ovirtlago now, not only used by OST(demo tool).
2. I don't think we can add any more moving parts to OST at the moment(i.e. testing ovirtlago also).

Looking at https://github.com/lago-project/lago/issues/363 - I changed my mind about and agree with @ifireball , the entire reposetup flow needs major refactoring if we still want to move it out of ovirtlago into Lago.