Rework package management to allow composition of dependencies

This issue has the purpose of tracking progress for improving the way the puppet-python module manage packages.

Issues we want to solve

In these example, we assume a control-repo with a profile::python class that wraps the puppet-python module with site-specific config:

class profile::python {
  class { 'python':
    ensure  => present,
    # ...
  }
}

System packages with dependencies on `python-foo`

The syslog-ng-mod-python package depend on python-venv. When this package is added to the catalog, puppet will install syslog-ng-mod-python, which will install python-venv as a dependency. On next run, python-venv is removed (because the python class default to ensure this package is absent) and syslog-ng-mod-python which depend on it is also removed.

To workaround this issue, we can skip management of the python-venv in the python module with:

class profile::python {
  class { 'python':
    # ...
    manage_venv_package => false, # python-venv is a dependency of syslog-ng-mod-python
  }
}

If another node need to have the venv package installed, we have a few solutions:

Add a parameter to the profile::python and use Hiera to set it to a custom value on the nodes that must have different behavior;
Duplicate and adjust the profile::python class (e.g. profile::python_with_venv) and include one or the other of these two conflicting profiles, leading to some duplication;
Manage the python-venv package in the profile that need it, duplicating what the puppet-python module is supposed to manage.

None of these solutions is great.

Puppet modules needing a specific component

The puppet-puppetboard module rely on the puppet-python module. When installing from a venv, it include this code in the catalog:

    class { 'python':
      version => $python_version,
      dev     => 'present',
      venv    => 'present',
    }

This clash with the above profile::python. To cope with this, the module provide a manage_virtualenv parameter to skips this fragment and allow the user to handle this themself, with the same solutions and same drawbacks as seen in the previous example.

How puppetlabs-apache handle a similar issue

Apache has a lot of different modules, but each one is managed in a different class allowing to do some composition of the dependencies from another puppet module, e.g.:

class taiga::vhost {
  include apache
  include apache::mod::passenger

  # ...
}

For the end-user include taiga::vhost will do the right thing, and if they want to customize the apache::mod::passenger parameters, they can do this in their code:

class profile::taiga {
  class { 'apache::mod::passenger':
    # ... this would be in a passenger specific profile
  }

  class { 'taiga::vhost':
    # ...
  }
}

Roadmap

After this long context, I propose to rework the module package management to match what is done by puppetlabs-apache, make it easier to manage dependencies in consumer modules and avoid conflicts.

This is a three step process:

[x] Allow to opt-out of management of python-dev (enhancement) #669;
[ ] Move components installation code in dedicated classes (enhancement) #670;
[ ] Change defaults for packages which are ensured absent by default: switch them to installed but do not manage them by default (backwards-incompatible).

voxpupuli / puppet-python