Implement install of things with dep solving (for trad roles and collections)

Feature Request

Use Case

When installing a collection or a traditional role, automatically install any other galaxy repositories that the collections notates as being requirements or dependencies.

For a collection

For example,
alikins.collections_reqs_test is a galaxy collection that has:

./galaxy.yml: No requirements listed in galaxy.yml
./MANIFEST.json: not an artifact, so no MANIFEST.json
./meta/main.yml: no meta/ dir, no meta/main.yml
./meta/requirements.yml: no meta/ dir, no meta/requirements.yml
./requirements.yml:

- src: yatesr.timezone

- src: alikins.ansible_testing_content
  name: does_name_even_matter

- src: geerlingguy.apache
  version: 2.1.1

- src: geerlingguy.ntp
  version: 1.5.0

Running: mazer install alikins.collections_reqs_test

should install 'alikins.collections_reqs_test' and it's requirements ('yatesr.timezone' trad role, 'alikins.ansible_testing_content' collection, and the two geerlingguy trad roles).

For a traditional role

For example,
'geerlingguy.php-xdebug' is a galaxy traditional role that has:

./galaxy.yml: Not a collection, so no galaxy.yml
./MANIFEST.json: not an artifact, so no MANIFEST.json
./requirements.yml:
./meta/requirements.yml: no meta/requirements.yml
./meta/main.yml:

---
dependencies:
  - geerlingguy.php

galaxy_info:
  author: geerlingguy
  description: PHP XDebug for Linux
  company: "Midwestern Mac, LLC"
  license: "license (BSD, MIT)"
  min_ansible_version: 2.4
  platforms:
    - name: EL
      versions:
        - all
    - name: GenericUNIX
      versions:
        - all
  galaxy_tags:
    - development
    - web
    - php
    - xdebug
    - debug
    - profile

Running:

    `mazer install geerlingguy.php-xdebug`

should install 'geerlingguy.php-xdebug' and it's requirements ('geerlingguy.php')

Proposed Solution

Describe the solution you'd like A clear and concise description of what you want to happen. Add any considered drawbacks.

Alternatives

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Implementation

Teachability, Documentation, Adoption, Migration Strategy

Very work-in-progress branch at https://github.com/alikins/mazer/tree/instally

approximate punch list / TODOs / deps

[x] add models/objects/serializers for requirements, deps, and other deps related metadata
- [x] loader/serializer/models for ./requirements.yml, meta/requirements.yml
- [x] loader/serializer/models for meta/main.yml:dependencies,
- [x] loader/serializer/models for ./galaxy.yml requirements,
- [x] loader/serializer/models for ./MANIFEST.json requirements,
[x] Add serializer/loaders for various on disk formats of repositories/collections/roles/archives/artifacts
- [x] traditional role archive (as downloaded from github via mazer)
- [x] 'raw' collection archive (as downloaded from github via mazer)
- [x] requirements.yml
- [x] meta/main.yml deps (if it exists)
- [x] galaxy.yml requirements
- [ ] collection 'built' artifacts
  - [ ] requirements.yml
  - [ ] MANIFEST.json
- [ ] maybe use attr.ib 'metadata' fields to indicate where (which file and type) a models attribute could be loaded/saved to
[x] Update install action flow to support iteration of install steps (install something, find deps, install deps, repeat)
[x] update 'installed' content/repo/collection dbs and query matchers for resolving deps
[x] split Content/Repository/Collection models into ContentSpec and Content (etc).
- [x] ContentSpec types are just enough info to find/match/compare 'somenamespace.somename'
  - [x] implement RequirementSpec model
  - [ ] implement DependencySpec model
  - [ ] Allow requested DependencySpec to potentially remain unresolve since they are runtime deps and may reference content that is not available to mazer so mazer cant solve them.
[ ] investigate how ansible core plans to load collections if at all
- ie, if it will load a collection as data, will it need something like ansible.playbook.role.Role etc
[ ] collection deps grammar
- [ ] Decide/design collection requirements.yml
- [ ] Decide how compatible with role ansible-galaxy requirements.yml it needs to be.
- [ ] Decide/design how requirements are specified and what they can express?
- [ ] Decide/design if or how versions can be specified
  - ie, do we support '>', '>=', '=' version requirements?
  - Or if there should be a way to express requirements for semver ranges, to express the idea of 'this collection requires somenamespace.somename at major version '2' and minor version '1'.
[x] investigate serializer frameworks for attr
[ ] SHOULD build some real test collections for testing
- [ ] ideally convert all ansible/ansible bundled content to be in collections (example at https://github.com/alikins/ansible_collections)
- [ ] modules
  - [ ] potentially with meta collections
    - ie, a 'ansible_galaxy.cloud' would require 'amazon', 'azure', 'digitalocean', etc
    - ansible_galaxy.cloud_amazon require 'ansible_galaxy.cloud_amazon_ec2', '_s3', etc
  - [ ] find way to determine module->module_utils deps
  - [ ] add requirements.yml with info?
- [ ] plugins with deps
  - ie, all action plugins would require 'ansible_galaxy.plugins_action' (and 'ansible_galaxy.plugins_action' in ansible-minimal)
  - [ ] find way to determine plugins that depend on other plugins?
[ ] MAYBE - Rollup deps from each content in a collection would roll up into the collections metadata / MANIFEST.json for artifacts?
- [ ] MAYBE - for bare collections (not 'built' artifacts)
  - [ ] MAYBE create a MANIFEST.json or equivalent at 'install' time
  - [ ] MAYBE Decide how to hanble 'i-dont-want-to-install-anything-dev-workflow'
  - [ ] MAYBE install -e could create a MANIFEST.json or equivalent index at install time

ansible / mazer