search

ansible / molecule

Molecule aids in the development and testing of Ansible content: collections, playbooks and roles
https://ansible.readthedocs.io/projects/molecule/
MIT License
3.86k stars 659 forks source link

Switch from digital_ocean module to digital_ocean_droplet #2054

Closed tehsmyers closed 4 years ago

tehsmyers commented 5 years ago

Issue Type

Summary

Deprecations to the digital_ocean module in by ansible 2.8 have broken an import in functional tests and potentially change what dependencies are required based on the version of ansible being used.

Background

Molecule should, of course, stop using the deprecated module in favor of the new module added in ansible 2.8. Doing so introduces some complexity, which is that molecule will now need to inspect the ansible version to conditionally decide which module to use, both in playbooks that use the digitalocean modules, and in the supports_digitalocean function above. There are a few different ways that molecule can deal with this complexity, which I've tried to outline here.

Options

  1. Fix the broken import, continue using the deprecated module until ansible 2.10 comes out, at which point ansible <2.8 will no longer be supported and we can switch from the deprecated digital_ocean module, which will be removed in ansible 2.12 (so we'll be able to safely transition during the ansible 2.10 and 2.11 releases. This is a simple option, but is not very user-friendly since it requires ansible 2.8 users to install a dependency to use digitialocean that is no longer required in ansible 2.8.
  2. Fix the broken import, conditionally use the correct module based on which version of ansible is currently being used. This is less simple, but means that users should no longer have to install the dopy module for ansible >= 2.8. It will result in more complicated create.yml and destroy.yml playbooks for digitalocean scenarios.
  3. Fix the broken import, conditionally use a different cookiecutter template based on the version of ansible installed when the cookiecutter template is used to initialize a scenario. This is the opposite of simple, and while it is an option I don't think it's a very good one.
  4. Other options welcome. :)
tehsmyers commented 5 years ago

Every option begins with "fix the broken import", so I'm going to put a PR up for that real quick and reference this issue.

At the moment, I think option 2 is the best balance of maintainability and user-friendliness, but look forward to seeing other potential ways to deal with this.

decentral1se commented 5 years ago

Thanks for this comprehensive report! I knew we'd have to start paying this tax for the close internal importing and coupling to Ansible core at some point. What you've outlined here is a good picture of what we're facing.

I'm up for 2 for the short term and we've already seen a need and basically, a decision, to try this path over in https://github.com/ansible/molecule/pull/2047#discussion_r285072742. It may be the most obvious one for now which is covered by our CI and keeps us moving.

3 is tempting but may add more maintenance overhead. I'm just brain storming here but is there any value in thinking about some pre-generation cookiecutter logic which decides which modules are used based on what ansible --version returns? We could maintain a mapping of which modules to use somewhere based on versions and use the same templates ... this is quite meta but a POC might be useful.

decentral1se commented 5 years ago

As the original implementor in https://github.com/ansible/molecule/pull/1736, any chance we could summon your efforts once again on this one @bmildren? We could use the help :)

decentral1se commented 5 years ago

One further question: how does it work with setuptools when you want to skip the dopy dependency based on the version of Ansible? I assume we'll need to leave this extra for < 2.8? We'll just document that this requires now no external dependency or!?

bmildren commented 5 years ago

Sorry folks I was away last week, I'd be happy to take a look 👍

n0guest commented 5 years ago

Hello, guys.

Is there any news on this matter?

Sorry for hijacking this thread, but it's out of "despair". I've been trying to use Molecule and Ansible 2.8.x with DO ... And it ended very sad. Python 3.5 usually dies somewhere at parsing JSON from DO API. We moved to Python 3.6, but there we faced another roadblock - unresolved issues within "dead" dopy package. In the same time Linode driver is also marked as deprecated. Hetzner driver still in some early alpha stages. So, it's 3 out of 3 for small cloud providers - all of them are unusable with Molecule for now.

One more time sorry for offtopic, but I'm really hope to at least get an understading on how far this matter from working state. I found some PR and some following merges, but I still can't find a working recipe "Molecule+Ansible+DO".

PS: Sure, there are still "big" clouds (AWS/GCP/Azure), but they also little pricey in compare with little players.

decentral1se commented 5 years ago

Nice feedback @n0guest. We're working on it. It's going slow but at least it is going. Not many hands on deck here and the moving train that is Ansible is hard to keep up with since there are many issues in Molecule that slow things down.

Hopefully someone can pick this issue up soon.

n0guest commented 5 years ago

Nice feedback @n0guest. We're working on it. It's going slow but at least it is going. Not many hands on deck here and the moving train that is Ansible is hard to keep up with since there are many issues in Molecule that slow things down.

Is there anyway how I could help? But, please note, that I'm not a python developer. So, in some other ways maybe?

decentral1se commented 5 years ago

While a lot of things are just plain 'ol Ansible, the core of Molecule and the drivers do require some working Python development knowledge. It's not bad but well, it's not easy either ;) If you get started, we can help! Otherwise, we'll be sure to call on you when we need some Q&A testing for this new driver migration!

n0guest commented 5 years ago

Otherwise, we'll be sure to call on you when we need some Q&A testing for this new driver migration!

We're interested in working drivers for DO/Linode/Hetzner (needed for internal testing). So, we'd be happy to participate any way we can.

PS: But, I guess, this is a huge offtopic for this issue. And we should continue somewhere else?

decentral1se commented 5 years ago

We're interested in working drivers for DO/Linode/Hetzner

All of these are already working in some fashion btw. Hetzner just has no functional test suite but I use it. Linode is officially deprecated but there is no end of the support announced. This DO module is still in use. We're on #ansible-molecule @ freenode on IRC.

decentral1se commented 5 years ago

Hey people, we're moving our way to Molecule 3.0 and that means, we have a window for breaking changes. It would be nice to support the DO old modules here in molecule and the new ones but that is quite some work and we're low on capacity. If anyone has spare cycles, it'd be great to move to the new DO module ASAP.

decentral1se commented 5 years ago

Bump @bmildren @seandst :shark: :shark: :shark:

bmildren commented 4 years ago

Apologies, I lost track of this one - I'll take a look this week

ssbarnea commented 4 years ago

Driver moved to https://github.com/ansible-community/molecule-digitalocean/