Multiple platforms support (--platform option not working)

[michalgasek]

$ molecule --version
1.5.1

I would assume that Molecule would run molecule test for each platform listed in platforms but that doesn't seem to be the case. Below is my molecule.yml file.

vagrant:
  raw_config_args:
    - "ssh.insert_key = false"

  platforms:
    - name: trusty64
      box: ubuntu/trusty64
    - name: precise64
      box: ubuntu/precise64

  providers:
    - name: virtualbox
      type: virtualbox
      options:
        memory: 1024
        cpus: 2

  instances:
    - name: s1
      options:
        append_platform_to_hostname: no

When running molecule test:

Only trusty64 platform is tested in Vagrant. I would expect Molecule to run tests for each platform listed in platforms.

When running molecule test --platform=precise64:

Still trusty64 is being used.... --platform option doesn't change behaviour.

Bringing machine 's1' up with 'virtualbox' provider...
==> s1: Cloning VM...
==> s1: Matching MAC address for NAT networking...
==> s1: Checking if box 'ubuntu/trusty64' is up to date...

[michalgasek]

More observations:

molecule create --platform=precise64 has desired behaviour:

$ molecule create --platform=precise64
Bringing machine 's1' up with 'virtualbox' provider...
==> s1: Cloning VM...
==> s1: Matching MAC address for NAT networking...
==> s1: Checking if box 'ubuntu/precise64' is up to date...
...

After running above I ran molecule converge (without --platform=precise64 option) and it worked just fine. I would expect that this command should fail at this point since the default platform is trusty64 and molecule should use --platform=trusty64 as default.

Afterwards I ran molecule converge --platform=trusty64 and Molecule ran converge against previously created precise64 box, claiming that everything works...

--platform option behaviour seems to be very inconsistent and not failing when it should.

[retr0h]

I would assume that Molecule would run molecule test for each platform listed in platforms but that doesn't seem to be the case. Below is my molecule.yml file.

Yeah, I don't recall why or what problems we encountered. That bit of code was written long ago, and we are mostly a RHEL shop, so our own use case is very limited with multi-platform. Platforms need to be targeted via --platforms, it does not re-execute for each platform.

Pull requests always welcome.

[kireledan]

I was actually just about to fix this to work as @michalgasek thought it would. I ran into the exact same issue where I needed multiple providers. I'm a super big fan of doing it because testkitchen does this. But @michalgasek if you already started something I'll keep working on other things.

[michalgasek]

@kireledan no I haven't started working on it, in fact I haven't browsed codebase even yet. I've just managed to browse docs to test Molecule for the first time yesterday trying to find a right tool for my Ansible roles testing, Molecule seemed like a cool thing to try out.

It would be greatly appreciated if you would fix the issues mentioned.

[kireledan]

Yeah sure thing! Shouldn't take too long to fix this.

[conorsch]

@kireledan Happy to hear it! The single-platform approach of Molecule was counterintuitive for me getting started, mostly because I'd assumed it would follow the Test Kitchen pragma of building providers sequentially.

Consider supporting special var --platform all that would build and test sequentially, if folks prefer the single-platform setting as default.

[retr0h]

Yeah sure thing! Shouldn't take too long to fix this.

Yeah, that would actually be a nice addition.

[kireledan]

Should we do a --platform all approach? Because if we don't then it would be better to get rid of the --platform flag.

[conorsch]

Ditching the --platform flag would be a substantial change, and might surprise folks unnecessarily. It would also contradict the approach in #147, requiring that we instead default to appending platforms to hostname (which is test-kitchen's approach).

At the risk of sounding overly cautious, I'd say let's support a magic --platform=all var as a first step in migrating away from the single-platform-only approach Molecule currently espouses. If folks still find that too problematic, we can rather simply update it to be even more similar to test-kitchen, but I don't think that's necessary right now.

[techraf]

I am confused.

• The first post claims --platform argument is ignored when running module test.
• The title mentions two things "Multiple platforms support (--platform option not working)".
• The merge added all option which tangentially addresses first part of the title.

Why was this closed?

The issue described (--platform ignored when running test) is still a problem in current 1.7.0.

[kireledan]

I will investigate the platform flag issue. It seems that we mistakenly only focused on your comment about behavior with testing multiple platforms. Last time I checked, it appeared to be working for me when specifying a platform.

[hectcastro]

I can confirm that --platform appears to be getting ignored on 1.6.3 and 1.7.0.

molecule.yml

---
ansible:
  config_file: ansible.cfg
  sudo: True

vagrant:
  platforms:
    - name: trusty64
      box: ubuntu/trusty64

    - name: xenial64
      box: ubuntu/xenial64

  providers:
    - name: virtualbox
      type: virtualbox

  instances:
    - name: ansible-java

Test

❯ molecule --version
1.7.0
❯ molecule test --platform xenial64                                                                                                                                               

playbook: playbook.yml
Bringing machine 'ansible-java' up with 'virtualbox' provider...
==> ansible-java: Preparing master VM for linked clones...
    ansible-java: This is a one time operation. Once the master VM is prepared,
    ansible-java: it will be used as a base for linked clones, making the creation
    ansible-java: of new VMs take milliseconds on a modern system.
==> ansible-java: Importing base box 'ubuntu/trusty64'...

Note that I asked for xenial64, but it is importing the ubuntu/trusty64 base box.

[kireledan]

Yeah sure thing. Oddly enough, last time I was testing it was working just fine.

[retr0h]

Cherry picked to stable/1.x branch. Will close once we release a 1.8.3 molecule.

[retr0h]

Released 1.8.3 to pypi today.

[retr0h]

Still not fixed.

$ molecule test --platform=all

--> Executing serverspec tests found in spec/.
*** Run serverspec on demo-01 instance ***
/Users/jodewey/.rvm/rubies/ruby-2.2.0/bin/ruby -I/Users/jodewey/.rvm/gems/ruby-2.2.0/gems/rspec-support-3.4.1/lib:/Users/jodewey/.rvm/gems/ruby-2.2.0/gems/rspec-core-3.4.1/lib /Users/jodewey/.rvm/gems/ruby-2.2.0/gems/rspec-core-3.4.1/exe/rspec --pattern spec/\*_spec.rb,spec/demo-01/\*_spec.rb,spec/hosts/demo-01/\*_spec.rb,spec/demo/\*_spec.rb,spec/groups/demo/\*_spec.rb
/Users/jodewey/git/molecule/demo/molecule/spec/spec_helper.rb:8:in `<top (required)>': undefined method `push' for nil:NilClass (NoMethodError)
        from /Users/jodewey/.rvm/rubies/ruby-2.2.0/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
        from /Users/jodewey/.rvm/rubies/ruby-2.2.0/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
        from /Users/jodewey/git/molecule/demo/molecule/spec/default_spec.rb:1:in `<top (required)>'
        from /Users/jodewey/.rvm/gems/ruby-2.2.0/gems/rspec-core-3.4.1/lib/rspec/core/configuration.rb:1361:in `load'
        from /Users/jodewey/.rvm/gems/ruby-2.2.0/gems/rspec-core-3.4.1/lib/rspec/core/configuration.rb:1361:in `block in load_spec_files'
        from /Users/jodewey/.rvm/gems/ruby-2.2.0/gems/rspec-core-3.4.1/lib/rspec/core/configuration.rb:1359:in `each'
        from /Users/jodewey/.rvm/gems/ruby-2.2.0/gems/rspec-core-3.4.1/lib/rspec/core/configuration.rb:1359:in `load_spec_files'
        from /Users/jodewey/.rvm/gems/ruby-2.2.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:102:in `setup'
        from /Users/jodewey/.rvm/gems/ruby-2.2.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:88:in `run'
        from /Users/jodewey/.rvm/gems/ruby-2.2.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:73:in `run'
        from /Users/jodewey/.rvm/gems/ruby-2.2.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:41:in `invoke'
        from /Users/jodewey/.rvm/gems/ruby-2.2.0/gems/rspec-core-3.4.1/exe/rspec:4:in `<main>'
/Users/jodewey/.rvm/rubies/ruby-2.2.0/bin/ruby -I/Users/jodewey/.rvm/gems/ruby-2.2.0/gems/rspec-support-3.4.1/lib:/Users/jodewey/.rvm/gems/ruby-2.2.0/gems/rspec-core-3.4.1/lib /Users/jodewey/.rvm/gems/ruby-2.2.0/gems/rspec-core-3.4.1/exe/rspec --pattern spec/\*_spec.rb,spec/demo-01/\*_spec.rb,spec/hosts/demo-01/\*_spec.rb,spec/demo/\*_spec.rb,spec/groups/demo/\*_spec.rb failed
ERROR:

  RAN: '/Users/jodewey/.rvm/rubies/ruby-2.2.0/bin/rake --rakefile=.molecule/rakefile'

[retr0h]

Close once #287 backported to stable stable/1.x and released to pypi as 1.8.5.

/cc @abrown-sg

[retr0h]

Closing we will release this fix to pypi as 1.9.0. Back porting this is going to be troublesome.