Option to disable Serverspec / Testinfra tests

[tknerr]

I switched the testinfra_dir to "spec" because I'm using the pytest-spec formatter, and naming it spec is more natural for me ;-)

---
molecule:
    testinfra_dir: spec

Now this fails because it believes there are also serverspec tests in it, but I don't even have rubocop and serverspec installed:

Y:\repo\devops\base-jenkins-master>molecule verify                                                        
--> Executing testinfra tests found in spec/.                                                             
============================= test session starts ==============================                          
platform cygwin -- Python 2.7.10, pytest-2.9.2, py-1.4.31, pluggy-0.3.1                                   
rootdir: /cygdrive/y/repo/devops/base-jenkins-master/spec, inifile:                                       
plugins: xdist-1.14, testinfra-1.4.1                                                                      
collected 1 itemss                                                                                        

spec/test_hostsfile.py .                                                                                  

=========================== 1 passed in 1.13 seconds ===========================                          
--> Executing rubocop on *.rb files found in spec/.                                                       
Traceback (most recent call last):                                                                        
  File "/usr/bin/molecule", line 11, in <module>                                                          
    sys.exit(main())                                                                                      
  File "/usr/lib/python2.7/site-packages/molecule/cli.py", line 68, in main                               
    CLI().main()                                                                                          
  File "/usr/lib/python2.7/site-packages/molecule/cli.py", line 64, in main                               
    utilities.sysexit(c.execute()[0])                                                                     
  File "/usr/lib/python2.7/site-packages/molecule/commands/verify.py", line 96, in execute                
    validators.rubocop(serverspec_dir, **serverspec_kwargs)                                               
  File "/usr/lib/python2.7/site-packages/molecule/validators.py", line 139, in rubocop                    
    return sh.rubocop(match, **kwargs)                                                                    
  File "/usr/lib/python2.7/site-packages/sh.py", line 2301, in __getattr__                                
    return self.__env[name]                                                                               
  File "/usr/lib/python2.7/site-packages/sh.py", line 2232, in __getitem__                                
    return Command._create(k, **self.baked_args)                                                          
  File "/usr/lib/python2.7/site-packages/sh.py", line 776, in _create                                     
    raise CommandNotFound(program)                                                                        
sh.CommandNotFound: rubocop                                                                               

It would be nice if there were a flag in the config to disable the serverspec / rubocop / testinfra validators.

Workaround is to set the serverspec_dir to a non-existing directory:

---
molecule:
    serverspec_dir: does-not-exist
    testinfra_dir: spec

Resulting in:

Y:\repo\devops\base-jenkins-master>molecule verify
--> Executing testinfra tests found in spec/.
============================= test session starts ==============================
platform cygwin -- Python 2.7.10, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /cygdrive/y/repo/devops/base-jenkins-master/spec, inifile:
plugins: xdist-1.14, testinfra-1.4.1
collected 1 itemss

spec/test_hostsfile.py .

=========================== 1 passed in 1.27 seconds ===========================
No serverspec tests found in does-not-exist/.

[kireledan]

I can definitely see a reason to skip those. For now, you can actually just skip the verify in test by customizing the test cycle.

molecule:
  test:
    # sequence of commands to run when performing `molecule test`
    sequence:
      - destroy
      - syntax
      - create
      - converge
      - idempotence
      #- verify

You can put in every step except for verify in either your local config(~/.config/molecule/config.yml or project config to skip the verify step.

[retr0h]

We have already stated before we aren't going to disable serverspec / rubocop / testinfra validators. Molecule is opinionated. Doing things differently is going to have limited mileage. We can't account for every use case.

@kireledan provided a work-a-round.

[tknerr]

@retr0h :+1:

Happy with just setting the serverspec_dir to a non-existent directory as a workaround. Also, after reading the docs again, it clearly says "test" is for testinfra and "spec" where it looks for serverspec. So my case is really a bit against the recommendation, so no need to account for that ;)