Ansible lint is executed for every file

[mcico]

Hey,

We started to use ansible-lint in combination with sonar. When we activate the sonar plugin (by picking the quality profile containing the ansible rules) the sonar execution takes a lot more time.

My understanding is that the sonar plugin runs ansible-lint for every file that is given as input and therefore starts a lot of ansible-lint processes. When running ansible-lint on the command line it checks all the files at once.

If my understanding is correct, would there be a way to run ansible-lint just once instead of multiple times? Here is an excerpt of an execution log

13:55:46.594 DEBUG: Ansible sensor executed with context: org.sonar.scanner.sensor.ModuleSensorContext@6f915016
13:55:46.594 DEBUG: Analyzing file: demo-playbook.yml
13:55:46.595 DEBUG: Executing command: [ansible-lint, -p, --nocolor, /home/mcico/projects/deleng/deleng-demo-ansible/demo-playbook.yml]
13:55:46.595 DEBUG: Work directory: /home/mcico/projects/deleng/deleng-demo-ansible
13:55:49.396 WARN: Errors happened during analysis:
Added ANSIBLE_ROLES_PATH=~/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:roles
Loading custom .yamllint config file, this extends our internal yamllint config.
13:55:49.397 DEBUG: 0 issue(s) found
13:55:49.399 DEBUG: Analyzing file: main.yml
13:55:49.400 DEBUG: Executing command: [ansible-lint, -p, --nocolor, /home/mcico/projects/deleng/deleng-demo-ansible/roles/demo/tasks/main.yml]
13:55:49.400 DEBUG: Work directory: /home/mcico/projects/deleng/deleng-demo-ansible
13:55:51.460 WARN: Errors happened during analysis:
Added ANSIBLE_ROLES_PATH=~/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:roles
Loading custom .yamllint config file, this extends our internal yamllint config.
13:55:51.461 DEBUG: 0 issue(s) found
13:55:51.461 DEBUG: Saving issues for file:///home/mcico/projects/deleng/deleng-demo-ansible/demo-playbook.yml
13:55:51.463 DEBUG: Saving issues for file:///home/mcico/projects/deleng/deleng-demo-ansible/roles/demo/tasks/main.yml
13:55:51.464 INFO: Sensor Ansible Lint Sensor [ansible] (done) | time=4870ms
13:55:51.464 INFO: Sensor JavaXmlSensor [java]
13:55:51.464 INFO: Sensor JavaXmlSensor [java] (done) | time=0ms
13:55:51.465 INFO: Sensor HTML [web]
13:55:51.468 INFO: Sensor HTML [web] (done) | time=3ms
13:55:51.468 INFO: Sensor YAML Sensor [yaml]
13:55:51.468 DEBUG: YAML sensor executed with context: org.sonar.scanner.sensor.ModuleSensorContext@6f915016
13:55:51.468 DEBUG: Parsing error rule key found: null
13:55:51.469 DEBUG: Analyzing file: demo-playbook.yml
13:55:51.473 DEBUG: Evaluate issue exclusions for 'demo-playbook.yml'
13:55:51.477 INFO: Load project repositories
13:55:51.500 DEBUG: GET 200 http://sonar-prod.zh.adnovum.ch:9000/sonar/batch/project.protobuf?key=deleng%3Adeleng-demo-ansible | time=23ms
13:55:51.506 INFO: Load project repositories (done) | time=29ms
13:55:51.506 DEBUG: 'demo-playbook.yml' generated metadata with charset 'UTF-8'
13:55:51.554 DEBUG: File file:///home/mcico/projects/deleng/deleng-demo-ansible/demo-playbook.yml has syntax error? false
13:55:51.555 DEBUG: Count lines in demo-playbook.yml

In this particular case ansible-lint is started twice, once for the playbook and once for the main.yml file in the demo role. For small projects this is no big deal, but for projects with many files the execution time increases dramatically.

cheers & thanks, marco

[sbaudoin]

Hello,

This is something I'm aware of and unfortunately I have not found a way yet to prevent this. The reason is that it is hard to guess if a file needs to be analyzed or not: is it a playbook? Is it a role? Is this role called from a playbook that is going to be analyzed? If not it needs to be analyzed on its own... The simplest approach I've found without going into the details of includes, playbooks and so on is to analyze all files.

If you can figure out a way to simplify this I'll be happy to have your input.

Regards,

Sylvain

[mcico]

I understand. I will have a look to get a better understanding and if I come up with an idea I will let you know.

[LvffY]

Hello

I fall into this issue because we have very large inventories/playbooks repositories (>1000 files) and therefore our analysis are quite long (>1 hour).

When I run Ansible lint CLI in a local environment, each files are analysed regarding of their types (i.e playbook are not analysed as simple "YAML files").

I didn't try on too specific stuff (like roles or collections) but it seems that the ansible-lint CLI is clever enough to guess the type of each file.

Moreover, if you use Ansible lint configuration files, you have the kind anchor where you can define some specific file type you'd have to analyse.

@sbaudoin May be if you could give us an example of analysis that would fail without the "analyse each file", we could start looking for something more "clever" ?

N.B: Apart of this corner case, your plugin is great ! :)