Can not get nodes from ansible hosts

[zwxk14]

My environment:

CentOS6.7 Rundeck-2.6.4.jar java version "1.7.0_67" ansible 1.9.4 ansible-plugin-1.0.0.jar.

My ansible hosts file:

[ansible_dev] AnsibleDev ansible_ssh_host=127.0.0.1 [Test] 192.168.20.10

I create a project followed your instructions, but can not get nodes from ansible hosts with this error:

2016-03-17 00:02:30,933 [qtp10900158-29] ERROR com.dtolabs.rundeck.core.resources.ExceptionCatchingResourceModelSource - [ResourceMo delSource: 1.com.batix.rundeck.AnsibleResourceModelSourceFactory (Ansible Resource Model Source), project: test-project] java.lang.UnsupportedOperationException: JsonObject at com.google.gson.JsonElement.getAsString(JsonElement.java:191) at com.batix.rundeck.AnsibleResourceModelSource.getNodes(AnsibleResourceModelSource.java:206) at com.dtolabs.rundeck.core.resources.ExceptionCatchingResourceModelSource.getNodes(ExceptionCatchingResourceModelSource.java:45) at com.dtolabs.rundeck.core.common.ProjectNodeSupport.getNodeSet(ProjectNodeSupport.java:88) at com.dtolabs.rundeck.core.common.IProjectNodes$getNodeSet.call(Unknown Source) at rundeck.services.nodes.CachedProjectNodes.reloadNodeSet(CachedProjectNodes.groovy:24) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:324) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1207) at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1110) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1074) at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1110) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1016)

Then, I turn off the option "Gather Facts", but still can't get nodes. The error is: 2016-03-17 00:26:36,951 [qtp10900158-31] ERROR com.dtolabs.rundeck.core.resources.ExceptionCatchingResourceModelSource - [ResourceModelSource: 1.com.batix.rundeck.AnsibleResourceModelSourceFactory (Ansible Resource Model Source), project: test-project] java.lang.NullPointerException at com.batix.rundeck.AnsibleResourceModelSource.getNodes(AnsibleResourceModelSource.java:206) at com.dtolabs.rundeck.core.resources.ExceptionCatchingResourceModelSource.getNodes(ExceptionCatchingResourceModelSource.java:45) at com.dtolabs.rundeck.core.common.ProjectNodeSupport.getNodeSet(ProjectNodeSupport.java:88) at com.dtolabs.rundeck.core.common.IProjectNodes$getNodeSet.call(Unknown Source) at rundeck.services.nodes.CachedProjectNodes.reloadNodeSet(CachedProjectNodes.groovy:24) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:324)

[frozenice]

I haven't tested with Ansible < v2, seems like there are some differences.

Please try the latest release!

[zwxk14]

nice work! it seems to have fixed the issue now.

Thanks!

[frozenice]

Good to hear! Feel free to close this issue once you confirmed this error is gone.

[rilindo]

I am running into the same issue. In this case, I am running Ansible within an EC2 instance, using IAM roles. Ansible is able to do inventory from AWS via the cli. The config as follows:

rundeck@rundeck:~$ cat .ansible.cfg

[defaults]
remote_user = ubuntu
timeout = 300
roles_path = /srv/rundeck/ansible/roles
callback_plugins = ~/.ansible/callback_plugins/

[ssh_connection]
scp_if_ssh = True

The ec2.py is in the /etc/ansible/inventory directory.

[frozenice]

@rilindo have you tried v1.0.1?

[rilindo]

@frozenice I am running 1.0.1:

rundeck@rundeck:~/libext$ uname -a
Linux rundeck.connectedhealth.com 3.13.0-79-generic #123~precise1-Ubuntu SMP Fri Feb 19 14:48:42 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
rundeck@rundeck:~/libext$ ls -la ansible-plugin-1.0.1.jar 
-rw-rw---- 1 rundeck rundeck 236494 Mar 17 20:34 ansible-plugin-1.0.1.jar
rundeck@rundeck:~/libext$ 

[frozenice]

@rilindo is there any stacktrace or error message?

Can your Rundeck user run Ansible? Try with something like su rundeck -s /bin/bash -c "ansible all -m ping"

[rilindo]

com.dtolabs.rundeck.core.NodesetEmptyException: No matched nodes: MultiNodeSelector{nodenames=[localhost]}
    at com.dtolabs.rundeck.core.execution.workflow.NodeFirstWorkflowStrategy.validateNodeSet(NodeFirstWorkflowStrategy.java:341)
    at com.dtolabs.rundeck.core.execution.workflow.NodeFirstWorkflowStrategy.executeWorkflowImpl(NodeFirstWorkflowStrategy.java:84)
    at com.dtolabs.rundeck.core.execution.workflow.BaseWorkflowStrategy.executeWorkflow(BaseWorkflowStrategy.java:182)
    at com.dtolabs.rundeck.core.execution.WorkflowExecutionServiceThread.run(WorkflowExecutionServiceThread.java:55)
Exception: class com.dtolabs.rundeck.core.NodesetEmptyException: No matched nodes: MultiNodeSelector{nodenames=[localhost]}
No matched nodes: MultiNodeSelector{nodenames=[localhost]}

Looking at the service.log, I see this:

https://gist.github.com/rilindo/8b2670ee4bbac07021c8

rundeck.log says:

https://gist.github.com/rilindo/6626a143186d9313d89a

Running with su command gives me:

root@rundeck:~# su rundeck -s /bin/bash -c "ansible all -m ping"
ERROR! Specified hosts options do not match any hosts

But that is to be expected, as the rundeck user is running under the root environment, which does not have the ansible configured. To run the above above command with the rundeck env (which has the Ansible ENV setup, you have to run:

root@rundeck:~# su - rundeck -s /bin/bash -c "ansible all -m ping"
THISIPREDACTED | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
SOMEOTHERIPREDACTED | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}

So does that mean the plugin runs as the root user or the rundeck user? Before you ask, yes, the service is running as the rundeck user:

rundeck   1542     1  0 Mar10 ?        00:00:00 su -s /bin/bash rundeck -c /usr/bin/java -Djava.security.auth.login.config=/etc/rundeck/jaas-loginmodule.conf ?-Dloginmodule.name=RDpropertyfilelogin ?-Drdeck.config=/etc/rundeck ?-Drdeck.base=/var/lib/rundeck ?-Drundeck.server.configDir=/etc/rundeck ?-Dserver.datastore.path=/var/lib/rundeck/data ?-Drundeck.server.serverDir=/var/lib/rundeck ?-Drdeck.projects=/var/rundeck/projects ?-Drdeck.runlogs=/var/lib/rundeck/logs ?-Drundeck.config.location=/etc/rundeck/rundeck-config.properties ?-Djava.io.tmpdir=/tmp/rundeck -Xmx1024m -Xms256m -XX:MaxPermSize=256m -server -cp /var/lib/rundeck/bootstrap/log4j-1.2.16.jar:/var/lib/rundeck/bootstrap/not-yet-commons-ssl-0.3.11.jar:/var/lib/rundeck/bootstrap/servlet-api-2.5.jar:/var/lib/rundeck/bootstrap/libpam4j-1.5.jar:/var/lib/rundeck/bootstrap/jetty-all-7.6.0.v20120127.jar:/var/lib/rundeck/bootstrap/jna-3.2.2.jar:/var/lib/rundeck/bootstrap/rundeck-jetty-server-2.6.1.jar: com.dtolabs.rundeck.RunServer /var/lib/rundeck 4440
rundeck   1576  1542  0 Mar10 ?        00:24:35 /usr/bin/java -Djava.security.auth.login.config=/etc/rundeck/jaas-loginmodule.conf -Dloginmodule.name=RDpropertyfilelogin -Drdeck.config=/etc/rundeck -Drdeck.base=/var/lib/rundeck -Drundeck.server.configDir=/etc/rundeck -Dserver.datastore.path=/var/lib/rundeck/data -Drundeck.server.serverDir=/var/lib/rundeck -Drdeck.projects=/var/rundeck/projects -Drdeck.runlogs=/var/lib/rundeck/logs -Drundeck.config.location=/etc/rundeck/rundeck-config.properties -Djava.io.tmpdir=/tmp/rundeck -Xmx1024m -Xms256m -XX:MaxPermSize=256m -server -cp /var/lib/rundeck/bootstrap/log4j-1.2.16.jar:/var/lib/rundeck/bootstrap/not-yet-commons-ssl-0.3.11.jar:/var/lib/rundeck/bootstrap/servlet-api-2.5.jar:/var/lib/rundeck/bootstrap/libpam4j-1.5.jar:/var/lib/rundeck/bootstrap/jetty-all-7.6.0.v20120127.jar:/var/lib/rundeck/bootstrap/jna-3.2.2.jar:/var/lib/rundeck/bootstrap/rundeck-jetty-server-2.6.1.jar: com.dtolabs.rundeck.RunServer /var/lib/rundeck 4440

[frozenice]

Well, Ansible is run as a new process under whatever environment Rundeck runs, see here.

You mentioned you need to use a login shell (the single - after su), but this option seems to be missing in your last ps output. So maybe Rundeck is not running under the right environment and can't access Ansible? The NoSuchFileException: /tmp/rundeck/ansible-hosts2835726529371937698/data suggests something like that.

Another thing I noticed from your first stacktrace (No matched nodes: MultiNodeSelector{nodenames=[localhost]}): I get the same one if I create a job and set it to "Execute locally", because my Ansible inventory doesn't have a node named like whatever Rundeck thinks is the local node's name and thus isn't available in Rundeck. I guess you get it, because you don't have any nodes in your Rundeck project because of the error with Ansible.

[rilindo]

That was it. I was using the stock rundeck user init file, so I thought that it is okay. Once I added the login shell, it was able to pull the nodes.

There are other issues, but I'll just open a new issue. Thanks!

[frozenice]

Awesome! :)

Closing this one.

[rangsan]

@rilindo I face the same issue you had. Could you please explain more on below statement ? Which file I have to update and what is it? Thank you very much.

That was it. I was using the stock rundeck user init file, so I thought that it is okay. Once I added the login shell, it was able to pull the nodes.