Rendering SLS 'base:hadoop.hdfs' failed: Jinja variable No first item, sequence was empty.

[kitplummer]

I'm pretty sure it's my pillar config, but I can't get it. Anybody else see this when trying to use hadoop.hdfs?

hdfs:
  config:
    hdfs-site:
      dfs.datanode.synconclose:
        value: true
      dfs.durable.sync:
        value: true
      dfs.permission:
        value: false
    namenode_http_port: 50070
    namenode_port: 8020
    secondarynamenode_http_port: 50090
  datanode_target: "roles:hadoop_slave"
  namenode_target: "roles:hadoop_master"

[sroegner]

What happens if you try to run completely without the pillar block?

[kitplummer]

It still happens. I've made a little bit of progress though.

Caveat - I'm trying to get a master and slave working under Vagrant and two VMs. It boils down to being able to resolve this:

{%- set namenode_host       = salt['mine.get'](namenode_target,
'network.interfaces', expr_form=targeting_method)|first %}

The error is basically saying that first can't return anything because there isn't anything there at all. I have the role getting set in the minion config, but I'm not sure how to get the mine_functions stuff to work in this enviro.

I'm just guessing that because I'm running 'masterless' with a local setup that I'm stuck.

On Wed, Mar 4, 2015 at 4:20 PM, Steffen Roegner notifications@github.com wrote:

What happens if you try to run completely without the pillar block?

— Reply to this email directly or view it on GitHub https://github.com/saltstack-formulas/hadoop-formula/issues/20#issuecomment-77272623 .

[sroegner]

You might have guessed that what the code is trying to do is to collect the list of hosts that match expression and targeting_method (in your case the default - grains). I always use a saltmaster - even in vagrant so I cannot speak to the chances of querying salt mine without one. What you could try is change "hadoop:targeting_method" to something like compound and then adapt the targets to that like below:

hadoop:
  targeting_method: compound
hdfs:
  datanode_target: "G@hostname:my_vagrant_datanode"
  namenode_target: "G@hostname:my_vagrant_namenode"
  config:
    hdfs-site:
      dfs.datanode.synconclose:
        value: true
      dfs.durable.sync:
        value: true
      dfs.permission:
        value: false
    namenode_http_port: 50070
    namenode_port: 8020
    secondarynamenode_http_port: 50090

[kitplummer]

Ah. That might be better/easier than what I ended up doing:

{%- set namenode_host       = gc.get('namenode_host',
pc.get('namenode_host', salt['mine.get'](namenode_target,
'network.interfaces', expr_form=targeting_method))) %}
{%- set datanode_hosts      = gc.get('datanode_hosts',
pc.get('datanode_hosts', salt['mine.get'](datanode_target,
'network.interfaces', expr_form=targeting_method).keys())) %}

And manually spelling 'em out in the pillar config:

    namenode_host: 192.168.100.100
    datanode_hosts: [192.168.100.200]

On Thu, Mar 5, 2015 at 7:09 AM, Steffen Roegner notifications@github.com wrote:

You might have guessed that what the code is trying to do is to collect the list of hosts that match expression and targeting_method (in your case the default - grains). I always use a saltmaster - even in vagrant so I cannot speak to the chances of querying salt mine without one. What you could try is change "hadoop:targeting_method" to something like compound and then adapt the targets to that like below:

hadoop: targeting_method: compound hdfs: datanode_target: "G@hostname:my_vagrant_datanode" namenode_target: "G@hostname:my_vagrant_namenode" config: hdfs-site: dfs.datanode.synconclose: value: true dfs.durable.sync: value: true dfs.permission: value: false namenode_http_port: 50070 namenode_port: 8020 secondarynamenode_http_port: 50090

— Reply to this email directly or view it on GitHub https://github.com/saltstack-formulas/hadoop-formula/issues/20#issuecomment-77369103 .

[blbradley]

I'm experiencing this too right out of the box. I have tried with and without the pillar block from the example. Can we call it a bug? Even if it is just a documentation issue.

[blbradley]

FWIW, I'm not using a masterless setup.

[sroegner]

@blbradley Can you share the result of 'salt * grains.get roles' ?

[blbradley]

I have this working now. My minion config did have the roles set but was still getting this. I am not able to reproduce again.

Can this error be put in a troubleshooting section? It's hard to debug because no real contextual info (line number, rendered jinja) is given.

[blbradley]

I'll do it if someone says it's alright.

[puneetk]

send a pull request with the update

[lextoumbourou]

I ran into the same issue. I had my Pillar conf setup like this:

hdfs:
    namenode_target: "role:hadoop-namenode"
    datanode_target: "role:hadoop-datanode"

excepting the default grain matching to work, but it doesn't seem to? I was getting the Jinja variable No first item, sequence was empty. error.

I changed the Pillar to this:

hadoop:
    targeting_method: compound

 hdfs:
   namenode_target: "G@role:hadoop-namenode"
   datanode_target: "G@role:hadoop-datanode"

and it worked, though I'm not sure if it was the config change or if Salt cleared its cache or something?

[sroegner]

@lextoumbourou FWIW: when these problems come up (with the ever same unhelpful error message) I'd try to investigate a bit on the command line trying to figure out what to expect from the salt setup and then maybe change the files. Comparing

salt -C 'G@role:hadoop-namenode' test.ping

with

salt -G 'role:hadoop-namenode' test.ping

should be the equivalent of the two examples you gave. In my experience the targeting depends on both the salt version mix in use and the current state of the cluster - at the same time the jinja code doesn't really allow for error handling. The good news: it usually works pretty reliably once all the pieces are in place.

hth