No nodes exist?

[witchbutter]

I set this up on a running master that has at least 65 nodes reporting to it all times but all I get is this as output:

time="2020-02-28T05:59:18Z" level=error msg="failed to get nodes: failed to get nodes: failed to unmarshal: invalid character '<' looking for beginning of value"

I do not really understand what this error means. The master has postgres and puppetdb running locally, but I set it up to hit the https endpoint in jetty.ini. There is nothing in the way the config is written that suggests I could use a different connection than this. I also am not clear what cert would be used other than the certs specified in jetty.ini. I think I've eliminated ssl as an issue by using PUPPETDB_SSL_SKIP_VERIFY=true

Versions:

root@puppet:~# dpkg -l | grep puppet
ii  puppet-agent                   6.13.0-1bionic                      amd64        The Puppet Agent package contains all of the elements needed to run puppet, including ruby, facter, and hiera.
ii  puppet6-release                6.0.0-5bionic                       all          Release packages for the Puppet 6 repository
ii  puppetdb                       6.9.0-1bionic                       all          Puppet Labs puppetdb
ii  puppetdb-termini               6.9.0-1bionic                       all          Termini for puppetdb
ii  puppetserver                   6.9.0-1bionic                       all          Puppet Labs puppetserver

[trondk]

I seems to have the same problem, is it a puppetdb version issue ?

[raphink]

Does the PuppetDB produce any useful logs?

[trondk]

==> puppetdb-access.log <== 10.1.79.34 - - [04/Jun/2020:07:39:43 +0200] "GET /v4/nodes?query=%5B%22or%22%2C+%5B%22%3D%22%2C+%5B%22node%22%2C+%22active%22%5D%2C+false%5D%2C+%5B%22%3D%22%2C+%5B%22node%22%2C+%22active%22%5D%2C+true%5D%5D HTTP/1.1" 404 240 "-" "Go-http-client/1.1" 0 10.1.79.34 - - [04/Jun/2020:07:39:48 +0200] "GET /v4/nodes?query=%5B%22or%22%2C+%5B%22%3D%22%2C+%5B%22node%22%2C+%22active%22%5D%2C+false%5D%2C+%5B%22%3D%22%2C+%5B%22node%22%2C+%22active%22%5D%2C+true%5D%5D HTTP/1.1" 404 240 "-" "Go-http-client/1.1" 0

[trondk]

so no not really.....

[raphink]

Well at least we can tell it's a 404, not a 403 or 500, so that helps already.

[raphink]

What are you passing as the PuppetDB URL?

It needs to finish with /pdb/query, e.g. https://puppetdb:8081/pdb/query.

Granted, the README is not very clear on that.

[witchbutter]

I see the same error in the access log:

172.17.0.3 - - [04/Jun/2020:15:53:07 +0000] "GET /v4/nodes?query=%5B%22or%22%2C+%5B%22%3D%22%2C+%5B%22node%22%2C+%22active%22%5D%2C+false%5D%2C+%5B%22%3D%22%2C+%5B%22node%22%2C+%22active%22%5D%2C+true%5D%5D HTTP/1.1" 404 240 "-" "Go-http-client/1.1" 0

There are no entries in puppetdb.log

[witchbutter]

The suggested addition of /pdb/query works for me. I am actually running it in docker on the master:

  docker::run{ 'puppetdb-exporter':
    image => 'camptocamp/prometheus-puppetdb-exporter',
    ports => [ '9121:9121' ],
    volumes => [ '/etc/puppetlabs/puppetdb/ssl:/etc/puppetlabs/puppetdb/ssl' ],
    memory_limit => '512m',
    env => [
      "PUPPETDB_URL=https://${trusted['certname']}:8081/pdb/query",
      'PUPPETDB_CA_FILE=/etc/puppetlabs/puppetdb/ssl/ca.pem',
      'PUPPETDB_KEY_FILE=/etc/puppetlabs/puppetdb/ssl/private.pem',
      'PUPPETDB_CERT_FILE=/etc/puppetlabs/puppetdb/ssl/public.pem',
      'PUPPETDB_SSL_SKIP_VERIFY=true',
      'PUPPETDB_SCRAPE_INTERVAL=60s'
    ],
    restart_service => true,
  }

I see the successful accesses:

172.17.0.3 - - [04/Jun/2020:16:03:57 +0000] "GET /pdb/query/v4/reports/5eabe499266aae467c0a35b0236077a50387db4b/metrics HTTP/1.1" 200 2798 "-" "Go-http-client/1.1" 6

And curl of the endpoint now shows data.