Add jvm stats collection

[sysadmind]

This uses the admin/metrics API endpoint to collect JVM stats. This does not appear to be available for solr4, but the solr6 docs do include this endpoint.I'm not sure about solr5. This moves the http client setup and solr base URL building to main.go so that both the existing exporter and the jvm exporter can share the config.

[noony]

Hi, Thanks for contibuting. I will review an merge it this night.

[noony]

Hi& @sysadmind , I got this error :

ERRO[0003] Failed to collect metrics: Failed to unmarshal solr jvm JSON into struct: json: cannot unmarshal object into Go struct field .gc.ConcurrentMarkSweep.count of type int64  source="jvm.go:362"

It seems that all fields returned by this endpoint got a ['value'] field (tested with last solr6) :

{"responseHeader":{"status":0,"QTime":27},"metrics":{"solr.jvm":{"buffers.direct.Count":{"value":10},"buffers.direct.MemoryUsed":{"value":106496},"buffers.direct.TotalCapacity":{"value":106496},"buffers.mapped.Count":{"value":0},"buffers.mapped.MemoryUsed":{"value":0},"buffers.mapped.TotalCapacity":{"value":0},"classes.loaded":{"value":5938},"classes.unloaded":{"value":0},"gc.ConcurrentMarkSweep.count":{"value":1},"gc.ConcurrentMarkSweep.time":{"value":47},"gc.ParNew.count":{"value":5},"gc.ParNew.time":{"value":152},"memory.heap.committed":{"value":514523136},"memory.heap.init":{"value":536870912},"memory.heap.max":{"value":514523136},"memory.heap.usage":{"value":0.1806011537642498},"memory.heap.used":{"value":92923472},"memory.non-heap.committed":{"value":48451584},"memory.non-heap.init":{"value":2555904},"memory.non-heap.max":{"value":-1},"memory.non-heap.usage":{"value":-4.6175728E7},"memory.non-heap.used":{"value":46175728},"memory.pools.CMS-Old-Gen.committed":{"value":402653184},"memory.pools.CMS-Old-Gen.init":{"value":402653184},"memory.pools.CMS-Old-Gen.max":{"value":402653184},"memory.pools.CMS-Old-Gen.usage":{"value":0.0},"memory.pools.CMS-Old-Gen.used":{"value":0},"memory.pools.CMS-Old-Gen.used-after-gc":{"value":0},"memory.pools.Code-Cache.committed":{"value":10027008},"memory.pools.Code-Cache.init":{"value":2555904},"memory.pools.Code-Cache.max":{"value":251658240},"memory.pools.Code-Cache.usage":{"value":0.0366058349609375},"memory.pools.Code-Cache.used":{"value":9212160},"memory.pools.Compressed-Class-Space.committed":{"value":4272128},"memory.pools.Compressed-Class-Space.init":{"value":0},"memory.pools.Compressed-Class-Space.max":{"value":1073741824},"memory.pools.Compressed-Class-Space.usage":{"value":0.0036751776933670044},"memory.pools.Compressed-Class-Space.used":{"value":3946192},"memory.pools.Metaspace.committed":{"value":34152448},"memory.pools.Metaspace.init":{"value":0},"memory.pools.Metaspace.max":{"value":-1},"memory.pools.Metaspace.usage":{"value":0.9667645493523627},"memory.pools.Metaspace.used":{"value":33017376},"memory.pools.Par-Eden-Space.committed":{"value":89522176},"memory.pools.Par-Eden-Space.init":{"value":89522176},"memory.pools.Par-Eden-Space.max":{"value":89522176},"memory.pools.Par-Eden-Space.usage":{"value":0.8814111042162335},"memory.pools.Par-Eden-Space.used":{"value":78905840},"memory.pools.Par-Eden-Space.used-after-gc":{"value":0},"memory.pools.Par-Survivor-Space.committed":{"value":22347776},"memory.pools.Par-Survivor-Space.init":{"value":22347776},"memory.pools.Par-Survivor-Space.max":{"value":22347776},"memory.pools.Par-Survivor-Space.usage":{"value":0.6272495303335777},"memory.pools.Par-Survivor-Space.used":{"value":14017632},"memory.pools.Par-Survivor-Space.used-after-gc":{"value":14017632},"memory.total.committed":{"value":562974720},"memory.total.init":{"value":539426816},"memory.total.max":{"value":514523135},"memory.total.used":{"value":139099200},"os.arch":{"value":"amd64"},"os.availableProcessors":{"value":2},"os.committedVirtualMemorySize":{"value":3126353920},"os.freePhysicalMemorySize":{"value":130756608},"os.freeSwapSpaceSize":{"value":1042546688},"os.maxFileDescriptorCount":{"value":1048576},"os.name":{"value":"Linux"},"os.openFileDescriptorCount":{"value":185},"os.processCpuLoad":{"value":0.0016191709844559587},"os.processCpuTime":{"value":13840000000},"os.systemCpuLoad":{"value":0.003670120898100173},"os.systemLoadAverage":{"value":0.0},"os.totalPhysicalMemorySize":{"value":2096066560},"os.totalSwapSpaceSize":{"value":1073737728},"os.version":{"value":"4.9.87-linuxkit-aufs"},"threads.blocked.count":{"value":0},"threads.count":{"value":21},"threads.daemon.count":{"value":7},"threads.deadlock.count":{"value":0},"threads.deadlocks":{"value":[]},"threads.new.count":{"value":0},"threads.runnable.count":{"value":6},"threads.terminated.count":{"value":0},"threads.timed_waiting.count":{"value":11},"threads.waiting.count":{"value":4}}}}

Which version do you had tested ?

[sysadmind]

Interesting. I don't have solr6 to test, but I am running sorl-spec 7.2.1. I might be able to find a docker container for 6 to try and run locally to test it out.

[noony]

Oh ok, Maybe it's working on solr7, let me try.

[noony]

Ok It's working with solr7. I will merge it and add little fixes. Thanks.

[fabriziofortino]

+1

I confirm solr6 has a value field. Here is a snippet.

metrics: {
  solr.jvm: {
    buffers.direct.Count: {
      value: 273
    },
    buffers.direct.MemoryUsed: {
      value: 10787456
    },
    buffers.direct.TotalCapacity: {
      value: 10787456
    }
  }
}

It would be nice to have support for both solr6 and solr7.

[Stono]

Would love to see this in :)

[noony]

I'm merging this. And make it works for both for solr 6 and 7 versions. Thanks !

[Stono]

Amaaaaazzzinnggg

On Wed, 4 Jul 2018, 8:21 pm Thomas Colomb, notifications@github.com wrote:

Merged #12 https://github.com/noony/prometheus-solr-exporter/pull/12.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/noony/prometheus-solr-exporter/pull/12#event-1716834394, or mute the thread https://github.com/notifications/unsubscribe-auth/ABaviYstVnX5z7_GXrY67CUbplSxIe58ks5uDRXQgaJpZM4TNJom .

[Stono]

Could you do a release / push to docker hub?