search

mjtrangoni / flexlm_exporter

Prometheus exporter for FLEXlm License Manager information
Apache License 2.0
83 stars 30 forks source link

unsupported value type #80

Closed cloudrck closed 1 year ago

cloudrck commented 1 year ago

Host operating system: output of uname -a: Linux 4.18.0-477.15.1.el8_8.x86_64 #1 SMP Fri Jun 2 08:27:19 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux

flexlm_exporter version: output of flexlm_exporter --version

flexlm_exporter, version  (branch: non-git, revision: non-git)
  build user:       myuser@918d31421ff1
  build date:       20231002-16:54:34
  go version:       go1.20.7
  platform:         linux/amd64
  tags:             netgo static_build

Current master copy built from source

flexlm_exporter command line flags and relevant YAML configuration file information

--path.config="/opt/flexlm_exporter/licenses.yml" \
    --path.lmutil="/opt/flexlm_exporter/lmutil" \
    --log.level=debug

Are you running flexlm_exporter in Docker?

No

What did you do that produced an error?

start flexlm daemon

What did you expect to see?

metrics from license servers

What did you see instead?

the /metrics endpoint only shows the stock Go mem statistics. No info from license servers

Oct 02 18:55:16 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:55:16.183Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:53:44 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:53:44.203Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:48:44 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:48:44.203Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:43:44 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:43:44.202Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:38:44 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:38:44.202Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:33:44 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:33:44.203Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:28:45 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:28:45.394Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:28:44 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:28:44.206Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:26:02 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:26:02.906Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:25:28 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:25:28.625Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=0.0.0.0:9319
Oct 02 18:25:28 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:25:28.625Z caller=tls_config.go:274 level=info msg="Listening on" address=0.0.0.0:9319
Oct 02 18:25:28 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:25:28.616Z caller=config.go:49 level=info -=/opt/flexlm_exporter/licenses.yml
Oct 02 18:25:28 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:25:28.616Z caller=config.go:48 level=info msg="Loading license config file:"
Oct 02 18:25:28 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:25:28.616Z caller=flexlm_exporter.go:123 level=info msg="Enabled collectors"
Oct 02 18:25:28 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:25:28.616Z caller=flexlm_exporter.go:202 level=warn msg="FLEXlm Exporter is running as root user. This exporter is designed to run as unprivileged user, root is not required."
Oct 02 18:25:28 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:25:28.616Z caller=flexlm_exporter.go:200 level=info msg="Build context" build_context="(go=go1.20.7, platform=linux/amd64, user=myuser@918d31421ff1, date=20231002-16:54:34, tags=netgo static_build)"
Oct 02 18:25:28 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:25:28.616Z caller=flexlm_exporter.go:199 level=info msg="Starting flexlm_exporter" version="(version=, branch=non-git, revision=non-git)"
Oct 02 18:25:28 example.server.net systemd[1]: Started flexlm_exporter for Prometheus.

Thanks to the bug fix for #77 I can start the flexlm daemon.

However, now, there appears to be issue getting the actual information to pull. The licenses.yml is the same one I used in previous versions without issue.

licenses:
  - name: server1
    license_server: 1055@server1
    monitor_users: True
    monitor_reservations: True
  - name: server2
    license_server: 27000@server2
    monitor_users: True
    monitor_reservations: True

It appears the changes between versions is not liking the data retrieved from lmutil.

This is the version of lmutil:


[~]# ./lmutil lmstat -v
lmutil - Copyright (c) 1989-2018 Flexera. All Rights Reserved.
lmstat v11.16.2.0 build 242433 x64_lsb

Is this an issue where compatibility is broken with my version of lmutil?

mjtrangoni commented 1 year ago

@cloudrck thanks for reporting. Indeed it could be related to your lmutil output. unsupported value type is not related to your issue, but more a logging library problem.

If you could check the lmutil output against the format of the fixtures here, which are the one this exporter is tested against, you should note the differences. Let me know if you find a different format there.

cloudrck commented 1 year ago

Thanks for the reply and info.

Here is the output of about 3 license servers.

[~ flexlm_exporter]# ./lmutil lmstat -c 4100@l-server01 -a
lmutil - Copyright (c) 1989-2018 Flexera. All Rights Reserved.
Flexible License Manager status on Mon 10/2/2023 18:32

License server status: 4100@license-server01
    License file(s) on license-server01: C:\Magicdraw_license_manager\license.lic:

license-server01: license server UP (MASTER) v11.19.0

Vendor daemon status (on license-server01):

     cameo: UP v11.19.0
Feature usage info:

Users of CC888888Connections:  (Total of 2 licenses issued;  Total of 2 licenses in use)

  "CC888888Connections" v2022, vendor: cameo, expiry: 02-jun-2025
  vendor_string: product=CC4TWC|profile=0|edition=10\u0020Connections|version=Commercial\u0020with\u0020expiration|versionNumber=2022x|licenseType=Floating|keyVersion=30|protected=fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff|userLimit=10|user=GENERAL\u0020DYNAMICS\u0020MISSION\u0020SYSTEMS\u0020INC|productType=Application|
  floating license

    license-server002$ license-server002 JavaDisplay (v2022) (license-server01/4100 4501), start Sun 9/24 13:21
    root ip-.compute.internal JavaDisplay (v2022) (license-server01/4100 15509), start Wed 9/27 13:16

++++++++++++++
[~ flexlm_exporter]# ./lmutil lmstat -c 4100@l-server01 -i
lmutil - Copyright (c) 1989-2018 Flexera. All Rights Reserved.
Mon 10/2/2023 20:46

NOTE: lmstat -i does not give information from the server,
      but only reads the license file.  For this reason,
      lmstat -a is recommended instead.

Feature                         Version     #licenses    Vendor        Expires
_______                         _________   _________    ______        ________
CC888888Connections             2022         2           cameo         02-jun-2025
CC999999Connections              2022         1           cameo         02-jun-2025
CC000000Connections              2022         1           cameo         02-jun-2025

=======================
[~ flexlm_exporter]# ./lmutil lmstat -c 1055@appserver99 -i
lmutil - Copyright (c) 1989-2018 Flexera. All Rights Reserved.
Mon 10/2/2023 20:55

NOTE: lmstat -i does not give information from the server,
      but only reads the license file.  For this reason,
      lmstat -a is recommended instead.

Feature                         Version     #licenses    Vendor        Expires
_______                         _________   _________    ______        ________
a_spaceclaim_dirmod             9999.9999    2           ansyslmd      31-may-2024
a_spaceclaim_dirmod             9999.9999    3           ansyslmd      31-may-2024
a_spaceclaim_dirmod             9999.9999    3           ansyslmd      31-may-2024
a_spaceclaim_dirmod             9999.9999    30          ansyslmd      30-jun-2023
a_spaceclaim_dirmod             9999.9999    1           ansyslmd      31-may-2024
ema_gui                         9999.9999    2           ansyslmd      31-may-2024
ema_solve                       9999.9999    2           ansyslmd      31-may-2024
lum_eme_solve                   9999.9999    1           ansyslmd      27-nov-2023
lum_eme_solve                   9999.9999    1           ansyslmd      31-may-2024
lum_eme_solve                   9999.9999    1           ansyslmd      31-may-2024
lum_fde_solve                   9999.9999    1           ansyslmd      27-nov-2023

================================
[~ flexlm_exporter]# ./lmutil lmstat -c 19353@my-license -i
lmutil - Copyright (c) 1989-2018 Flexera. All Rights Reserved.
Mon 10/2/2023 21:34

NOTE: lmstat -i does not give information from the server,
      but only reads the license file.  For this reason,
      lmstat -a is recommended instead.

Feature                         Version     #licenses    Vendor        Expires
_______                         _________   _________    ______        ________
TLSTOK                          2.0          2400        ibmratl       30-dec-2026
Change                          5000.32      1           telelogic     30-dec-2026
Change-Base                     5000.32      1           telelogic     30-dec-2026
Change-Distributed              5000.32      1           telelogic     30-dec-2026
DCCC                            9000.62      1           telelogic     30-dec-2026
DCCQ                            9000.62      1           telelogic     30-dec-2026
DCTD                            9000.62      1           telelogic     30-dec-2026
DOORS                           9000.62      1           telelogic     30-dec-2026
DRL                             9000.62      1           telelogic     30-dec-2026
DWA_EDIT                        9000.62      1           telelogic     30-dec-2026
DWA_Review                      9000.62      1           telelogic     30-dec-2026
Reporter                        8000.132     1           telelogic     30-dec-2026

+++++++++
[~ flexlm_exporter]# ./lmutil lmstat -c 19353@my-license -a
lmutil - Copyright (c) 1989-2018 Flexera. All Rights Reserved.
Flexible License Manager status on Mon 10/2/2023 21:34

License server status: 19353@license-server01
    License file(s) on license-server01: /data/app/server/config/server_license.lic:

license-server01: license server UP (MASTER) v11.16.2

Vendor daemon status (on license-server01):

   ibmratl: UP v11.16.2
Feature usage info:

Users of TLSTOK:  (Total of 2400 licenses issued;  Total of 1050 licenses in use)

  "TLSTOK" v2.0, vendor: ibmratl, expiry: 30-dec-2026
  floating license

    svc.useris.JazzServic eadc-e-clmprd03 /dev/tty #u#svc.useris.JazzServic#u##s#JAZZ(77847c02-c045-4a34-9ea2-c6c2d695d55a)#s##p#Rational Quality Manager#p##jazzrole#QualityProfessional#jazzrole# (v2.0) (license-server01/19353 101), start Sun 9/17 11:31, 10 licenses  (linger: 0 / 1800)
    user.last1 eadc-e-clmprd03 /dev/tty #u#user.last1#u##s#JAZZ(fa050840-e4d7-42b7-b4bf-3505277870cd)#s##p#Rational Team Concert#p##jazzrole#Contributor#jazzrole# (v2.0) (license-server01/19353 201), start Sun 9/17 11:31, 5 licenses  (linger: 0 / 1800)
    another-user2 mmmmmm-appp02 mmmmmm-appp02 ClearCase (v2.0) (license-server01/19353 701), start Sun 9/17 11:31, 5 licenses  (linger: 0 / 1800)
    LQE Functional User Key another-server01 another-server01 DOORS#u#LQE Functional User Key#u# (v1.0) (license-server01/19353 13402), start Mon 10/2 5:08, 10 licenses
cloudrck commented 1 year ago

Running in a container gives me the same issue. Same flexlm flags as above

Linux 5.15.0-1035-azure #42-Ubuntu SMP Tue Feb 28 19:41:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Updating the lmutil to:

lmutil lmstat -v
lmutil - Copyright (c) 1989-2021 Flexera. All Rights Reserved.
lmstat v11.18.1.0 build 279526 x64_lsb

Gives the same issue, no meaningful logs either:

ts=2023-10-04T18:17:47.691Z caller=flexlm_exporter.go:199 level=info msg="Starting flexlm_exporter" version="(version=, branch=non-git, revision=non-git)"
ts=2023-10-04T18:17:47.691Z caller=flexlm_exporter.go:200 level=info msg="Build context" build_context="(go=go1.20.7, platform=linux/amd64, user=app@918d31421ff1, date=20231002-16:54:34, tags=netgo static_build)"
ts=2023-10-04T18:17:47.691Z caller=flexlm_exporter.go:123 level=info msg="Enabled collectors"
ts=2023-10-04T18:17:47.692Z caller=config.go:48 level=info msg="Loading license config file:"
ts=2023-10-04T18:17:47.692Z caller=config.go:49 level=info -=/home/app/config/licenses.yml
ts=2023-10-04T18:17:47.692Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9319
ts=2023-10-04T18:17:47.692Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9319

It's almost like the the exporter isn't even trying to execute the lmutil binary. Running the lmutil lmstat commands run fine by itself

mjtrangoni commented 1 year ago

@cloudrck about the last case, the container will try to execute lmutil only when you scrape the port 9319, and not before.

Dont you see any error in the logs after some scrapes?

cloudrck commented 1 year ago

@mjtrangoni Accessing the /metrics endpoint on that port just shows the generic GO GC metrics. No error messages from the exporter

TheRaspyDev commented 1 year ago

I do have the same problem. Any update on this?

TheRaspyDev commented 1 year ago

I found the problem but as I'm not a go developer, I can't solve it.

The collectors are not "loaded". The flag variable in the registerCollector function is false --> line 78 of collector.go.

Then in line 124 of collector.go the if !*enabled results to true and therefore the collector isn't loaded.

Commenting lines 124-126 in collector.go solves the problem. When these lines are commented, the collectors are loaded and everything works as expected.

So the line 78 in collector.go has to be analyzed why it returns false.

Maybe the bug comes from an update of kingpin?

I've also noticed that the flags collector.lmstat and collectors.lmstat_feature_exp are not available as kingpin flags. The line 78 in collector.go defines flags for each collector to be enabled/disabled from command line.

Calling ./flexlm_exporter --collector.lmstat='false' gives an error. In my understanding there should be flags to enable/disable the collectors from command line.

Hope this helps for fixing this bug.

Cheers, Maurus

mjtrangoni commented 1 year ago

@TheRaspyDev @cloudrck Please confirm if the issue is fixes for everyone, and I will create a new release/ new images as well.

mjtrangoni commented 1 year ago

@cloudrck With this patch, I cant reproduce the segmentation fault you are pointing to. How are you building and testing it?

cloudrck commented 1 year ago

For building: I'm using the docker image golang:1.20.7-bookworm, and having the entrypoint run make to build the flexlm binary.

It's being ran in k8s.

I'll try and build it different ways and see what happens

TheRaspyDev commented 1 year ago

@TheRaspyDev @cloudrck Please confirm if the issue is fixes for everyone, and I will create a new release/ new images as well.

@mjtrangoni: Works with the new docker image. Thanks a lot!

cloudrck commented 1 year ago

@mjtrangoni I found the nil pointer reference error only happens with one specific license server. When I remove it, the exporter works fine. And when it is the only server in the config it fails.

The error:

github.com/mjtrangoni/flexlm_exporter/collector.parseLmstatLicenseInfoServer({0xc000966000, 0x25a, 0x8000?})
        /home/myuser/app/collector/lmstat.go:233 +0x232
github.com/mjtrangoni/flexlm_exporter/collector.(*lmstatCollector).collect(0xc000206730, 0xc00060bf78, 0xc0001cecc0?)
        /home/myuser/app/collector/lmstat.go:426 +0x25c
github.com/mjtrangoni/flexlm_exporter/collector.(*lmstatCollector).getLmstatLicensesInfo.func1({{0xc000226834, 0x5}, {0x0, 0x0}, {0xc000238258, 0x16}, {0x0, 0x0}, {0x0, 0x0}, ...})
        /home/myuser/app/collector/lmstat.go:388 +0x85
created by github.com/mjtrangoni/flexlm_exporter/collector.(*lmstatCollector).getLmstatLicensesInfo
        /home/myuser/app/collector/lmstat.go:385 +0xa9

On that line it appears to be referencing this regex

This particular license sever is the only one that returns multiple license server UP

~$ ./lmutil lmstat -c 1055@problem-server -a
lmutil - Copyright (c) 1989-2018 Flexera. All Rights Reserved.
Flexible License Manager status on Thu 11/9/2023 21:07

License server status: 1055@problem-server
    License file(s) on problem-server: C:\Program Files\ANSYS Inc\Shared Files\Licensing\license_files\ansyslmd.lic:C:\Program Files\ANSYS Inc\Shared Files\Licensing\license_files\ansyslmd_1.lic:C:\Program Files\ANSYS Inc\Shared Files\Licensing\license_files\ansyslmd_2.lic:C:\Program Files\ANSYS Inc\Shared Files\Licensing\license_files\ansyslmd_3.lic:C:\Program Files\ANSYS Inc\Shared Files\Licensing\license_files\ansyslmd_4.lic:C:\Program Files\ANSYS Inc\Shared Files\Licensing\license_files\ansyslmd_5expire9_13.lic:

  problem-server: license server UP (MASTER) v11.19.0
  problem-server: license server UP v11.19.0

Vendor daemon status (on problem-server):

  ansyslmd: UP v11.19.0

Partial License server info:

~$ ./lmutil lmstat -c 1055@problem-server -i
lmutil - Copyright (c) 1989-2018 Flexera. All Rights Reserved.
Thu 11/9/2023 21:06

NOTE: lmstat -i does not give information from the server,
      but only reads the license file.  For this reason,
      lmstat -a is recommended instead.

Feature                         Version     #licenses    Vendor        Expires
_______                         _________   _________    ______        ________
a_spaceclaim_dirmod             9999.9999    2           ansyslmd      31-may-2024
a_spaceclaim_dirmod             9999.9999    3           ansyslmd      31-may-2024
a_spaceclaim_dirmod             9999.9999    3           ansyslmd      31-may-2024
mjtrangoni commented 1 year ago

@cloudrck nice, this is pointing in the right direction! But, I fear the problem is related to the hostname itself and not to the rest of the file, or some other special character over there.

Would you mind sending me lmutil lmstat -c 1055@problem-server -a again? Anonymized but in a more transparent way, so that you replace the same type of symbol with something else?

Another option would be sending the original output to me privately at mjtrangoni@gmail.com.

Or if you want to give it a try on your side, there is a windows test case, collector/fixtures/lmstat_server_up_win.txt. Replacing the file with your output should point in the right direction running make test.

Thank you!

mjtrangoni commented 1 year ago

Fixed with v0.0.10