Juniper / open-nti

Open Network Telemetry Collector build with open source tools
Apache License 2.0
232 stars 93 forks source link

JTI cpu data not appearing in the InfluxDB #227

Open henriklb opened 6 years ago

henriklb commented 6 years ago

I'm trying to stream the /junos/system/linecard/cpu/memory/ sensor with JTI. The problem is that I do not see any cpu data from the sensor in the influx database.

I'm running the latest open-nti with the new fluent JTI parser/plugin from @psagrera. Running junos 17.3R2 tested with both MX80 and MX104.

Using tcpdump and protoc I see the packets are being sent correctly onto port 50000.

/junos/system/linecard/interface/ works just fine.

Any idea what could be wrong?

This is my Junos config

streaming-server OPEN-NTI {
    remote-address 99.99.99.99;
    remote-port 50000;
}
export-profile PROFILE1 {
    local-address 11.11.11.11;
    local-port 21111;
    reporting-rate 1;
    format gpb;
    transport udp;
}
sensor SENSOR1 {
    server-name OPEN-NTI;
    export-name PROFILE1;
    resource /junos/system/linecard/interface/;
}
sensor SENSOR2 {
    server-name OPEN-NTI;
    export-name PROFILE1;
    resource /junos/system/linecard/cpu/memory/;
}
henriklb commented 6 years ago

I might see some initial data in the "juniperNetworks.cpu_memory_util_ext" table, but it stop collecting/parsing data instantly right after one initial "push"/entry in the database. Every entry has identical timestamp, then it stop receiving cpu data altogether. cpu.db.log.txt

psagrera commented 6 years ago

Could you please execute the following command inside the influxdb container ?: select * from "juniperNetworks.cpu_memory_util_ext" group by " utilization.application_utilization.name" order by desc limit 5

Regards.

2018-07-09 13:45 GMT+02:00 henriklb notifications@github.com:

I might see some initial data in the "juniperNetworks.cpu_memory_util_ext" table, but it stop receiving data instantly right after one initial "push"/entry in the database. Every entry has identical timestamp, then it stop receiving cpu data altogether. cpu.db.log.txt https://github.com/Juniper/open-nti/files/2175961/cpu.db.log.txt

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-403451608, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZRGJgng6ESYiuHMFCLlO3dah78A7ks5uE0JhgaJpZM4VHgjE .

henriklb commented 6 years ago

Here you go, and thanks! pasto.txt

psagrera commented 6 years ago

Could you please set "precision rfc3339" before execute that command :) ?

precision rfc3339 select * from "juniperNetworks.cpu_memory_util_ext" group by " utilization.application_utilization.name" order by desc limit 5

2018-07-09 16:02 GMT+02:00 henriklb notifications@github.com:

Here you go, and thanks! pasto.txt https://github.com/Juniper/open-nti/files/2176465/pasto.txt

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-403489981, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZasnu8moK4sfr_IB30s5Kg9w26-Zks5uE2J7gaJpZM4VHgjE .

henriklb commented 6 years ago

My bad. pasto.txt

henriklb commented 6 years ago

Did that give you any clues?

henriklb commented 6 years ago

If I restart the juniper/open-nti-input-jti container the cpu/mem datapoints gets inserted once and then stops. I attached my fluentd logs.

fluentdlogs.txt

psagrera commented 6 years ago

Hi

Could you please type : docker images ?

and go inside the jti-container and execute : ls -lrt under /fluent-plugin-udp-native-sensors dierctory ?

Anoher thing you can try is restart all containers (make stop / make start)

Regards

2018-07-11 10:38 GMT+02:00 henriklb notifications@github.com:

If I restart the juniper/open-nti-input-jti container the cpu/mem datapoints gets inserted once and then stops. I attached my fluentd logs.

fluentdlogs.txt https://github.com/Juniper/open-nti/files/2183670/fluentdlogs.txt

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404090599, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZZo-AcqSJsOoJVlbJQ-_IdpiXevhks5uFblpgaJpZM4VHgjE .

henriklb commented 6 years ago

Let me know if you need anything else. Docker images:

REPOSITORY                            TAG                 IMAGE ID            CREATED             SIZE
kapacitor                             1.5.0               fb3f94851c72        4 days ago          254MB
juniper/open-nti                      latest              c0b2d2db3a32        4 days ago          1.09GB
juniper/open-nti-input-jti            latest              33fc98958829        4 days ago          2.53GB
juniper/open-nti-input-syslog         latest              b80ffd1a2372        4 days ago          110MB
open-nti_input-oc                     latest              5feb22b05c88        5 days ago          211MB
juniper/open-nti                      <none>              4a61d113c345        7 days ago          1.09GB
juniper/open-nti-input-jti            <none>              9c95511ac762        7 days ago          2.53GB
juniper/open-nti-input-syslog         <none>              090304f89a81        7 days ago          110MB
telegraf                              1.7                 5347c50a412b        2 weeks ago         211MB
quay.io/influxdb/chronograf           1.5.0.1             fda001d20828        5 weeks ago         43.1MB
root@84367655c790:/fluent-plugin-udp-native-sensors# ls -lrt
total 72
drwxr-xr-x 2 root root  4096 Jun 18 10:18 test
-rw-r--r-- 1 root root    37 Jun 18 10:18 requirements.txt
-rw-r--r-- 1 root root  9551 Jun 18 10:18 juniper_telemetry_udp_lib.rb
-rw-r--r-- 1 root root  1094 Jun 18 10:18 fluent-plugin-udp-native-sensors.gemspec
-rw-r--r-- 1 root root   225 Jun 18 10:18 Rakefile
-rw-r--r-- 1 root root  2801 Jun 18 10:18 README.md
-rw-r--r-- 1 root root 10142 Jun 18 10:18 LICENSE
-rw-r--r-- 1 root root   114 Jun 18 10:18 Gemfile
drwxr-xr-x 1 root root  4096 Jul  3 13:57 lib
-rw-r--r-- 1 root root 18944 Jul  3 15:19 fluent-plugin-udp-native-sensors-0.0.1.gem
henriklb commented 6 years ago

I've tried open-nti make start and make stop, but same result.

psagrera commented 6 years ago

Could you please remove all images with TAG ? Also, I miss this image : psagrera/fluent-jti:1.1.

That's my setup

root@opennti1:~# docker images REPOSITORY TAG IMAGE ID CREATED SIZE juniper/open-nti latest 1e5b12569cc9 4 days ago 1.09 GB opennti_input-oc latest 6cdb7d2da4b8 6 days ago 211 MB juniper/open-nti-input-internal latest 9a92af7259a3 6 days ago 131 MB juniper/open-nti-input-oc latest 487e864bc99a 6 days ago 211 MB juniper/open-nti-input-syslog latest 29b3569d3d2a 6 days ago 110 MB juniper/open-nti-input-jti latest 31c6905ee6ee 6 days ago 2.53 GB psagrera/fluent-jti 1.1 1c75c66a971a 7 days ago 2.53 GB telegraf 1.7 5347c50a412b 2 weeks ago 211 MB kapacitor 1.5.0 0663d599c590 2 weeks ago 254 MB quay.io/influxdb/chronograf 1.5.0.1 fda001d20828 5 weeks ago 43.1 MB

So, I would remove images with TAG and would re-build everything (make build). If possible, send me this file : cpu_memory_utilization.pb.rb (inside jti-container, under /fluent-plugin-udp-native-sensors/lib directory)

Regards

2018-07-11 12:38 GMT+02:00 henriklb notifications@github.com:

Let me know if you need anything else. Docker images:

REPOSITORY TAG IMAGE ID CREATED SIZE kapacitor 1.5.0 fb3f94851c72 4 days ago 254MB juniper/open-nti latest c0b2d2db3a32 4 days ago 1.09GB juniper/open-nti-input-jti latest 33fc98958829 4 days ago 2.53GB juniper/open-nti-input-syslog latest b80ffd1a2372 4 days ago 110MB open-nti_input-oc latest 5feb22b05c88 5 days ago 211MB juniper/open-nti 4a61d113c345 7 days ago 1.09GB juniper/open-nti-input-jti 9c95511ac762 7 days ago 2.53GB juniper/open-nti-input-syslog 090304f89a81 7 days ago 110MB telegraf 1.7 5347c50a412b 2 weeks ago 211MB

root@84367655c790:/fluent-plugin-udp-native-sensors# ls -lrt total 72 drwxr-xr-x 2 root root 4096 Jun 18 10:18 test -rw-r--r-- 1 root root 37 Jun 18 10:18 requirements.txt -rw-r--r-- 1 root root 9551 Jun 18 10:18 juniper_telemetry_udp_lib.rb -rw-r--r-- 1 root root 1094 Jun 18 10:18 fluent-plugin-udp-native-sensors.gemspec -rw-r--r-- 1 root root 225 Jun 18 10:18 Rakefile -rw-r--r-- 1 root root 2801 Jun 18 10:18 README.md -rw-r--r-- 1 root root 10142 Jun 18 10:18 LICENSE -rw-r--r-- 1 root root 114 Jun 18 10:18 Gemfile drwxr-xr-x 1 root root 4096 Jul 3 13:57 lib -rw-r--r-- 1 root root 18944 Jul 3 15:19 fluent-plugin-udp-native-sensors-0.0.1.gem

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404124437, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZYQ3u9P_AsRddTC8xkcX5elY0Vauks5uFdXBgaJpZM4VHgjE .

henriklb commented 6 years ago

Removed ALL images, cloned from repo again, 'make build' followed by 'make start' (also tried once with only make start).

Still exactly the same behavior cpu and memory data pushed once in the database then stops , (*/interface/ works fine once again)

cpu_memory_utilization.pb.rb from "juniper/open-nti-input-jti"

REPOSITORY                            TAG                 IMAGE ID            CREATED              SIZE
open-nti_input-oc                     latest              13fc7f107d61        About a minute ago   211MB
juniper/open-nti-input-internal       latest              decb27812bb6        3 minutes ago        131MB
juniper/open-nti-input-oc             latest              e35702494754        3 minutes ago        211MB
juniper/open-nti-input-syslog         latest              f38866a1be96        4 minutes ago        110MB
juniper/open-nti-input-jti            latest              e2c7483add93        5 minutes ago        2.53GB
juniper/open-nti                      latest              0fa5ab15d24b        6 minutes ago        1.09GB
telegraf                              1.7                 476f4269538d        4 days ago           211MB
kapacitor                             1.5.0               fb3f94851c72        4 days ago           254MB
psagrera/fluent-jti                   1.1                 1c75c66a971a        7 days ago           2.53GB
quay.io/influxdb/chronograf           1.5.0.1             fda001d20828        5 weeks ago          43.1MB
docker.elastic.co/logstash/logstash   6.2.4               00a38ba5444c        2 months ago         657MB
phusion/baseimage                     0.9.22              877509368a8d        13 months ago        225MB
juniper/pyez                          2.0.1               e61c159ee89f        22 months ago        93MB
fluent/fluentd                        v0.12.29            d0e2862496ea        22 months ago        36MB

screenshot from 2018-07-09 14-12-43

psagrera commented 6 years ago

Are you still getting those errors in fluentd ? it would be possible to test with another JUNOS release ?

2018-07-11 15:05 GMT+02:00 henriklb notifications@github.com:

Removed ALL images, cloned from repo again, 'make build' followed by 'make start' (also tried once with only make start).

Still exactly the same behavior cpu and memory data pushed once in the database then stops , (*/interface/ works fine once again)

cpu_memory_utilization.pb.rb.txt https://github.com/Juniper/open-nti/files/2184530/cpu_memory_utilization.pb.rb.txt

REPOSITORY TAG IMAGE ID CREATED SIZE open-nti_input-oc latest 13fc7f107d61 About a minute ago 211MB juniper/open-nti-input-internal latest decb27812bb6 3 minutes ago 131MB juniper/open-nti-input-oc latest e35702494754 3 minutes ago 211MB juniper/open-nti-input-syslog latest f38866a1be96 4 minutes ago 110MB juniper/open-nti-input-jti latest e2c7483add93 5 minutes ago 2.53GB juniper/open-nti latest 0fa5ab15d24b 6 minutes ago 1.09GB telegraf 1.7 476f4269538d 4 days ago 211MB kapacitor 1.5.0 fb3f94851c72 4 days ago 254MB psagrera/fluent-jti 1.1 1c75c66a971a 7 days ago 2.53GBquay.io/influxdb/chronograf 1.5.0.1 fda001d20828 5 weeks ago 43.1MBdocker.elastic.co/logstash/logstash 6.2.4 00a38ba5444c 2 months ago 657MB phusion/baseimage 0.9.22 877509368a8d 13 months ago 225MB juniper/pyez 2.0.1 e61c159ee89f 22 months ago 93MB fluent/fluentd v0.12.29 d0e2862496ea 22 months ago 36MB

[image: screenshot from 2018-07-09 14-12-43] https://user-images.githubusercontent.com/11011393/42573219-c5f98bf2-851b-11e8-9929-49cf569b5148.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404161800, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZTZX7U3SUp-l4fVHJe_Y-i-EnLJDks5uFfgqgaJpZM4VHgjE .

psagrera commented 6 years ago

Could you log in into the Linecard and execute :

MX-0(HL3-1-2 vty)# show agent sensors id "xxxxxx" , being "xxxxxx" cpu-sensor ID

2018-07-11 15:42 GMT+02:00 Pablo Sagrera Garcia psagrerag@gmail.com:

Are you still getting those errors in fluentd ? it would be possible to test with another JUNOS release ?

2018-07-11 15:05 GMT+02:00 henriklb notifications@github.com:

Removed ALL images, cloned from repo again, 'make build' followed by 'make start' (also tried once with only make start).

Still exactly the same behavior cpu and memory data pushed once in the database then stops , (*/interface/ works fine once again)

cpu_memory_utilization.pb.rb.txt https://github.com/Juniper/open-nti/files/2184530/cpu_memory_utilization.pb.rb.txt

REPOSITORY TAG IMAGE ID CREATED SIZE open-nti_input-oc latest 13fc7f107d61 About a minute ago 211MB juniper/open-nti-input-internal latest decb27812bb6 3 minutes ago 131MB juniper/open-nti-input-oc latest e35702494754 3 minutes ago 211MB juniper/open-nti-input-syslog latest f38866a1be96 4 minutes ago 110MB juniper/open-nti-input-jti latest e2c7483add93 5 minutes ago 2.53GB juniper/open-nti latest 0fa5ab15d24b 6 minutes ago 1.09GB telegraf 1.7 476f4269538d 4 days ago 211MB kapacitor 1.5.0 fb3f94851c72 4 days ago 254MB psagrera/fluent-jti 1.1 1c75c66a971a 7 days ago 2.53GBquay.io/influxdb/chronograf 1.5.0.1 fda001d20828 5 weeks ago 43.1MBdocker.elastic.co/logstash/logstash 6.2.4 00a38ba5444c 2 months ago 657MB phusion/baseimage 0.9.22 877509368a8d 13 months ago 225MB juniper/pyez 2.0.1 e61c159ee89f 22 months ago 93MB fluent/fluentd v0.12.29 d0e2862496ea 22 months ago 36MB

[image: screenshot from 2018-07-09 14-12-43] https://user-images.githubusercontent.com/11011393/42573219-c5f98bf2-851b-11e8-9929-49cf569b5148.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404161800, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZTZX7U3SUp-l4fVHJe_Y-i-EnLJDks5uFfgqgaJpZM4VHgjE .

henriklb commented 6 years ago

Same errors in fluentd, I can try a different junos version, do you have a particular one in mind? I do not have the root password at hand so that would not be possible right now (maybe tomorrow).

psagrera commented 6 years ago

My last test was done with 18.1R1.7 (but never tested neither MX80 nor MX104 , so whatever different release to 17.3R2 (preferably 18.X ) is fine in order to see if the problem is seen in more than one release). Bear in mind that you have to update network-agent package accordingly

Regards

2018-07-11 16:54 GMT+02:00 henriklb notifications@github.com:

Same errors in fluentd, I can try a different junos version, do you have a particular one in mind? I do not have the root password at hand so that would not be possible right now (maybe tomorrow).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404199215, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZalWYZK7bVVAkM4bh-zFKsQugwmQks5uFhGWgaJpZM4VHgjE .

henriklb commented 6 years ago

Tested with 18.1 on an MX80, same parse error in fluentd.

anaaman commented 6 years ago

I have the same problem with QFX5110 running Junos 18.1R1. No CPU/MEM data and interface statistics in the database

psagrera commented 6 years ago

Hi,

I'll try to get an MX80 to test it, in the meantime could you test other sensors ?

Regards

2018-07-12 17:50 GMT+02:00 henriklb notifications@github.com:

Tested with 18.1 on an MX80, same parse error in fluentd.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404559816, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZZlS9PGsSOS3YFqNkTN0WW17uYrHks5uF3BhgaJpZM4VHgjE .

anaaman commented 6 years ago

Tested with vMX, after updating to latest version stoped working.

On Fri, 13 Jul 2018 at 1:27 PM psagrera notifications@github.com wrote:

Hi,

I'll try to get an MX80 to test it, in the meantime could you test other sensors ?

Regards

2018-07-12 17:50 GMT+02:00 henriklb notifications@github.com:

Tested with 18.1 on an MX80, same parse error in fluentd.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404559816, or mute the thread < https://github.com/notifications/unsubscribe-auth/AKgjZZlS9PGsSOS3YFqNkTN0WW17uYrHks5uF3BhgaJpZM4VHgjE

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404793575, or mute the thread https://github.com/notifications/unsubscribe-auth/ATy4v0b0Pz9L84u5qXrTmnbF0bSbzdRaks5uGHYBgaJpZM4VHgjE .

psagrera commented 6 years ago

you mean only one insertion into the database ? same fluentd errors ? Only happen with cpu sensor ?

Regards

2018-07-13 14:14 GMT+02:00 anaaman notifications@github.com:

Tested with vMX, after updating to latest version stoped working.

On Fri, 13 Jul 2018 at 1:27 PM psagrera notifications@github.com wrote:

Hi,

I'll try to get an MX80 to test it, in the meantime could you test other sensors ?

Regards

2018-07-12 17:50 GMT+02:00 henriklb notifications@github.com:

Tested with 18.1 on an MX80, same parse error in fluentd.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/Juniper/open-nti/issues/227#issuecomment-404559816 , or mute the thread < https://github.com/notifications/unsubscribe-auth/ AKgjZZlS9PGsSOS3YFqNkTN0WW17uYrHks5uF3BhgaJpZM4VHgjE

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404793575, or mute the thread https://github.com/notifications/unsubscribe-auth/ ATy4v0b0Pz9L84u5qXrTmnbF0bSbzdRaks5uGHYBgaJpZM4VHgjE .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404815748, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZeeE80lUjFQH-eymUc4owN_FSbGGks5uGI80gaJpZM4VHgjE .

anaaman commented 6 years ago

you mean only one insertion into the database ? - I have no data at all, I see only the host IP in the device filtering. Only happen with cpu sensor ? No, This happen also with interface statistics. same fluentd errors ? didn't see any errors, This is the output of " docker logs opennti_input_jti" :

Running /etc/my_init.d/00_regen_ssh_host_keys.sh... Running /etc/rc.local... Booting runit daemon... Runit started as PID 7 [root@telemtry open-nti]# docker logs chronograf_con time="2018-07-16T07:30:23Z" level=info msg="Running migration 59b0cda4fc7909ff84ee5c4f9cb4b655b6a26620" time="2018-07-16T07:30:23Z" level=info msg="Serving chronograf at http://[::]:8888" component=server time="2018-07-16T07:30:23Z" level=info msg="Reporting usage stats" component=usage freq=24h reporting_addr="https://usage.influxdata.com" stats="os,arch,version,cluster_id,uptime" [root@telemtry open-nti]# docker logs opennti_input_jti 2018-07-16 10:30:24 +0300 [info]: reading config file path="/tmp/fluent.conf" 2018-07-16 10:30:24 +0300 [info]: starting fluentd-0.12.43 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.6.0' 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-udp-native-sensors' version '0.0.1' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.43' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.42' 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks" type="rewrite_tag_filter" 2018-07-16 10:30:25 +0300 [warn]: rewrite_tag_filter: [DEPRECATED] Use section instead of rewriterule1 2018-07-16 10:30:25 +0300 [info]: adding rewrite_tag_filter rule: rewriterule1 ["sensor_name", /(.+)/, "", "${tag}.$1"] 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.cpu_memory_util_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_packet_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_lsp_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks." type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="debug." type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="fluent.**" type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding source type="forward" 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [warn]: 'body_size_limit' parameter is deprecated: use message_length_limit instead. 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [info]: adding source type="monitor_agent" 2018-07-16 10:30:25 +0300 [info]: adding source type="debug_agent" 2018-07-16 10:30:25 +0300 [info]: using configuration file:

@type forward
@id forward_input

@type udp
tag juniperNetworks
format juniper_udp_native
port 50000
bind 0.0.0.0
body_size_limit 5000

@type udp
tag jnpr.analyticsd
format juniper_analyticsd
message_length_limit 5000
remove_newline false
port 50020
bind 0.0.0.0

@type rewrite_tag_filter rewriterule1 sensor_name (.+) ${tag}.$1 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","utilization.application_utilization.name","utilization.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","packet_stats.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","lsp_stats_records.name"] tag_keys_field key_fields buffer_type memory flush_interval 2

<match juniperNetworks.**> type copy

type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device"] tag_keys_field key_fields buffer_type memory flush_interval 2

@type monitor_agent
@id monitor_agent_input
port 24220

@type debug_agent
@id debug_agent_input
bind 127.0.0.1
port 24230

<match debug.> @type stdout @id stdout_output <match fluent.> @type stdout 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: listening fluent socket on 0.0.0.0:24224 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50000 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50020 2018-07-16 10:30:25 +0300 [info]: listening dRuby uri="druby://127.0.0.1:24230" object="Engine"

psagrera commented 6 years ago

@anaaman , are you receiving data from the vMX ? Could you please execute a tcpdump in the server ? is there any measurement in the juniper database ?

@henriklb, I'm testing with vMX running 18.1 and it's working fine in my case. I'm still trying to get a MX80 or MX104 for testing in that models.

2018-07-16 9:55 GMT+02:00 anaaman notifications@github.com:

you mean only one insertion into the database ? - I have no data at all, I see only the host IP in the device filtering. Only happen with cpu sensor ? No, This happen also with interface statistics. same fluentd errors ? didn't see any errors, This is the output of " docker logs opennti_input_jti" :

Running /etc/my_init.d/00_regen_ssh_host_keys.sh... Running /etc/rc.local... Booting runit daemon... Runit started as PID 7 [root@telemtry open-nti]# docker logs chronograf_con time="2018-07-16T07:30:23Z" level=info msg="Running migration 59b0cda4fc7909ff84ee5c4f9cb4b655b6a26620" time="2018-07-16T07:30:23Z" level=info msg="Serving chronograf at http://[::]:8888" component=server time="2018-07-16T07:30:23Z" level=info msg="Reporting usage stats" component=usage freq=24h reporting_addr="https://usage.influxdata.com" stats="os,arch,version,cluster_id,uptime" [root@telemtry open-nti]# docker logs opennti_input_jti 2018-07-16 10:30:24 +0300 [info]: reading config file path="/tmp/fluent.conf" 2018-07-16 10:30:24 +0300 [info]: starting fluentd-0.12.43 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.6.0' 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-udp-native-sensors' version '0.0.1' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.43' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.42' 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks" type="rewrite_tag_filter" 2018-07-16 10:30:25 +0300 [warn]: rewrite_tag_filter: [DEPRECATED] Use section instead of rewriterule1 2018-07-16 10:30:25 +0300 [info]: adding rewrite_tag_filter rule: rewriterule1 ["sensor_name", /(.+)/, "", "${tag}.$1"] 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.cpu_memory_util_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_packet_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_lsp_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks. " type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="debug." type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="fluent.**" type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding source type="forward" 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [warn]: 'body_size_limit' parameter is deprecated: use message_length_limit instead. 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [info]: adding source type="monitor_agent" 2018-07-16 10:30:25 +0300 [info]: adding source type="debug_agent" 2018-07-16 10:30:25 +0300 [info]: using configuration file: @type https://github.com/type forward @id https://github.com/id forward_input @type https://github.com/type udp tag juniperNetworks format juniper_udp_native port 50000 bind 0.0.0.0 body_size_limit 5000 @type https://github.com/type udp tag jnpr.analyticsd format juniper_analyticsd message_length_limit 5000 remove_newline false port 50020 bind 0.0.0.0 @type https://github.com/type rewrite_tag_filter rewriterule1 sensor_name (.+) ${tag}.$1 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","utilization. application_utilization.name","utilization.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","packet_stats.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","lsp_stats_records.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device"] tag_keys_field key_fields buffer_type memory flush_interval 2 @type https://github.com/type monitor_agent @id https://github.com/id monitor_agent_input port 24220 @type https://github.com/type debug_agent @id https://github.com/id debug_agent_input bind 127.0.0.1 port 24230 @type https://github.com/type stdout @id https://github.com/id stdout_output @type https://github.com/type stdout 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: listening fluent socket on 0.0.0.0:24224 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50000 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50020 2018-07-16 10:30:25 +0300 [info]: listening dRuby uri="druby://127.0.0.1:24230" object="Engine"

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-405172215, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZb9JjS3eZPfVM-kNbEzrOorGiS-Yks5uHEcAgaJpZM4VHgjE .

anaaman commented 6 years ago

Hi, I have older version of Open-NTI and when I set it as the collector everything work fine. I ran tcpdump on my docker host and I can see the data coming in. In the “juniper” database I have those measurements:

juniperNetworks.cpu_memory_util_ext juniperNetworks.jnprLogicalInterfaceExt

It looks like the measurements are empty.

Regrads

From: psagrera Sent: Monday, July 16, 2018 4:59 PM To: Juniper/open-nti Cc: anaaman; Mention Subject: Re: [Juniper/open-nti] JTI cpu data not appearing in the InfluxDB(#227)

@anaaman , are you receiving data from the vMX ? Could you please execute a tcpdump in the server ? is there any measurement in the juniper database ?

@henriklb, I'm testing with vMX running 18.1 and it's working fine in my case. I'm still trying to get a MX80 or MX104 for testing in that models.

2018-07-16 9:55 GMT+02:00 anaaman notifications@github.com:

you mean only one insertion into the database ? - I have no data at all, I see only the host IP in the device filtering. Only happen with cpu sensor ? No, This happen also with interface statistics. same fluentd errors ? didn't see any errors, This is the output of " docker logs opennti_input_jti" :

Running /etc/my_init.d/00_regen_ssh_host_keys.sh... Running /etc/rc.local... Booting runit daemon... Runit started as PID 7 [root@telemtry open-nti]# docker logs chronograf_con time="2018-07-16T07:30:23Z" level=info msg="Running migration 59b0cda4fc7909ff84ee5c4f9cb4b655b6a26620" time="2018-07-16T07:30:23Z" level=info msg="Serving chronograf at http://[::]:8888" component=server time="2018-07-16T07:30:23Z" level=info msg="Reporting usage stats" component=usage freq=24h reporting_addr="https://usage.influxdata.com" stats="os,arch,version,cluster_id,uptime" [root@telemtry open-nti]# docker logs opennti_input_jti 2018-07-16 10:30:24 +0300 [info]: reading config file path="/tmp/fluent.conf" 2018-07-16 10:30:24 +0300 [info]: starting fluentd-0.12.43 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.6.0' 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-udp-native-sensors' version '0.0.1' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.43' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.42' 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks" type="rewrite_tag_filter" 2018-07-16 10:30:25 +0300 [warn]: rewrite_tag_filter: [DEPRECATED] Use section instead of rewriterule1 2018-07-16 10:30:25 +0300 [info]: adding rewrite_tag_filter rule: rewriterule1 ["sensor_name", /(.+)/, "", "${tag}.$1"] 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.cpu_memory_util_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_packet_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_lsp_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks. " type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="debug." type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="fluent.**" type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding source type="forward" 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [warn]: 'body_size_limit' parameter is deprecated: use message_length_limit instead. 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [info]: adding source type="monitor_agent" 2018-07-16 10:30:25 +0300 [info]: adding source type="debug_agent" 2018-07-16 10:30:25 +0300 [info]: using configuration file: @type https://github.com/type forward @id https://github.com/id forward_input @type https://github.com/type udp tag juniperNetworks format juniper_udp_native port 50000 bind 0.0.0.0 body_size_limit 5000 @type https://github.com/type udp tag jnpr.analyticsd format juniper_analyticsd message_length_limit 5000 remove_newline false port 50020 bind 0.0.0.0 @type https://github.com/type rewrite_tag_filter rewriterule1 sensor_name (.+) ${tag}.$1 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","utilization. application_utilization.name","utilization.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","packet_stats.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","lsp_stats_records.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device"] tag_keys_field key_fields buffer_type memory flush_interval 2 @type https://github.com/type monitor_agent @id https://github.com/id monitor_agent_input port 24220 @type https://github.com/type debug_agent @id https://github.com/id debug_agent_input bind 127.0.0.1 port 24230 @type https://github.com/type stdout @id https://github.com/id stdout_output @type https://github.com/type stdout 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: listening fluent socket on 0.0.0.0:24224 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50000 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50020 2018-07-16 10:30:25 +0300 [info]: listening dRuby uri="druby://127.0.0.1:24230" object="Engine"

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-405172215, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZb9JjS3eZPfVM-kNbEzrOorGiS-Yks5uHEcAgaJpZM4VHgjE .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

psagrera commented 6 years ago

so if you run "select * from "juniperNetworks.cpu_memory_util_ext" order by desc limit 1" it doesn't show anything, right ? is there any way to gain access to your setup ?

2018-07-16 16:23 GMT+02:00 anaaman notifications@github.com:

Hi, I have older version of Open-NTI and when I set it as the collector everything work fine. I ran tcpdump on my docker host and I can see the data coming in. In the “juniper” database I have those measurements:

juniperNetworks.cpu_memory_util_ext juniperNetworks.jnprLogicalInterfaceExt

It looks like the measurements are empty.

Regrads

From: psagrera Sent: Monday, July 16, 2018 4:59 PM To: Juniper/open-nti Cc: anaaman; Mention Subject: Re: [Juniper/open-nti] JTI cpu data not appearing in the InfluxDB(#227)

@anaaman , are you receiving data from the vMX ? Could you please execute a tcpdump in the server ? is there any measurement in the juniper database ?

@henriklb, I'm testing with vMX running 18.1 and it's working fine in my case. I'm still trying to get a MX80 or MX104 for testing in that models.

2018-07-16 9:55 GMT+02:00 anaaman notifications@github.com:

you mean only one insertion into the database ? - I have no data at all, I see only the host IP in the device filtering. Only happen with cpu sensor ? No, This happen also with interface statistics. same fluentd errors ? didn't see any errors, This is the output of " docker logs opennti_input_jti" :

Running /etc/my_init.d/00_regen_ssh_host_keys.sh... Running /etc/rc.local... Booting runit daemon... Runit started as PID 7 [root@telemtry open-nti]# docker logs chronograf_con time="2018-07-16T07:30:23Z" level=info msg="Running migration 59b0cda4fc7909ff84ee5c4f9cb4b655b6a26620" time="2018-07-16T07:30:23Z" level=info msg="Serving chronograf at http://

component=server time="2018-07-16T07:30:23Z" level=info msg="Reporting usage stats" component=usage freq=24h reporting_addr="https://usage.influxdata.com" stats="os,arch,version,cluster_id,uptime" [root@telemtry open-nti]# docker logs opennti_input_jti 2018-07-16 10:30:24 +0300 [info]: reading config file path="/tmp/fluent.conf" 2018-07-16 10:30:24 +0300 [info]: starting fluentd-0.12.43 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.6.0' 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-udp-native-sensors' version '0.0.1' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.43' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.42' 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks" type="rewrite_tag_filter" 2018-07-16 10:30:25 +0300 [warn]: rewrite_tag_filter: [DEPRECATED] Use section instead of rewriterule1 2018-07-16 10:30:25 +0300 [info]: adding rewrite_tag_filter rule: rewriterule1 ["sensor_name", /(.+)/, "", "${tag}.$1"] 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.cpu_memory_util_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_packet_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_lsp_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks. " type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="debug." type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="fluent.**" type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding source type="forward" 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [warn]: 'body_size_limit' parameter is deprecated: use message_length_limit instead. 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [info]: adding source type="monitor_agent" 2018-07-16 10:30:25 +0300 [info]: adding source type="debug_agent" 2018-07-16 10:30:25 +0300 [info]: using configuration file: @type https://github.com/type forward @id https://github.com/id forward_input @type https://github.com/type udp tag juniperNetworks format juniper_udp_native port 50000 bind 0.0.0.0 body_size_limit 5000 @type https://github.com/type udp tag jnpr.analyticsd format juniper_analyticsd message_length_limit 5000 remove_newline false port 50020 bind 0.0.0.0 @type https://github.com/type rewrite_tag_filter rewriterule1 sensor_name (.+) ${tag}.$1 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","utilization. application_utilization.name","utilization.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","packet_stats.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","lsp_stats_records.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device"] tag_keys_field key_fields buffer_type memory flush_interval 2 @type https://github.com/type monitor_agent @id https://github.com/id monitor_agent_input port 24220 @type https://github.com/type debug_agent @id https://github.com/id debug_agent_input bind 127.0.0.1 port 24230 @type https://github.com/type stdout @id https://github.com/id stdout_output @type https://github.com/type stdout 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: listening fluent socket on 0.0.0.0:24224 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50000 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50020 2018-07-16 10:30:25 +0300 [info]: listening dRuby uri="druby://127.0.0.1:24230" object="Engine"

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-405172215, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZb9JjS3eZPfVM- kNbEzrOorGiS-Yks5uHEcAgaJpZM4VHgjE .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-405264483, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZaYoPj29YdUUdWbCbqJ76VKqm6ICks5uHKHdgaJpZM4VHgjE .

anaaman commented 6 years ago

There is no data in the database. Now I can see this in open_nti_input_jti log:

2018-07-16 18:01:20 +0300 fluent.warn: {"next_retry":"2018-07-16 18:01:21 +0300","error_class":"InfluxDB::Error","error":"{\"error\":\"partial write: unable to parse 'juniperNetworks.cpu_memory_util_ext,utilization.name=Kernel,utilization.application_utilization.name=nh,device=vmx-17-4:192.168.1.252,_seq=9 utilization.size=2147479528i,utilization.bytes_allocated=164003368i,utilization.utilization=7i,utilization.application_utilization.bytes_allocated=18446744073704795472i,utilization.application_utilization.allocations=627853i,utilization.application_utilization.frees=942536i,utilization.application_utilization.allocations_failed=0i,host=\\"136a4ef27333\\",sensor_name=\\"cpu_memory_util_ext\\" 1531753278598': unable to parse integer 18446744073704795472: strconv.ParseInt: parsing \\"18446744073704795472\\": value out of range dropped=0\"}\n","plugin_id":"object:2acbf96c9928","message":"temporarily failed to flush the buffer. next_retry=2018-07-16 18:01:21 +0300 error_class=\"InfluxDB::Error\" error=\"{\\"error\\":\\"partial write: unable to parse 'juniperNetworks.cpu_memory_util_ext,utilization.name=Kernel,utilization.application_utilization.name=nh,device=vmx-17-4:192.168.1.252,_seq=9 utilization.size=2147479528i,utilization.bytes_allocated=164003368i,utilization.utilization=7i,utilization.application_utilization.bytes_allocated=18446744073704795472i,utilization.application_utilization.allocations=627853i,utilization.application_utilization.frees=942536i,utilization.application_utilization.allocations_failed=0i,host=\\\\"136a4ef27333\\\\",sensor_name=\\\\"cpu_memory_util_ext\\\\" 1531753278598': unable to parse integer 18446744073704795472: strconv.ParseInt: parsing \\\\"18446744073704795472\\\\": value out of range dropped=0\\"}\n\" plugin_id=\"object:2acbf96c9928\""} 2018-07-16 18:01:21 +0300 [warn]: temporarily failed to flush the buffer. next_retry=2018-07-16 18:01:23 +0300 error_class="InfluxDB::Error" error="{\"error\":\"partial write: unable to parse 'juniperNetworks.cpu_memory_util_ext,utilization.name=Kernel,utilization.application_utilization.name=nh,device=vmx-17-4:192.168.1.252,_seq=9 utilization.size=2147479528i,utilization.bytes_allocated=164003368i,utilization.utilization=7i,utilization.application_utilization.bytes_allocated=18446744073704795472i,utilization.application_utilization.allocations=627853i,utilization.application_utilization.frees=942536i,utilization.application_utilization.allocations_failed=0i,host=\\"136a4ef27333\\",sensor_name=\\"cpu_memory_util_ext\\" 1531753278598': unable to parse integer 18446744073704795472: strconv.ParseInt: parsing \\"18446744073704795472\\": value out of range dropped=0\"}\n" plugin_id="object:2acbf96c9928" 2018-07-16 18:01:21 +0300 [warn]: suppressed same stacktrace

Sent from Mail for Windows 10

From: psagrera Sent: Monday, July 16, 2018 5:43 PM To: Juniper/open-nti Cc: anaaman; Mention Subject: Re: [Juniper/open-nti] JTI cpu data not appearing in the InfluxDB(#227)

so if you run "select * from "juniperNetworks.cpu_memory_util_ext" order by desc limit 1" it doesn't show anything, right ? is there any way to gain access to your setup ?

2018-07-16 16:23 GMT+02:00 anaaman notifications@github.com:

Hi, I have older version of Open-NTI and when I set it as the collector everything work fine. I ran tcpdump on my docker host and I can see the data coming in. In the “juniper” database I have those measurements:

juniperNetworks.cpu_memory_util_ext juniperNetworks.jnprLogicalInterfaceExt

It looks like the measurements are empty.

Regrads

From: psagrera Sent: Monday, July 16, 2018 4:59 PM To: Juniper/open-nti Cc: anaaman; Mention Subject: Re: [Juniper/open-nti] JTI cpu data not appearing in the InfluxDB(#227)

@anaaman , are you receiving data from the vMX ? Could you please execute a tcpdump in the server ? is there any measurement in the juniper database ?

@henriklb, I'm testing with vMX running 18.1 and it's working fine in my case. I'm still trying to get a MX80 or MX104 for testing in that models.

2018-07-16 9:55 GMT+02:00 anaaman notifications@github.com:

you mean only one insertion into the database ? - I have no data at all, I see only the host IP in the device filtering. Only happen with cpu sensor ? No, This happen also with interface statistics. same fluentd errors ? didn't see any errors, This is the output of " docker logs opennti_input_jti" :

Running /etc/my_init.d/00_regen_ssh_host_keys.sh... Running /etc/rc.local... Booting runit daemon... Runit started as PID 7 [root@telemtry open-nti]# docker logs chronograf_con time="2018-07-16T07:30:23Z" level=info msg="Running migration 59b0cda4fc7909ff84ee5c4f9cb4b655b6a26620" time="2018-07-16T07:30:23Z" level=info msg="Serving chronograf at http://

component=server time="2018-07-16T07:30:23Z" level=info msg="Reporting usage stats" component=usage freq=24h reporting_addr="https://usage.influxdata.com" stats="os,arch,version,cluster_id,uptime" [root@telemtry open-nti]# docker logs opennti_input_jti 2018-07-16 10:30:24 +0300 [info]: reading config file path="/tmp/fluent.conf" 2018-07-16 10:30:24 +0300 [info]: starting fluentd-0.12.43 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.6.0' 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-udp-native-sensors' version '0.0.1' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.43' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.42' 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks" type="rewrite_tag_filter" 2018-07-16 10:30:25 +0300 [warn]: rewrite_tag_filter: [DEPRECATED] Use section instead of rewriterule1 2018-07-16 10:30:25 +0300 [info]: adding rewrite_tag_filter rule: rewriterule1 ["sensor_name", /(.+)/, "", "${tag}.$1"] 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.cpu_memory_util_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_packet_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_lsp_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks. " type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="debug." type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="fluent.**" type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding source type="forward" 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [warn]: 'body_size_limit' parameter is deprecated: use message_length_limit instead. 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [info]: adding source type="monitor_agent" 2018-07-16 10:30:25 +0300 [info]: adding source type="debug_agent" 2018-07-16 10:30:25 +0300 [info]: using configuration file: @type https://github.com/type forward @id https://github.com/id forward_input @type https://github.com/type udp tag juniperNetworks format juniper_udp_native port 50000 bind 0.0.0.0 body_size_limit 5000 @type https://github.com/type udp tag jnpr.analyticsd format juniper_analyticsd message_length_limit 5000 remove_newline false port 50020 bind 0.0.0.0 @type https://github.com/type rewrite_tag_filter rewriterule1 sensor_name (.+) ${tag}.$1 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","utilization. application_utilization.name","utilization.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","packet_stats.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","lsp_stats_records.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device"] tag_keys_field key_fields buffer_type memory flush_interval 2 @type https://github.com/type monitor_agent @id https://github.com/id monitor_agent_input port 24220 @type https://github.com/type debug_agent @id https://github.com/id debug_agent_input bind 127.0.0.1 port 24230 @type https://github.com/type stdout @id https://github.com/id stdout_output @type https://github.com/type stdout 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: listening fluent socket on 0.0.0.0:24224 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50000 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50020 2018-07-16 10:30:25 +0300 [info]: listening dRuby uri="druby://127.0.0.1:24230" object="Engine"

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-405172215, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZb9JjS3eZPfVM- kNbEzrOorGiS-Yks5uHEcAgaJpZM4VHgjE .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-405264483, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZaYoPj29YdUUdWbCbqJ76VKqm6ICks5uHKHdgaJpZM4VHgjE .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

henriklb commented 6 years ago

@henriklb, I'm testing with vMX running 18.1 and it's working fine in my case. I'm still trying to get a MX80 or MX104 for testing in that models.

Sound good. Thanks!

psagrera commented 6 years ago

@anaaman , could you try to remove all images related with open NTI, and then execute make build again ?

psagrera commented 6 years ago

@henriklb, did you do make start or make build , the time you removes all images and started from scratch ?

henriklb commented 6 years ago

I started from scratch twice. That means no images and no containers at each start.

At the 1st start I typed make build followed by make start. Then I deleted every image and container again. At the 2nd start I typed make start only

anaaman commented 6 years ago

Did as you wrote. No I have this in JTI logs:

2018-07-17 16:14:17 +0300 fluent.warn: {"next_retry":"2018-07-17 16:14:50 +0300","error_class":"InfluxDB::Error","error":"{\"error\":\"partial write: unable to parse 'juniperNetworks.cpu_memory_util_ext,utilization.name=Kernel,utilization.application_utilization.name=nh,device=vmx-17-4:192.168.1.252,_seq=9 utilization.size=2147479528i,utilization.bytes_allocated=163991448i,utilization.utilization=7i,utilization.application_utilization.bytes_allocated=18446744073704522012i,utilization.application_utilization.allocations=661817i,utilization.application_utilization.frees=993485i,utilization.application_utilization.allocations_failed=0i,host=\\"35bf8d8d8402\\",sensor_name=\\"cpu_memory_util_ext\\" 1531833225500': unable to parse integer 18446744073704522012: strconv.ParseInt: parsing \\"18446744073704522012\\": value out of range dropped=0\"}\n","plugin_id":"object:2ae8ab0094e8","message":"temporarily failed to flush the buffer. next_retry=2018-07-17 16:14:50 +0300 error_class=\"InfluxDB::Error\" error=\"{\\"error\\":\\"partial write: unable to parse 'juniperNetworks.cpu_memory_util_ext,utilization.name=Kernel,utilization.application_utilization.name=nh,device=vmx-17-4:192.168.1.252,_seq=9 utilization.size=2147479528i,utilization.bytes_allocated=163991448i,utilization.utilization=7i,utilization.application_utilization.bytes_allocated=18446744073704522012i,utilization.application_utilization.allocations=661817i,utilization.application_utilization.frees=993485i,utilization.application_utilization.allocations_failed=0i,host=\\\\"35bf8d8d8402\\\\",sensor_name=\\\\"cpu_memory_util_ext\\\\" 1531833225500': unable to parse integer 18446744073704522012: strconv.ParseInt: parsing \\\\"18446744073704522012\\\\": value out of range dropped=0\\"}\n\" plugin_id=\"object:2ae8ab0094e8\""}

From: henriklb Sent: Tuesday, July 17, 2018 3:33 PM To: Juniper/open-nti Cc: anaaman; Mention Subject: Re: [Juniper/open-nti] JTI cpu data not appearing in the InfluxDB(#227)

I started from scratch twice. That means no images and no containers at each start. At the 1st start I typed make build followed by make start. Then I deleted every image and container. At the 2nd start I typed make start only — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

psagrera commented 6 years ago

@anaaman,@henriklb it would be possible to gain access to your setup ?

anaaman commented 6 years ago

Hi, Sorry but this is impossible.

From: psagrera Sent: Tuesday, July 17, 2018 4:58 PM To: Juniper/open-nti Cc: anaaman; Mention Subject: Re: [Juniper/open-nti] JTI cpu data not appearing in the InfluxDB(#227)

@anaaman,@henriklb it would be possible to gain access to your setup ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

henriklb commented 6 years ago

Not possible, sorry.

On Wed, 18 Jul 2018 at 10:11, anaaman notifications@github.com wrote:

Hi, Sorry but this is impossible.

From: psagrera Sent: Tuesday, July 17, 2018 4:58 PM To: Juniper/open-nti Cc: anaaman; Mention Subject: Re: [Juniper/open-nti] JTI cpu data not appearing in the InfluxDB(#227)

@anaaman,@henriklb it would be possible to gain access to your setup ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-405847827, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgFQXjJPsbY_-32x52npSenSsq5b0Qfks5uHu3AgaJpZM4VHgjE .

-- Mvh Henrik

psagrera commented 6 years ago

Hi @henriklb ,

it would be possible to decode data sent from MX80-MX104 for the sensor is not working ? Here there is a link where explains how to decode data using unix utilities

https://www.juniper.net/documentation/en_US/junos/topics/reference/general/junos-telemetry-interface-decoding-data.html

Regards

henriklb commented 6 years ago

Hi! I have already done that actually, it worked fine for every .proto file on MX104 and MX80.

I also tested if the cpu/mem sensor worked with the collector Logstash (with a official plugin provided by Juniper). CPU/mem sensor worked fine in Logstash.

I know Juniper has a similar Plugin for Fluentd (https://github.com/Juniper/fluent-plugin-udp-native-sensors) but I haven't gotten around to test that one yet. If I understand correctly this FluentD-plugin provided by Juniper is (now) used in Open-NTI. I suspect the cpu/mem sensor parse error is either in the open-nti's fluentd-config-file. It could also be in the Juniper-plugin itself, but I somehow doubt that (haven't tested that yet tho)

Best Regards

psagrera commented 6 years ago

Actually, openNTI is based on https://github.com/Juniper/fluent-plugin-udp-native-sensors/tree/with_keys.

Try to remove this part from fluentd config file

#################
## Output     ###
#################

##### CPU #######

<match juniperNetworks.cpu_memory_util_ext>
    type copy
{% if OUTPUT_STDOUT == 'true' %}
    <store>
        @type stdout
        @id stdout_output
        localtime
    </store>
{% endif %}
{% if OUTPUT_INFLUXDB == 'true' %}
    <store>
        type influxdb

        host "{{ INFLUXDB_ADDR }}"
        port "{{ INFLUXDB_PORT }}"
        dbname "{{ INFLUXDB_DB }}"
        user "{{ INFLUXDB_USER }}"
        password "{{ INFLUXDB_PWD }}"
        time_precision ms
        tag_keys ["device","utilization.application_utilization.name","utilization.name"]
        tag_keys_field key_fields
        ####
        buffer_type memory
        # buffer_chunk_limit 524288 # 512 * 1024
        # buffer_queue_limit 1024
        flush_interval "{{ INFLUXDB_FLUSH_INTERVAL }}"
        # retry_limit 17
        # retry_wait 1.0
        # num_threads 1
    </store>
{% endif %}

## TODO investigate forest plugin https://github.com/tagomoris/fluent-plugin-forest

{% if OUTPUT_KAFKA == 'true' %}
    <store>
      @type kafka

      # Brokers: you can choose either brokers or zookeeper.
      brokers               {{ KAFKA_ADDR }}:{{ KAFKA_PORT }}
      # zookeeper           <zookeeper_host>:<zookeeper_port> # Set brokers via Zookeeper

      default_topic         {{ KAFKA_TOPIC }}
      # default_partition_key (string)   :default => nil
      output_data_type      {{ KAFKA_DATA_TYPE }} # |ltsv|msgpack|attr:<record name>|<formatter 
       name>)
      output_include_tag    true # |false) :default => false
      output_include_time   true # |false) :default => false

      # max_send_retries    (integer)    :default => 3
      # required_acks       (integer)    :default => 0
      # ack_timeout_ms      (integer)    :default => 1500
      compression_codec     {{ KAFKA_COMPRESSION_CODEC }} # (none|gzip|snappy) :default => 
      none
    </store>
{% endif %}
</match>

################ END CPU ################   

it will match here

################## ALL #############
<match juniperNetworks.**>
    type copy
{% if OUTPUT_STDOUT == 'true' %}
    <store>
        @type stdout
        @id stdout_output
        localtime
    </store>
{% endif %}

if you could share your .gpb data I would really appreciate.

henriklb commented 6 years ago

I removed the requested CPU part, restarted the container and verified fluentd is indeed running the modified fluentd.conf file. Now after deleting the CPU part in the config file NONE of the sensors are working. When I added the CPU part back again all sensors are working (except the CPU/mem one).

Now I'm more clueless than ever.

henriklb commented 6 years ago

I verified the both the ~/fluentd/fluent.conf and the /tmp/fluent.conf files in the opennti_input_jti container.

psagrera commented 6 years ago

It's really weird Could you send me fluentd file and logs , with and without cpu sensor ?
What sensors are configured ? cpu and interface stats ?

psagrera commented 6 years ago

In my setup , I'm removing that part :

2018-07-20 06:32:49 -0700 [warn]: rewrite_tag_filter: [DEPRECATED] Use section instead of rewriterule1 2018-07-20 06:32:49 -0700 [info]: adding rewrite_tag_filter rule: rewriterule1 ["sensor_name", /(.+)/, "", "${tag}.$1"] 2018-07-20 06:32:49 -0700 [info]: adding match pattern="juniperNetworks.jnpr_packet_statistics_ext" type="copy" 2018-07-20 06:32:49 -0700 [info]: adding match pattern="juniperNetworks.jnpr_lsp_statistics_ext" type="copy" 2018-07-20 06:32:49 -0700 [info]: adding match pattern="juniperNetworks." type="copy" 2018-07-20 06:32:49 -0700 [info]: adding match pattern="debug." type="stdout" 2018-07-20 06:32:49 -0700 [info]: adding match pattern="fluent.**" type="stdout"

and measurememts are created

show MEASUREMENTS name: measurements name

juniperNetworks.cpu_memory_util_ext juniperNetworks.jnpr_interface_ext

and we got data :

select * from "juniperNetworks.cpu_memory_util_ext" order by desc limit 1 name: juniperNetworks.cpu_memory_util_ext time _seq device host sensor_name utilization.application_utilization.allocations utilization.application_utilization.allocations_failed utilization.application_utilization.bytes_allocated utilization.application_utilization.frees utilization.application_utilization.name utilization.bytes_allocated utilization.name utilization.size utilization.utilization


2018-07-20T13:36:32.208Z 9 mx1_re:10.235.1.2 0965f615e337 cpu_memory_util_ext 132 0 62136 3 nh 181156328 Kernel 536866792 33

BTW, you can turn on more verbosity in input-jti , changing that ENV variable in the docker-compose.yml file

henriklb commented 6 years ago

How very odd. How do you start your open-nti system?

All I do is :

git clone https://github.com/Juniper/open-nti.git
cd open-nti
make start

Are you doing something different? I'm able to reproduce this error on a clean seperate machine as well, which makes me think that maybe you do not start up docker the same way?

psagrera commented 6 years ago

I do:

git clone https://github.com/Juniper/open-nti.git cd open-nti make build make start

when you removed cpu part from fluentd.conf , what errors were you getting ?

or simply there wasn't measurement ?

Regards

henriklb commented 6 years ago

Oh OK so I removed the CPU part of the fluent.conf again and now I see the data in influxdb correctly!

I only checked the CPU_JTI dashboard last time. As you know, the CPU dashboard is dependent on that the tags are defined. My bad.

So it is indeed related to that CPU part in the flunet.conf!

psagrera commented 6 years ago

Ok, the current status is:

With cpu part in the fluent.conf it doesn't work (only one insertion into the database) Without cpu part in the fluent.conf "it works" (but only device is inserted as TAG because is hitting in the default part)

Could you please try with the following CPU config (only device in tag_keys)?

<match juniperNetworks.cpu_memory_util_ext>
    type copy
  {% if OUTPUT_STDOUT == 'true' %}
   <store>
       @type stdout
       @id stdout_output
       localtime
   </store>
{% endif %}
{% if OUTPUT_INFLUXDB == 'true' %}
   <store>
       type influxdb

       host "{{ INFLUXDB_ADDR }}"
       port "{{ INFLUXDB_PORT }}"
       dbname "{{ INFLUXDB_DB }}"
       user "{{ INFLUXDB_USER }}"
       password "{{ INFLUXDB_PWD }}"
       time_precision ms
       tag_keys ["device"]
       tag_keys_field key_fields
       ####
       buffer_type memory
       # buffer_chunk_limit 524288 # 512 * 1024
       # buffer_queue_limit 1024
       flush_interval "{{ INFLUXDB_FLUSH_INTERVAL }}"
       # retry_limit 17
       # retry_wait 1.0
       # num_threads 1
   </store>
{% endif %}

{% if OUTPUT_KAFKA == 'true' %}
    <store>
      @type kafka

     # Brokers: you can choose either brokers or zookeeper.
     brokers               {{ KAFKA_ADDR }}:{{ KAFKA_PORT }}
     # zookeeper           <zookeeper_host>:<zookeeper_port> # Set brokers via Zookeeper

     default_topic         {{ KAFKA_TOPIC }}
     # default_partition_key (string)   :default => nil
     output_data_type      {{ KAFKA_DATA_TYPE }} # |ltsv|msgpack|attr:<record name>|<formatter 
      name>)
     output_include_tag    true # |false) :default => false
     output_include_time   true # |false) :default => false

     # max_send_retries    (integer)    :default => 3
     # required_acks       (integer)    :default => 0
     # ack_timeout_ms      (integer)    :default => 1500
     compression_codec     {{ KAFKA_COMPRESSION_CODEC }} # (none|gzip|snappy) :default => 
     none
   </store>
{% endif %}

psagrera commented 6 years ago

@henriklb

What's the current status ?

Regards

Pablo

henriklb commented 6 years ago

Oh OK so I removed the CPU part of the fluent.conf again and now I see the data in influxdb correctly!

I only checked the CPU_JTI dashboard last time. As you know, the CPU dashboard is dependent on that the tags are defined. My bad.

So it is indeed related to that CPU part in the flunet.conf!

I'm not able to reproduce this. So the current status is that CPU data still does not work. It inserts data at startup once, then stops. So the CPU issue still stands. Also I do not have access to the Juniper equipment anymore. so I wont' be able to assist further.

Where you able to test it with an MX80 or MX104?

psagrera commented 6 years ago

Hi,

Unfortunately , I couldn’t get an mx80/104 for testing. I will try to get one in the upcoming weeks to see if I can reproduce the issue. In the meantime I will close that issue.

Regards

Pablo

Enviado desde mi iPhone

El 14 sept 2018, a las 13:13, henriklb notifications@github.com escribió:

Oh OK so I removed the CPU part of the fluent.conf again and now I see the data in influxdb correctly!

I only checked the CPU_JTI dashboard last time. As you know, the CPU dashboard is dependent on that the tags are defined. My bad.

So it is indeed related to that CPU part in the flunet.conf! I'm not able to reproduce this. So the current status is that CPU data still does not work. It inserts data at startup once, then stops. So the CPU issue still stands. Also I do not have access to the Juniper equipment anymore. so I wont' be able to assist further.

Where you able to test it with an MX80 or MX104?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

drogbeer commented 5 years ago

Also get these errors, can't parse a tag or one of the values. Using MX480 Junos 17.3 R3.10

unable to parse 'juniperNetworks.cpu_memory_util_ext,utilization.name=Kernel,utilization.application_utilization.name=nh,device=DEVICEX,_seq=9 utilization.size=3275747304i,utilization.bytes_allocated=619012272i,utilization.utilization=18i,utilization.application_utilization.bytes_allocated=18446744073703923088i,utilization.application_utilization.allocations=3782591i,utilization.application_utilization.frees=4302452i,utilization.application_utilization.allocations_failed=0i,host=\\\\"someserver.server.com\\\\",sensor_name=\\\\"cpu_memory_util_ext\\\\" 1550036800163':

Could this be due to https://github.com/influxdata/influxdb/issues/5204 ?