Open henriklb opened 6 years ago
I might see some initial data in the "juniperNetworks.cpu_memory_util_ext" table, but it stop collecting/parsing data instantly right after one initial "push"/entry in the database. Every entry has identical timestamp, then it stop receiving cpu data altogether. cpu.db.log.txt
Could you please execute the following command inside the influxdb container ?: select * from "juniperNetworks.cpu_memory_util_ext" group by " utilization.application_utilization.name" order by desc limit 5
Regards.
2018-07-09 13:45 GMT+02:00 henriklb notifications@github.com:
I might see some initial data in the "juniperNetworks.cpu_memory_util_ext" table, but it stop receiving data instantly right after one initial "push"/entry in the database. Every entry has identical timestamp, then it stop receiving cpu data altogether. cpu.db.log.txt https://github.com/Juniper/open-nti/files/2175961/cpu.db.log.txt
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-403451608, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZRGJgng6ESYiuHMFCLlO3dah78A7ks5uE0JhgaJpZM4VHgjE .
Could you please set "precision rfc3339" before execute that command :) ?
precision rfc3339 select * from "juniperNetworks.cpu_memory_util_ext" group by " utilization.application_utilization.name" order by desc limit 5
2018-07-09 16:02 GMT+02:00 henriklb notifications@github.com:
Here you go, and thanks! pasto.txt https://github.com/Juniper/open-nti/files/2176465/pasto.txt
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-403489981, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZasnu8moK4sfr_IB30s5Kg9w26-Zks5uE2J7gaJpZM4VHgjE .
Did that give you any clues?
If I restart the juniper/open-nti-input-jti container the cpu/mem datapoints gets inserted once and then stops. I attached my fluentd logs.
Hi
Could you please type : docker images ?
and go inside the jti-container and execute : ls -lrt under /fluent-plugin-udp-native-sensors dierctory ?
Anoher thing you can try is restart all containers (make stop / make start)
Regards
2018-07-11 10:38 GMT+02:00 henriklb notifications@github.com:
If I restart the juniper/open-nti-input-jti container the cpu/mem datapoints gets inserted once and then stops. I attached my fluentd logs.
fluentdlogs.txt https://github.com/Juniper/open-nti/files/2183670/fluentdlogs.txt
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404090599, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZZo-AcqSJsOoJVlbJQ-_IdpiXevhks5uFblpgaJpZM4VHgjE .
Let me know if you need anything else. Docker images:
REPOSITORY TAG IMAGE ID CREATED SIZE
kapacitor 1.5.0 fb3f94851c72 4 days ago 254MB
juniper/open-nti latest c0b2d2db3a32 4 days ago 1.09GB
juniper/open-nti-input-jti latest 33fc98958829 4 days ago 2.53GB
juniper/open-nti-input-syslog latest b80ffd1a2372 4 days ago 110MB
open-nti_input-oc latest 5feb22b05c88 5 days ago 211MB
juniper/open-nti <none> 4a61d113c345 7 days ago 1.09GB
juniper/open-nti-input-jti <none> 9c95511ac762 7 days ago 2.53GB
juniper/open-nti-input-syslog <none> 090304f89a81 7 days ago 110MB
telegraf 1.7 5347c50a412b 2 weeks ago 211MB
quay.io/influxdb/chronograf 1.5.0.1 fda001d20828 5 weeks ago 43.1MB
root@84367655c790:/fluent-plugin-udp-native-sensors# ls -lrt
total 72
drwxr-xr-x 2 root root 4096 Jun 18 10:18 test
-rw-r--r-- 1 root root 37 Jun 18 10:18 requirements.txt
-rw-r--r-- 1 root root 9551 Jun 18 10:18 juniper_telemetry_udp_lib.rb
-rw-r--r-- 1 root root 1094 Jun 18 10:18 fluent-plugin-udp-native-sensors.gemspec
-rw-r--r-- 1 root root 225 Jun 18 10:18 Rakefile
-rw-r--r-- 1 root root 2801 Jun 18 10:18 README.md
-rw-r--r-- 1 root root 10142 Jun 18 10:18 LICENSE
-rw-r--r-- 1 root root 114 Jun 18 10:18 Gemfile
drwxr-xr-x 1 root root 4096 Jul 3 13:57 lib
-rw-r--r-- 1 root root 18944 Jul 3 15:19 fluent-plugin-udp-native-sensors-0.0.1.gem
I've tried open-nti make start and make stop, but same result.
Could you please remove all images with TAG
That's my setup
root@opennti1:~# docker images REPOSITORY TAG IMAGE ID CREATED SIZE juniper/open-nti latest 1e5b12569cc9 4 days ago 1.09 GB opennti_input-oc latest 6cdb7d2da4b8 6 days ago 211 MB juniper/open-nti-input-internal latest 9a92af7259a3 6 days ago 131 MB juniper/open-nti-input-oc latest 487e864bc99a 6 days ago 211 MB juniper/open-nti-input-syslog latest 29b3569d3d2a 6 days ago 110 MB juniper/open-nti-input-jti latest 31c6905ee6ee 6 days ago 2.53 GB psagrera/fluent-jti 1.1 1c75c66a971a 7 days ago 2.53 GB telegraf 1.7 5347c50a412b 2 weeks ago 211 MB kapacitor 1.5.0 0663d599c590 2 weeks ago 254 MB quay.io/influxdb/chronograf 1.5.0.1 fda001d20828 5 weeks ago 43.1 MB
So, I would remove images with TAG
Regards
2018-07-11 12:38 GMT+02:00 henriklb notifications@github.com:
Let me know if you need anything else. Docker images:
REPOSITORY TAG IMAGE ID CREATED SIZE kapacitor 1.5.0 fb3f94851c72 4 days ago 254MB juniper/open-nti latest c0b2d2db3a32 4 days ago 1.09GB juniper/open-nti-input-jti latest 33fc98958829 4 days ago 2.53GB juniper/open-nti-input-syslog latest b80ffd1a2372 4 days ago 110MB open-nti_input-oc latest 5feb22b05c88 5 days ago 211MB juniper/open-nti
4a61d113c345 7 days ago 1.09GB juniper/open-nti-input-jti 9c95511ac762 7 days ago 2.53GB juniper/open-nti-input-syslog 090304f89a81 7 days ago 110MB telegraf 1.7 5347c50a412b 2 weeks ago 211MB root@84367655c790:/fluent-plugin-udp-native-sensors# ls -lrt total 72 drwxr-xr-x 2 root root 4096 Jun 18 10:18 test -rw-r--r-- 1 root root 37 Jun 18 10:18 requirements.txt -rw-r--r-- 1 root root 9551 Jun 18 10:18 juniper_telemetry_udp_lib.rb -rw-r--r-- 1 root root 1094 Jun 18 10:18 fluent-plugin-udp-native-sensors.gemspec -rw-r--r-- 1 root root 225 Jun 18 10:18 Rakefile -rw-r--r-- 1 root root 2801 Jun 18 10:18 README.md -rw-r--r-- 1 root root 10142 Jun 18 10:18 LICENSE -rw-r--r-- 1 root root 114 Jun 18 10:18 Gemfile drwxr-xr-x 1 root root 4096 Jul 3 13:57 lib -rw-r--r-- 1 root root 18944 Jul 3 15:19 fluent-plugin-udp-native-sensors-0.0.1.gem
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404124437, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZYQ3u9P_AsRddTC8xkcX5elY0Vauks5uFdXBgaJpZM4VHgjE .
Removed ALL images, cloned from repo again, 'make build' followed by 'make start' (also tried once with only make start).
Still exactly the same behavior cpu and memory data pushed once in the database then stops , (*/interface/ works fine once again)
cpu_memory_utilization.pb.rb from "juniper/open-nti-input-jti"
REPOSITORY TAG IMAGE ID CREATED SIZE
open-nti_input-oc latest 13fc7f107d61 About a minute ago 211MB
juniper/open-nti-input-internal latest decb27812bb6 3 minutes ago 131MB
juniper/open-nti-input-oc latest e35702494754 3 minutes ago 211MB
juniper/open-nti-input-syslog latest f38866a1be96 4 minutes ago 110MB
juniper/open-nti-input-jti latest e2c7483add93 5 minutes ago 2.53GB
juniper/open-nti latest 0fa5ab15d24b 6 minutes ago 1.09GB
telegraf 1.7 476f4269538d 4 days ago 211MB
kapacitor 1.5.0 fb3f94851c72 4 days ago 254MB
psagrera/fluent-jti 1.1 1c75c66a971a 7 days ago 2.53GB
quay.io/influxdb/chronograf 1.5.0.1 fda001d20828 5 weeks ago 43.1MB
docker.elastic.co/logstash/logstash 6.2.4 00a38ba5444c 2 months ago 657MB
phusion/baseimage 0.9.22 877509368a8d 13 months ago 225MB
juniper/pyez 2.0.1 e61c159ee89f 22 months ago 93MB
fluent/fluentd v0.12.29 d0e2862496ea 22 months ago 36MB
Are you still getting those errors in fluentd ? it would be possible to test with another JUNOS release ?
2018-07-11 15:05 GMT+02:00 henriklb notifications@github.com:
Removed ALL images, cloned from repo again, 'make build' followed by 'make start' (also tried once with only make start).
Still exactly the same behavior cpu and memory data pushed once in the database then stops , (*/interface/ works fine once again)
cpu_memory_utilization.pb.rb.txt https://github.com/Juniper/open-nti/files/2184530/cpu_memory_utilization.pb.rb.txt
REPOSITORY TAG IMAGE ID CREATED SIZE open-nti_input-oc latest 13fc7f107d61 About a minute ago 211MB juniper/open-nti-input-internal latest decb27812bb6 3 minutes ago 131MB juniper/open-nti-input-oc latest e35702494754 3 minutes ago 211MB juniper/open-nti-input-syslog latest f38866a1be96 4 minutes ago 110MB juniper/open-nti-input-jti latest e2c7483add93 5 minutes ago 2.53GB juniper/open-nti latest 0fa5ab15d24b 6 minutes ago 1.09GB telegraf 1.7 476f4269538d 4 days ago 211MB kapacitor 1.5.0 fb3f94851c72 4 days ago 254MB psagrera/fluent-jti 1.1 1c75c66a971a 7 days ago 2.53GBquay.io/influxdb/chronograf 1.5.0.1 fda001d20828 5 weeks ago 43.1MBdocker.elastic.co/logstash/logstash 6.2.4 00a38ba5444c 2 months ago 657MB phusion/baseimage 0.9.22 877509368a8d 13 months ago 225MB juniper/pyez 2.0.1 e61c159ee89f 22 months ago 93MB fluent/fluentd v0.12.29 d0e2862496ea 22 months ago 36MB
[image: screenshot from 2018-07-09 14-12-43] https://user-images.githubusercontent.com/11011393/42573219-c5f98bf2-851b-11e8-9929-49cf569b5148.png
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404161800, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZTZX7U3SUp-l4fVHJe_Y-i-EnLJDks5uFfgqgaJpZM4VHgjE .
Could you log in into the Linecard and execute :
MX-0(HL3-1-2 vty)# show agent sensors id "xxxxxx" , being "xxxxxx" cpu-sensor ID
2018-07-11 15:42 GMT+02:00 Pablo Sagrera Garcia psagrerag@gmail.com:
Are you still getting those errors in fluentd ? it would be possible to test with another JUNOS release ?
2018-07-11 15:05 GMT+02:00 henriklb notifications@github.com:
Removed ALL images, cloned from repo again, 'make build' followed by 'make start' (also tried once with only make start).
Still exactly the same behavior cpu and memory data pushed once in the database then stops , (*/interface/ works fine once again)
cpu_memory_utilization.pb.rb.txt https://github.com/Juniper/open-nti/files/2184530/cpu_memory_utilization.pb.rb.txt
REPOSITORY TAG IMAGE ID CREATED SIZE open-nti_input-oc latest 13fc7f107d61 About a minute ago 211MB juniper/open-nti-input-internal latest decb27812bb6 3 minutes ago 131MB juniper/open-nti-input-oc latest e35702494754 3 minutes ago 211MB juniper/open-nti-input-syslog latest f38866a1be96 4 minutes ago 110MB juniper/open-nti-input-jti latest e2c7483add93 5 minutes ago 2.53GB juniper/open-nti latest 0fa5ab15d24b 6 minutes ago 1.09GB telegraf 1.7 476f4269538d 4 days ago 211MB kapacitor 1.5.0 fb3f94851c72 4 days ago 254MB psagrera/fluent-jti 1.1 1c75c66a971a 7 days ago 2.53GBquay.io/influxdb/chronograf 1.5.0.1 fda001d20828 5 weeks ago 43.1MBdocker.elastic.co/logstash/logstash 6.2.4 00a38ba5444c 2 months ago 657MB phusion/baseimage 0.9.22 877509368a8d 13 months ago 225MB juniper/pyez 2.0.1 e61c159ee89f 22 months ago 93MB fluent/fluentd v0.12.29 d0e2862496ea 22 months ago 36MB
[image: screenshot from 2018-07-09 14-12-43] https://user-images.githubusercontent.com/11011393/42573219-c5f98bf2-851b-11e8-9929-49cf569b5148.png
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404161800, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZTZX7U3SUp-l4fVHJe_Y-i-EnLJDks5uFfgqgaJpZM4VHgjE .
Same errors in fluentd, I can try a different junos version, do you have a particular one in mind? I do not have the root password at hand so that would not be possible right now (maybe tomorrow).
My last test was done with 18.1R1.7 (but never tested neither MX80 nor MX104 , so whatever different release to 17.3R2 (preferably 18.X ) is fine in order to see if the problem is seen in more than one release). Bear in mind that you have to update network-agent package accordingly
Regards
2018-07-11 16:54 GMT+02:00 henriklb notifications@github.com:
Same errors in fluentd, I can try a different junos version, do you have a particular one in mind? I do not have the root password at hand so that would not be possible right now (maybe tomorrow).
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404199215, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZalWYZK7bVVAkM4bh-zFKsQugwmQks5uFhGWgaJpZM4VHgjE .
Tested with 18.1 on an MX80, same parse error in fluentd.
I have the same problem with QFX5110 running Junos 18.1R1. No CPU/MEM data and interface statistics in the database
Hi,
I'll try to get an MX80 to test it, in the meantime could you test other sensors ?
Regards
2018-07-12 17:50 GMT+02:00 henriklb notifications@github.com:
Tested with 18.1 on an MX80, same parse error in fluentd.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404559816, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZZlS9PGsSOS3YFqNkTN0WW17uYrHks5uF3BhgaJpZM4VHgjE .
Tested with vMX, after updating to latest version stoped working.
On Fri, 13 Jul 2018 at 1:27 PM psagrera notifications@github.com wrote:
Hi,
I'll try to get an MX80 to test it, in the meantime could you test other sensors ?
Regards
2018-07-12 17:50 GMT+02:00 henriklb notifications@github.com:
Tested with 18.1 on an MX80, same parse error in fluentd.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404559816, or mute the thread < https://github.com/notifications/unsubscribe-auth/AKgjZZlS9PGsSOS3YFqNkTN0WW17uYrHks5uF3BhgaJpZM4VHgjE
.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404793575, or mute the thread https://github.com/notifications/unsubscribe-auth/ATy4v0b0Pz9L84u5qXrTmnbF0bSbzdRaks5uGHYBgaJpZM4VHgjE .
you mean only one insertion into the database ? same fluentd errors ? Only happen with cpu sensor ?
Regards
2018-07-13 14:14 GMT+02:00 anaaman notifications@github.com:
Tested with vMX, after updating to latest version stoped working.
On Fri, 13 Jul 2018 at 1:27 PM psagrera notifications@github.com wrote:
Hi,
I'll try to get an MX80 to test it, in the meantime could you test other sensors ?
Regards
2018-07-12 17:50 GMT+02:00 henriklb notifications@github.com:
Tested with 18.1 on an MX80, same parse error in fluentd.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/Juniper/open-nti/issues/227#issuecomment-404559816 , or mute the thread < https://github.com/notifications/unsubscribe-auth/ AKgjZZlS9PGsSOS3YFqNkTN0WW17uYrHks5uF3BhgaJpZM4VHgjE
.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404793575, or mute the thread https://github.com/notifications/unsubscribe-auth/ ATy4v0b0Pz9L84u5qXrTmnbF0bSbzdRaks5uGHYBgaJpZM4VHgjE .
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-404815748, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZeeE80lUjFQH-eymUc4owN_FSbGGks5uGI80gaJpZM4VHgjE .
you mean only one insertion into the database ? - I have no data at all, I see only the host IP in the device filtering. Only happen with cpu sensor ? No, This happen also with interface statistics. same fluentd errors ? didn't see any errors, This is the output of " docker logs opennti_input_jti" :
Running /etc/my_init.d/00_regen_ssh_host_keys.sh...
Running /etc/rc.local...
Booting runit daemon...
Runit started as PID 7
[root@telemtry open-nti]# docker logs chronograf_con
time="2018-07-16T07:30:23Z" level=info msg="Running migration 59b0cda4fc7909ff84ee5c4f9cb4b655b6a26620"
time="2018-07-16T07:30:23Z" level=info msg="Serving chronograf at http://[::]:8888" component=server
time="2018-07-16T07:30:23Z" level=info msg="Reporting usage stats" component=usage freq=24h reporting_addr="https://usage.influxdata.com" stats="os,arch,version,cluster_id,uptime"
[root@telemtry open-nti]# docker logs opennti_input_jti
2018-07-16 10:30:24 +0300 [info]: reading config file path="/tmp/fluent.conf"
2018-07-16 10:30:24 +0300 [info]: starting fluentd-0.12.43
2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.6.0'
2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-udp-native-sensors' version '0.0.1'
2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.43'
2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.42'
2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks" type="rewrite_tag_filter"
2018-07-16 10:30:25 +0300 [warn]: rewrite_tag_filter: [DEPRECATED] Use
<match juniperNetworks.**> type copy
@anaaman , are you receiving data from the vMX ? Could you please execute a tcpdump in the server ? is there any measurement in the juniper database ?
@henriklb, I'm testing with vMX running 18.1 and it's working fine in my case. I'm still trying to get a MX80 or MX104 for testing in that models.
2018-07-16 9:55 GMT+02:00 anaaman notifications@github.com:
you mean only one insertion into the database ? - I have no data at all, I see only the host IP in the device filtering. Only happen with cpu sensor ? No, This happen also with interface statistics. same fluentd errors ? didn't see any errors, This is the output of " docker logs opennti_input_jti" :
Running /etc/my_init.d/00_regen_ssh_host_keys.sh... Running /etc/rc.local... Booting runit daemon... Runit started as PID 7 [root@telemtry open-nti]# docker logs chronograf_con time="2018-07-16T07:30:23Z" level=info msg="Running migration 59b0cda4fc7909ff84ee5c4f9cb4b655b6a26620" time="2018-07-16T07:30:23Z" level=info msg="Serving chronograf at http://[::]:8888" component=server time="2018-07-16T07:30:23Z" level=info msg="Reporting usage stats" component=usage freq=24h reporting_addr="https://usage.influxdata.com" stats="os,arch,version,cluster_id,uptime" [root@telemtry open-nti]# docker logs opennti_input_jti 2018-07-16 10:30:24 +0300 [info]: reading config file path="/tmp/fluent.conf" 2018-07-16 10:30:24 +0300 [info]: starting fluentd-0.12.43 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.6.0' 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-udp-native-sensors' version '0.0.1' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.43' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.42' 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks" type="rewrite_tag_filter" 2018-07-16 10:30:25 +0300 [warn]: rewrite_tag_filter: [DEPRECATED] Use section instead of rewriterule1 2018-07-16 10:30:25 +0300 [info]: adding rewrite_tag_filter rule: rewriterule1 ["sensor_name", /(.+)/, "", "${tag}.$1"] 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.cpu_memory_util_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_packet_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_lsp_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks. " type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="debug." type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="fluent.**" type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding source type="forward" 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [warn]: 'body_size_limit' parameter is deprecated: use message_length_limit instead. 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [info]: adding source type="monitor_agent" 2018-07-16 10:30:25 +0300 [info]: adding source type="debug_agent" 2018-07-16 10:30:25 +0300 [info]: using configuration file: @type https://github.com/type forward @id https://github.com/id forward_input @type https://github.com/type udp tag juniperNetworks format juniper_udp_native port 50000 bind 0.0.0.0 body_size_limit 5000 @type https://github.com/type udp tag jnpr.analyticsd format juniper_analyticsd message_length_limit 5000 remove_newline false port 50020 bind 0.0.0.0 @type https://github.com/type rewrite_tag_filter rewriterule1 sensor_name (.+) ${tag}.$1 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","utilization. application_utilization.name","utilization.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","packet_stats.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","lsp_stats_records.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device"] tag_keys_field key_fields buffer_type memory flush_interval 2 @type https://github.com/type monitor_agent @id https://github.com/id monitor_agent_input port 24220 @type https://github.com/type debug_agent @id https://github.com/id debug_agent_input bind 127.0.0.1 port 24230 @type https://github.com/type stdout @id https://github.com/id stdout_output @type https://github.com/type stdout 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: listening fluent socket on 0.0.0.0:24224 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50000 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50020 2018-07-16 10:30:25 +0300 [info]: listening dRuby uri="druby://127.0.0.1:24230" object="Engine"
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-405172215, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZb9JjS3eZPfVM-kNbEzrOorGiS-Yks5uHEcAgaJpZM4VHgjE .
Hi, I have older version of Open-NTI and when I set it as the collector everything work fine. I ran tcpdump on my docker host and I can see the data coming in. In the “juniper” database I have those measurements:
juniperNetworks.cpu_memory_util_ext juniperNetworks.jnprLogicalInterfaceExt
It looks like the measurements are empty.
Regrads
From: psagrera Sent: Monday, July 16, 2018 4:59 PM To: Juniper/open-nti Cc: anaaman; Mention Subject: Re: [Juniper/open-nti] JTI cpu data not appearing in the InfluxDB(#227)
@anaaman , are you receiving data from the vMX ? Could you please execute a tcpdump in the server ? is there any measurement in the juniper database ?
@henriklb, I'm testing with vMX running 18.1 and it's working fine in my case. I'm still trying to get a MX80 or MX104 for testing in that models.
2018-07-16 9:55 GMT+02:00 anaaman notifications@github.com:
you mean only one insertion into the database ? - I have no data at all, I see only the host IP in the device filtering. Only happen with cpu sensor ? No, This happen also with interface statistics. same fluentd errors ? didn't see any errors, This is the output of " docker logs opennti_input_jti" :
Running /etc/my_init.d/00_regen_ssh_host_keys.sh... Running /etc/rc.local... Booting runit daemon... Runit started as PID 7 [root@telemtry open-nti]# docker logs chronograf_con time="2018-07-16T07:30:23Z" level=info msg="Running migration 59b0cda4fc7909ff84ee5c4f9cb4b655b6a26620" time="2018-07-16T07:30:23Z" level=info msg="Serving chronograf at http://[::]:8888" component=server time="2018-07-16T07:30:23Z" level=info msg="Reporting usage stats" component=usage freq=24h reporting_addr="https://usage.influxdata.com" stats="os,arch,version,cluster_id,uptime" [root@telemtry open-nti]# docker logs opennti_input_jti 2018-07-16 10:30:24 +0300 [info]: reading config file path="/tmp/fluent.conf" 2018-07-16 10:30:24 +0300 [info]: starting fluentd-0.12.43 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.6.0' 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-udp-native-sensors' version '0.0.1' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.43' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.42' 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks" type="rewrite_tag_filter" 2018-07-16 10:30:25 +0300 [warn]: rewrite_tag_filter: [DEPRECATED] Use section instead of rewriterule1 2018-07-16 10:30:25 +0300 [info]: adding rewrite_tag_filter rule: rewriterule1 ["sensor_name", /(.+)/, "", "${tag}.$1"] 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.cpu_memory_util_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_packet_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_lsp_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks. " type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="debug." type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="fluent.**" type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding source type="forward" 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [warn]: 'body_size_limit' parameter is deprecated: use message_length_limit instead. 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [info]: adding source type="monitor_agent" 2018-07-16 10:30:25 +0300 [info]: adding source type="debug_agent" 2018-07-16 10:30:25 +0300 [info]: using configuration file: @type https://github.com/type forward @id https://github.com/id forward_input @type https://github.com/type udp tag juniperNetworks format juniper_udp_native port 50000 bind 0.0.0.0 body_size_limit 5000 @type https://github.com/type udp tag jnpr.analyticsd format juniper_analyticsd message_length_limit 5000 remove_newline false port 50020 bind 0.0.0.0 @type https://github.com/type rewrite_tag_filter rewriterule1 sensor_name (.+) ${tag}.$1 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","utilization. application_utilization.name","utilization.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","packet_stats.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","lsp_stats_records.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device"] tag_keys_field key_fields buffer_type memory flush_interval 2 @type https://github.com/type monitor_agent @id https://github.com/id monitor_agent_input port 24220 @type https://github.com/type debug_agent @id https://github.com/id debug_agent_input bind 127.0.0.1 port 24230 @type https://github.com/type stdout @id https://github.com/id stdout_output @type https://github.com/type stdout 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: listening fluent socket on 0.0.0.0:24224 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50000 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50020 2018-07-16 10:30:25 +0300 [info]: listening dRuby uri="druby://127.0.0.1:24230" object="Engine"
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-405172215, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZb9JjS3eZPfVM-kNbEzrOorGiS-Yks5uHEcAgaJpZM4VHgjE .
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
so if you run "select * from "juniperNetworks.cpu_memory_util_ext" order by desc limit 1" it doesn't show anything, right ? is there any way to gain access to your setup ?
2018-07-16 16:23 GMT+02:00 anaaman notifications@github.com:
Hi, I have older version of Open-NTI and when I set it as the collector everything work fine. I ran tcpdump on my docker host and I can see the data coming in. In the “juniper” database I have those measurements:
juniperNetworks.cpu_memory_util_ext juniperNetworks.jnprLogicalInterfaceExt
It looks like the measurements are empty.
Regrads
From: psagrera Sent: Monday, July 16, 2018 4:59 PM To: Juniper/open-nti Cc: anaaman; Mention Subject: Re: [Juniper/open-nti] JTI cpu data not appearing in the InfluxDB(#227)
@anaaman , are you receiving data from the vMX ? Could you please execute a tcpdump in the server ? is there any measurement in the juniper database ?
@henriklb, I'm testing with vMX running 18.1 and it's working fine in my case. I'm still trying to get a MX80 or MX104 for testing in that models.
2018-07-16 9:55 GMT+02:00 anaaman notifications@github.com:
you mean only one insertion into the database ? - I have no data at all, I see only the host IP in the device filtering. Only happen with cpu sensor ? No, This happen also with interface statistics. same fluentd errors ? didn't see any errors, This is the output of " docker logs opennti_input_jti" :
Running /etc/my_init.d/00_regen_ssh_host_keys.sh... Running /etc/rc.local... Booting runit daemon... Runit started as PID 7 [root@telemtry open-nti]# docker logs chronograf_con time="2018-07-16T07:30:23Z" level=info msg="Running migration 59b0cda4fc7909ff84ee5c4f9cb4b655b6a26620" time="2018-07-16T07:30:23Z" level=info msg="Serving chronograf at http://
component=server time="2018-07-16T07:30:23Z" level=info msg="Reporting usage stats" component=usage freq=24h reporting_addr="https://usage.influxdata.com" stats="os,arch,version,cluster_id,uptime" [root@telemtry open-nti]# docker logs opennti_input_jti 2018-07-16 10:30:24 +0300 [info]: reading config file path="/tmp/fluent.conf" 2018-07-16 10:30:24 +0300 [info]: starting fluentd-0.12.43 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.6.0' 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-udp-native-sensors' version '0.0.1' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.43' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.42' 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks" type="rewrite_tag_filter" 2018-07-16 10:30:25 +0300 [warn]: rewrite_tag_filter: [DEPRECATED] Use section instead of rewriterule1 2018-07-16 10:30:25 +0300 [info]: adding rewrite_tag_filter rule: rewriterule1 ["sensor_name", /(.+)/, "", "${tag}.$1"] 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.cpu_memory_util_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_packet_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_lsp_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks. " type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="debug." type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="fluent.**" type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding source type="forward" 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [warn]: 'body_size_limit' parameter is deprecated: use message_length_limit instead. 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [info]: adding source type="monitor_agent" 2018-07-16 10:30:25 +0300 [info]: adding source type="debug_agent" 2018-07-16 10:30:25 +0300 [info]: using configuration file: @type https://github.com/type forward @id https://github.com/id forward_input @type https://github.com/type udp tag juniperNetworks format juniper_udp_native port 50000 bind 0.0.0.0 body_size_limit 5000 @type https://github.com/type udp tag jnpr.analyticsd format juniper_analyticsd message_length_limit 5000 remove_newline false port 50020 bind 0.0.0.0 @type https://github.com/type rewrite_tag_filter rewriterule1 sensor_name (.+) ${tag}.$1 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","utilization. application_utilization.name","utilization.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","packet_stats.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","lsp_stats_records.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device"] tag_keys_field key_fields buffer_type memory flush_interval 2 @type https://github.com/type monitor_agent @id https://github.com/id monitor_agent_input port 24220 @type https://github.com/type debug_agent @id https://github.com/id debug_agent_input bind 127.0.0.1 port 24230 @type https://github.com/type stdout @id https://github.com/id stdout_output @type https://github.com/type stdout 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: listening fluent socket on 0.0.0.0:24224 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50000 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50020 2018-07-16 10:30:25 +0300 [info]: listening dRuby uri="druby://127.0.0.1:24230" object="Engine"
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-405172215, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZb9JjS3eZPfVM- kNbEzrOorGiS-Yks5uHEcAgaJpZM4VHgjE .
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-405264483, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZaYoPj29YdUUdWbCbqJ76VKqm6ICks5uHKHdgaJpZM4VHgjE .
There is no data in the database. Now I can see this in open_nti_input_jti log:
2018-07-16 18:01:20 +0300 fluent.warn: {"next_retry":"2018-07-16 18:01:21 +0300","error_class":"InfluxDB::Error","error":"{\"error\":\"partial write: unable to parse 'juniperNetworks.cpu_memory_util_ext,utilization.name=Kernel,utilization.application_utilization.name=nh,device=vmx-17-4:192.168.1.252,_seq=9 utilization.size=2147479528i,utilization.bytes_allocated=164003368i,utilization.utilization=7i,utilization.application_utilization.bytes_allocated=18446744073704795472i,utilization.application_utilization.allocations=627853i,utilization.application_utilization.frees=942536i,utilization.application_utilization.allocations_failed=0i,host=\\"136a4ef27333\\",sensor_name=\\"cpu_memory_util_ext\\" 1531753278598': unable to parse integer 18446744073704795472: strconv.ParseInt: parsing \\"18446744073704795472\\": value out of range dropped=0\"}\n","plugin_id":"object:2acbf96c9928","message":"temporarily failed to flush the buffer. next_retry=2018-07-16 18:01:21 +0300 error_class=\"InfluxDB::Error\" error=\"{\\"error\\":\\"partial write: unable to parse 'juniperNetworks.cpu_memory_util_ext,utilization.name=Kernel,utilization.application_utilization.name=nh,device=vmx-17-4:192.168.1.252,_seq=9 utilization.size=2147479528i,utilization.bytes_allocated=164003368i,utilization.utilization=7i,utilization.application_utilization.bytes_allocated=18446744073704795472i,utilization.application_utilization.allocations=627853i,utilization.application_utilization.frees=942536i,utilization.application_utilization.allocations_failed=0i,host=\\\\"136a4ef27333\\\\",sensor_name=\\\\"cpu_memory_util_ext\\\\" 1531753278598': unable to parse integer 18446744073704795472: strconv.ParseInt: parsing \\\\"18446744073704795472\\\\": value out of range dropped=0\\"}\n\" plugin_id=\"object:2acbf96c9928\""} 2018-07-16 18:01:21 +0300 [warn]: temporarily failed to flush the buffer. next_retry=2018-07-16 18:01:23 +0300 error_class="InfluxDB::Error" error="{\"error\":\"partial write: unable to parse 'juniperNetworks.cpu_memory_util_ext,utilization.name=Kernel,utilization.application_utilization.name=nh,device=vmx-17-4:192.168.1.252,_seq=9 utilization.size=2147479528i,utilization.bytes_allocated=164003368i,utilization.utilization=7i,utilization.application_utilization.bytes_allocated=18446744073704795472i,utilization.application_utilization.allocations=627853i,utilization.application_utilization.frees=942536i,utilization.application_utilization.allocations_failed=0i,host=\\"136a4ef27333\\",sensor_name=\\"cpu_memory_util_ext\\" 1531753278598': unable to parse integer 18446744073704795472: strconv.ParseInt: parsing \\"18446744073704795472\\": value out of range dropped=0\"}\n" plugin_id="object:2acbf96c9928" 2018-07-16 18:01:21 +0300 [warn]: suppressed same stacktrace
Sent from Mail for Windows 10
From: psagrera Sent: Monday, July 16, 2018 5:43 PM To: Juniper/open-nti Cc: anaaman; Mention Subject: Re: [Juniper/open-nti] JTI cpu data not appearing in the InfluxDB(#227)
so if you run "select * from "juniperNetworks.cpu_memory_util_ext" order by desc limit 1" it doesn't show anything, right ? is there any way to gain access to your setup ?
2018-07-16 16:23 GMT+02:00 anaaman notifications@github.com:
Hi, I have older version of Open-NTI and when I set it as the collector everything work fine. I ran tcpdump on my docker host and I can see the data coming in. In the “juniper” database I have those measurements:
juniperNetworks.cpu_memory_util_ext juniperNetworks.jnprLogicalInterfaceExt
It looks like the measurements are empty.
Regrads
From: psagrera Sent: Monday, July 16, 2018 4:59 PM To: Juniper/open-nti Cc: anaaman; Mention Subject: Re: [Juniper/open-nti] JTI cpu data not appearing in the InfluxDB(#227)
@anaaman , are you receiving data from the vMX ? Could you please execute a tcpdump in the server ? is there any measurement in the juniper database ?
@henriklb, I'm testing with vMX running 18.1 and it's working fine in my case. I'm still trying to get a MX80 or MX104 for testing in that models.
2018-07-16 9:55 GMT+02:00 anaaman notifications@github.com:
you mean only one insertion into the database ? - I have no data at all, I see only the host IP in the device filtering. Only happen with cpu sensor ? No, This happen also with interface statistics. same fluentd errors ? didn't see any errors, This is the output of " docker logs opennti_input_jti" :
Running /etc/my_init.d/00_regen_ssh_host_keys.sh... Running /etc/rc.local... Booting runit daemon... Runit started as PID 7 [root@telemtry open-nti]# docker logs chronograf_con time="2018-07-16T07:30:23Z" level=info msg="Running migration 59b0cda4fc7909ff84ee5c4f9cb4b655b6a26620" time="2018-07-16T07:30:23Z" level=info msg="Serving chronograf at http://
component=server time="2018-07-16T07:30:23Z" level=info msg="Reporting usage stats" component=usage freq=24h reporting_addr="https://usage.influxdata.com" stats="os,arch,version,cluster_id,uptime" [root@telemtry open-nti]# docker logs opennti_input_jti 2018-07-16 10:30:24 +0300 [info]: reading config file path="/tmp/fluent.conf" 2018-07-16 10:30:24 +0300 [info]: starting fluentd-0.12.43 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.6.0' 2018-07-16 10:30:25 +0300 [info]: gem 'fluent-plugin-udp-native-sensors' version '0.0.1' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.43' 2018-07-16 10:30:25 +0300 [info]: gem 'fluentd' version '0.12.42' 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks" type="rewrite_tag_filter" 2018-07-16 10:30:25 +0300 [warn]: rewrite_tag_filter: [DEPRECATED] Use section instead of rewriterule1 2018-07-16 10:30:25 +0300 [info]: adding rewrite_tag_filter rule: rewriterule1 ["sensor_name", /(.+)/, "", "${tag}.$1"] 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.cpu_memory_util_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_packet_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks.jnpr_lsp_statistics_ext" type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="juniperNetworks. " type="copy" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="debug." type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding match pattern="fluent.**" type="stdout" 2018-07-16 10:30:25 +0300 [info]: adding source type="forward" 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [warn]: 'body_size_limit' parameter is deprecated: use message_length_limit instead. 2018-07-16 10:30:25 +0300 [info]: adding source type="udp" 2018-07-16 10:30:25 +0300 [info]: adding source type="monitor_agent" 2018-07-16 10:30:25 +0300 [info]: adding source type="debug_agent" 2018-07-16 10:30:25 +0300 [info]: using configuration file: @type https://github.com/type forward @id https://github.com/id forward_input @type https://github.com/type udp tag juniperNetworks format juniper_udp_native port 50000 bind 0.0.0.0 body_size_limit 5000 @type https://github.com/type udp tag jnpr.analyticsd format juniper_analyticsd message_length_limit 5000 remove_newline false port 50020 bind 0.0.0.0 @type https://github.com/type rewrite_tag_filter rewriterule1 sensor_name (.+) ${tag}.$1 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","utilization. application_utilization.name","utilization.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","packet_stats.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device","lsp_stats_records.name"] tag_keys_field key_fields buffer_type memory flush_interval 2 type copy type influxdb host opennti port 8086 dbname juniper user juniper password xxxxxx time_precision ms tag_keys ["device"] tag_keys_field key_fields buffer_type memory flush_interval 2 @type https://github.com/type monitor_agent @id https://github.com/id monitor_agent_input port 24220 @type https://github.com/type debug_agent @id https://github.com/id debug_agent_input bind 127.0.0.1 port 24230 @type https://github.com/type stdout @id https://github.com/id stdout_output @type https://github.com/type stdout 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: Connecting to database: juniper, host: opennti, port: 8086, username: juniper, use_ssl = false, verify_ssl = true 2018-07-16 10:30:25 +0300 [info]: listening fluent socket on 0.0.0.0:24224 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50000 2018-07-16 10:30:25 +0300 [info]: listening udp socket on 0.0.0.0:50020 2018-07-16 10:30:25 +0300 [info]: listening dRuby uri="druby://127.0.0.1:24230" object="Engine"
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-405172215, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZb9JjS3eZPfVM- kNbEzrOorGiS-Yks5uHEcAgaJpZM4VHgjE .
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-405264483, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgjZaYoPj29YdUUdWbCbqJ76VKqm6ICks5uHKHdgaJpZM4VHgjE .
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
@henriklb, I'm testing with vMX running 18.1 and it's working fine in my case. I'm still trying to get a MX80 or MX104 for testing in that models.
Sound good. Thanks!
@anaaman , could you try to remove all images related with open NTI, and then execute make build again ?
@henriklb, did you do make start or make build , the time you removes all images and started from scratch ?
I started from scratch twice. That means no images and no containers at each start.
At the 1st start I typed make build
followed by make start.
Then I deleted every image and container again.
At the 2nd start I typed make start
only
Did as you wrote. No I have this in JTI logs:
2018-07-17 16:14:17 +0300 fluent.warn: {"next_retry":"2018-07-17 16:14:50 +0300","error_class":"InfluxDB::Error","error":"{\"error\":\"partial write: unable to parse 'juniperNetworks.cpu_memory_util_ext,utilization.name=Kernel,utilization.application_utilization.name=nh,device=vmx-17-4:192.168.1.252,_seq=9 utilization.size=2147479528i,utilization.bytes_allocated=163991448i,utilization.utilization=7i,utilization.application_utilization.bytes_allocated=18446744073704522012i,utilization.application_utilization.allocations=661817i,utilization.application_utilization.frees=993485i,utilization.application_utilization.allocations_failed=0i,host=\\"35bf8d8d8402\\",sensor_name=\\"cpu_memory_util_ext\\" 1531833225500': unable to parse integer 18446744073704522012: strconv.ParseInt: parsing \\"18446744073704522012\\": value out of range dropped=0\"}\n","plugin_id":"object:2ae8ab0094e8","message":"temporarily failed to flush the buffer. next_retry=2018-07-17 16:14:50 +0300 error_class=\"InfluxDB::Error\" error=\"{\\"error\\":\\"partial write: unable to parse 'juniperNetworks.cpu_memory_util_ext,utilization.name=Kernel,utilization.application_utilization.name=nh,device=vmx-17-4:192.168.1.252,_seq=9 utilization.size=2147479528i,utilization.bytes_allocated=163991448i,utilization.utilization=7i,utilization.application_utilization.bytes_allocated=18446744073704522012i,utilization.application_utilization.allocations=661817i,utilization.application_utilization.frees=993485i,utilization.application_utilization.allocations_failed=0i,host=\\\\"35bf8d8d8402\\\\",sensor_name=\\\\"cpu_memory_util_ext\\\\" 1531833225500': unable to parse integer 18446744073704522012: strconv.ParseInt: parsing \\\\"18446744073704522012\\\\": value out of range dropped=0\\"}\n\" plugin_id=\"object:2ae8ab0094e8\""}
From: henriklb Sent: Tuesday, July 17, 2018 3:33 PM To: Juniper/open-nti Cc: anaaman; Mention Subject: Re: [Juniper/open-nti] JTI cpu data not appearing in the InfluxDB(#227)
I started from scratch twice. That means no images and no containers at each start. At the 1st start I typed make build followed by make start. Then I deleted every image and container. At the 2nd start I typed make start only — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
@anaaman,@henriklb it would be possible to gain access to your setup ?
Hi, Sorry but this is impossible.
From: psagrera Sent: Tuesday, July 17, 2018 4:58 PM To: Juniper/open-nti Cc: anaaman; Mention Subject: Re: [Juniper/open-nti] JTI cpu data not appearing in the InfluxDB(#227)
@anaaman,@henriklb it would be possible to gain access to your setup ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
Not possible, sorry.
On Wed, 18 Jul 2018 at 10:11, anaaman notifications@github.com wrote:
Hi, Sorry but this is impossible.
From: psagrera Sent: Tuesday, July 17, 2018 4:58 PM To: Juniper/open-nti Cc: anaaman; Mention Subject: Re: [Juniper/open-nti] JTI cpu data not appearing in the InfluxDB(#227)
@anaaman,@henriklb it would be possible to gain access to your setup ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/227#issuecomment-405847827, or mute the thread https://github.com/notifications/unsubscribe-auth/AKgFQXjJPsbY_-32x52npSenSsq5b0Qfks5uHu3AgaJpZM4VHgjE .
-- Mvh Henrik
Hi @henriklb ,
it would be possible to decode data sent from MX80-MX104 for the sensor is not working ? Here there is a link where explains how to decode data using unix utilities
Regards
Hi! I have already done that actually, it worked fine for every .proto file on MX104 and MX80.
I also tested if the cpu/mem sensor worked with the collector Logstash (with a official plugin provided by Juniper). CPU/mem sensor worked fine in Logstash.
I know Juniper has a similar Plugin for Fluentd (https://github.com/Juniper/fluent-plugin-udp-native-sensors) but I haven't gotten around to test that one yet. If I understand correctly this FluentD-plugin provided by Juniper is (now) used in Open-NTI. I suspect the cpu/mem sensor parse error is either in the open-nti's fluentd-config-file. It could also be in the Juniper-plugin itself, but I somehow doubt that (haven't tested that yet tho)
Best Regards
Actually, openNTI is based on https://github.com/Juniper/fluent-plugin-udp-native-sensors/tree/with_keys.
Try to remove this part from fluentd config file
#################
## Output ###
#################
##### CPU #######
<match juniperNetworks.cpu_memory_util_ext>
type copy
{% if OUTPUT_STDOUT == 'true' %}
<store>
@type stdout
@id stdout_output
localtime
</store>
{% endif %}
{% if OUTPUT_INFLUXDB == 'true' %}
<store>
type influxdb
host "{{ INFLUXDB_ADDR }}"
port "{{ INFLUXDB_PORT }}"
dbname "{{ INFLUXDB_DB }}"
user "{{ INFLUXDB_USER }}"
password "{{ INFLUXDB_PWD }}"
time_precision ms
tag_keys ["device","utilization.application_utilization.name","utilization.name"]
tag_keys_field key_fields
####
buffer_type memory
# buffer_chunk_limit 524288 # 512 * 1024
# buffer_queue_limit 1024
flush_interval "{{ INFLUXDB_FLUSH_INTERVAL }}"
# retry_limit 17
# retry_wait 1.0
# num_threads 1
</store>
{% endif %}
## TODO investigate forest plugin https://github.com/tagomoris/fluent-plugin-forest
{% if OUTPUT_KAFKA == 'true' %}
<store>
@type kafka
# Brokers: you can choose either brokers or zookeeper.
brokers {{ KAFKA_ADDR }}:{{ KAFKA_PORT }}
# zookeeper <zookeeper_host>:<zookeeper_port> # Set brokers via Zookeeper
default_topic {{ KAFKA_TOPIC }}
# default_partition_key (string) :default => nil
output_data_type {{ KAFKA_DATA_TYPE }} # |ltsv|msgpack|attr:<record name>|<formatter
name>)
output_include_tag true # |false) :default => false
output_include_time true # |false) :default => false
# max_send_retries (integer) :default => 3
# required_acks (integer) :default => 0
# ack_timeout_ms (integer) :default => 1500
compression_codec {{ KAFKA_COMPRESSION_CODEC }} # (none|gzip|snappy) :default =>
none
</store>
{% endif %}
</match>
################ END CPU ################
it will match here
################## ALL #############
<match juniperNetworks.**>
type copy
{% if OUTPUT_STDOUT == 'true' %}
<store>
@type stdout
@id stdout_output
localtime
</store>
{% endif %}
if you could share your .gpb data I would really appreciate.
I removed the requested CPU part, restarted the container and verified fluentd is indeed running the modified fluentd.conf file. Now after deleting the CPU part in the config file NONE of the sensors are working. When I added the CPU part back again all sensors are working (except the CPU/mem one).
Now I'm more clueless than ever.
I verified the both the ~/fluentd/fluent.conf and the /tmp/fluent.conf files in the opennti_input_jti container.
It's really weird
Could you send me fluentd file and logs , with and without cpu sensor ?
What sensors are configured ? cpu and interface stats ?
In my setup , I'm removing that part :
2018-07-20 06:32:49 -0700 [warn]: rewrite_tag_filter: [DEPRECATED] Use
and measurememts are created
show MEASUREMENTS name: measurements name
juniperNetworks.cpu_memory_util_ext juniperNetworks.jnpr_interface_ext
and we got data :
select * from "juniperNetworks.cpu_memory_util_ext" order by desc limit 1 name: juniperNetworks.cpu_memory_util_ext time _seq device host sensor_name utilization.application_utilization.allocations utilization.application_utilization.allocations_failed utilization.application_utilization.bytes_allocated utilization.application_utilization.frees utilization.application_utilization.name utilization.bytes_allocated utilization.name utilization.size utilization.utilization
2018-07-20T13:36:32.208Z 9 mx1_re:10.235.1.2 0965f615e337 cpu_memory_util_ext 132 0 62136 3 nh 181156328 Kernel 536866792 33
BTW, you can turn on more verbosity in input-jti , changing that ENV variable in the docker-compose.yml file
How very odd. How do you start your open-nti system?
All I do is :
git clone https://github.com/Juniper/open-nti.git
cd open-nti
make start
Are you doing something different? I'm able to reproduce this error on a clean seperate machine as well, which makes me think that maybe you do not start up docker the same way?
I do:
git clone https://github.com/Juniper/open-nti.git cd open-nti make build make start
when you removed cpu part from fluentd.conf , what errors were you getting ?
or simply there wasn't measurement ?
Regards
Oh OK so I removed the CPU part of the fluent.conf again and now I see the data in influxdb correctly!
I only checked the CPU_JTI dashboard last time. As you know, the CPU dashboard is dependent on that the tags are defined. My bad.
So it is indeed related to that CPU part in the flunet.conf!
Ok, the current status is:
With cpu part in the fluent.conf it doesn't work (only one insertion into the database) Without cpu part in the fluent.conf "it works" (but only device is inserted as TAG because is hitting in the default part)
Could you please try with the following CPU config (only device in tag_keys)?
<match juniperNetworks.cpu_memory_util_ext>
type copy
{% if OUTPUT_STDOUT == 'true' %}
<store>
@type stdout
@id stdout_output
localtime
</store>
{% endif %}
{% if OUTPUT_INFLUXDB == 'true' %}
<store>
type influxdb
host "{{ INFLUXDB_ADDR }}"
port "{{ INFLUXDB_PORT }}"
dbname "{{ INFLUXDB_DB }}"
user "{{ INFLUXDB_USER }}"
password "{{ INFLUXDB_PWD }}"
time_precision ms
tag_keys ["device"]
tag_keys_field key_fields
####
buffer_type memory
# buffer_chunk_limit 524288 # 512 * 1024
# buffer_queue_limit 1024
flush_interval "{{ INFLUXDB_FLUSH_INTERVAL }}"
# retry_limit 17
# retry_wait 1.0
# num_threads 1
</store>
{% endif %}
{% if OUTPUT_KAFKA == 'true' %}
<store>
@type kafka
# Brokers: you can choose either brokers or zookeeper.
brokers {{ KAFKA_ADDR }}:{{ KAFKA_PORT }}
# zookeeper <zookeeper_host>:<zookeeper_port> # Set brokers via Zookeeper
default_topic {{ KAFKA_TOPIC }}
# default_partition_key (string) :default => nil
output_data_type {{ KAFKA_DATA_TYPE }} # |ltsv|msgpack|attr:<record name>|<formatter
name>)
output_include_tag true # |false) :default => false
output_include_time true # |false) :default => false
# max_send_retries (integer) :default => 3
# required_acks (integer) :default => 0
# ack_timeout_ms (integer) :default => 1500
compression_codec {{ KAFKA_COMPRESSION_CODEC }} # (none|gzip|snappy) :default =>
none
</store>
{% endif %}
@henriklb
What's the current status ?
Regards
Pablo
Oh OK so I removed the CPU part of the fluent.conf again and now I see the data in influxdb correctly!
I only checked the CPU_JTI dashboard last time. As you know, the CPU dashboard is dependent on that the tags are defined. My bad.
So it is indeed related to that CPU part in the flunet.conf!
I'm not able to reproduce this. So the current status is that CPU data still does not work. It inserts data at startup once, then stops. So the CPU issue still stands. Also I do not have access to the Juniper equipment anymore. so I wont' be able to assist further.
Where you able to test it with an MX80 or MX104?
Hi,
Unfortunately , I couldn’t get an mx80/104 for testing. I will try to get one in the upcoming weeks to see if I can reproduce the issue. In the meantime I will close that issue.
Regards
Pablo
Enviado desde mi iPhone
El 14 sept 2018, a las 13:13, henriklb notifications@github.com escribió:
Oh OK so I removed the CPU part of the fluent.conf again and now I see the data in influxdb correctly!
I only checked the CPU_JTI dashboard last time. As you know, the CPU dashboard is dependent on that the tags are defined. My bad.
So it is indeed related to that CPU part in the flunet.conf! I'm not able to reproduce this. So the current status is that CPU data still does not work. It inserts data at startup once, then stops. So the CPU issue still stands. Also I do not have access to the Juniper equipment anymore. so I wont' be able to assist further.
Where you able to test it with an MX80 or MX104?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
Also get these errors, can't parse a tag or one of the values. Using MX480 Junos 17.3 R3.10
unable to parse 'juniperNetworks.cpu_memory_util_ext,utilization.name=Kernel,utilization.application_utilization.name=nh,device=DEVICEX,_seq=9 utilization.size=3275747304i,utilization.bytes_allocated=619012272i,utilization.utilization=18i,utilization.application_utilization.bytes_allocated=18446744073703923088i,utilization.application_utilization.allocations=3782591i,utilization.application_utilization.frees=4302452i,utilization.application_utilization.allocations_failed=0i,host=\\\\"someserver.server.com\\\\",sensor_name=\\\\"cpu_memory_util_ext\\\\" 1550036800163':
Could this be due to https://github.com/influxdata/influxdb/issues/5204 ?
I'm trying to stream the
/junos/system/linecard/cpu/memory/
sensor with JTI. The problem is that I do not see any cpu data from the sensor in the influx database.I'm running the latest open-nti with the new fluent JTI parser/plugin from @psagrera. Running junos 17.3R2 tested with both MX80 and MX104.
Using tcpdump and protoc I see the packets are being sent correctly onto port 50000.
/junos/system/linecard/interface/ works just fine.
Any idea what could be wrong?
This is my Junos config