search

Juniper / open-nti

Open Network Telemetry Collector build with open source tools
Apache License 2.0
233 stars 93 forks source link

Make start-persistent doesn't keep data #154

Closed zhanghaofeng closed 7 years ago

zhanghaofeng commented 7 years ago

I am using make start-persistent mode according to docs: By default it will start 3 containers and it's working in non-persistent mode, once you stop it all data are gone. It's possible to start the main container in persistent mode to save the database outside the container, b y using the startup script make start-persistent. Persistent mode on Mac OS requires at least v1.12

But even this, after I "make stop-persistent", "make-start-persisent" again, all the data in inFluxdb is gone. How can I keep my data?

zhanghaofeng commented 7 years ago

need to docker commit before stop the containers. After that, data stats are kept.

3fr61n commented 7 years ago

Hi

One question, with data do you mean influxdb data or grafana data?

Because, so far, the persistent mode consist of include a volume to map only influxdb database directory to a host directory outside of container.

If you need persistency on grafana I suggest to save you dashboard in json format in the /dashboard directory outside the container

Let me know if this works for you

Regards

Sent from my iPhone

On 18 May 2017, at 03:14, Haofeng Zhang notifications@github.com wrote:

need to docker commit before stop the containers. After that, data stats are kept.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

zhanghaofeng commented 7 years ago

I mean influxdb data. The problem is, if I only use “make start-persistent”, database data will be still lost after I restart dockers. The workaround is to run “docker commit” before restarting dockers. Thus I can keep the data.

Is it expected behavior?

From: Efrain [mailto:notifications@github.com] Sent: Thursday, May 18, 2017 2:07 PM To: Juniper/open-nti open-nti@noreply.github.com Cc: Haofeng Zhang hfzhang@juniper.net; State change state_change@noreply.github.com Subject: Re: [Juniper/open-nti] Make start-persistent doesn't keep data (#154)

Hi

One question, with data do you mean influxdb data or grafana data?

Because, so far, the persistent mode consist of include a volume to map only influxdb database directory to a host directory outside of container.

If you need persistency on grafana I suggest to save you dashboard in json format in the /dashboard directory outside the container

Let me know if this works for you

Regards

Sent from my iPhone

On 18 May 2017, at 03:14, Haofeng Zhang notifications@github.com<mailto:notifications@github.com> wrote:

need to docker commit before stop the containers. After that, data stats are kept.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHubhttps://github.com/Juniper/open-nti/issues/154#issuecomment-302309928, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AF27-gJbNPzTFRX47AdnKupzc9hhpNQ5ks5r6-AagaJpZM4NenEE.

3fr61n commented 7 years ago

Let me do some testing

Sent from my iPhone

On 18 May 2017, at 08:14, Haofeng Zhang notifications@github.com wrote:

I mean influxdb data. The problem is, if I only use “make start-persistent”, database data will be still lost after I restart dockers. The workaround is to run “docker commit” before restarting dockers. Thus I can keep the data.

Is it expected behavior?

From: Efrain [mailto:notifications@github.com] Sent: Thursday, May 18, 2017 2:07 PM To: Juniper/open-nti open-nti@noreply.github.com Cc: Haofeng Zhang hfzhang@juniper.net; State change state_change@noreply.github.com Subject: Re: [Juniper/open-nti] Make start-persistent doesn't keep data (#154)

Hi

One question, with data do you mean influxdb data or grafana data?

Because, so far, the persistent mode consist of include a volume to map only influxdb database directory to a host directory outside of container.

If you need persistency on grafana I suggest to save you dashboard in json format in the /dashboard directory outside the container

Let me know if this works for you

Regards

Sent from my iPhone

On 18 May 2017, at 03:14, Haofeng Zhang notifications@github.com<mailto:notifications@github.com> wrote:

need to docker commit before stop the containers. After that, data stats are kept.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHubhttps://github.com/Juniper/open-nti/issues/154#issuecomment-302309928, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AF27-gJbNPzTFRX47AdnKupzc9hhpNQ5ks5r6-AagaJpZM4NenEE. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

zhanghaofeng commented 7 years ago

Thanks for looking into this. Just FYI, the first run of “make start-persistent” will give out the following errors:

root@chinas:/opt/open-nti# make start-persistent Use docker compose file: docker-compose/opennti_persistent.yml IMAGE_TAG=latest docker-compose -f docker-compose/opennti_persistent.yml up -d ERROR: build path /opt/open-nti/docker-compose/plugins/input-snmp either does not exist, is not accessible, or is not a valid URL. Makefile:79: recipe for target 'start-persistent' failed make: *** [start-persistent] Error 1 root@chinas:

I have to copy the plugin directory to open-nti/docker-compose/ to make it work.

From: Efrain [mailto:notifications@github.com] Sent: Thursday, May 18, 2017 2:44 PM To: Juniper/open-nti open-nti@noreply.github.com Cc: Haofeng Zhang hfzhang@juniper.net; State change state_change@noreply.github.com Subject: Re: [Juniper/open-nti] Make start-persistent doesn't keep data (#154)

Let me do some testing

Sent from my iPhone

On 18 May 2017, at 08:14, Haofeng Zhang notifications@github.com<mailto:notifications@github.com> wrote:

I mean influxdb data. The problem is, if I only use “make start-persistent”, database data will be still lost after I restart dockers. The workaround is to run “docker commit” before restarting dockers. Thus I can keep the data.

Is it expected behavior?

From: Efrain [mailto:notifications@github.com] Sent: Thursday, May 18, 2017 2:07 PM To: Juniper/open-nti open-nti@noreply.github.com<mailto:open-nti@noreply.github.com> Cc: Haofeng Zhang hfzhang@juniper.net<mailto:hfzhang@juniper.net>; State change state_change@noreply.github.com<mailto:state_change@noreply.github.com> Subject: Re: [Juniper/open-nti] Make start-persistent doesn't keep data (#154)

Hi

One question, with data do you mean influxdb data or grafana data?

Because, so far, the persistent mode consist of include a volume to map only influxdb database directory to a host directory outside of container.

If you need persistency on grafana I suggest to save you dashboard in json format in the /dashboard directory outside the container

Let me know if this works for you

Regards

Sent from my iPhone

On 18 May 2017, at 03:14, Haofeng Zhang notifications@github.com<mailto:notifications@github.com<mailto:notifications@github.com%3cmailto:notifications@github.com>> wrote:

need to docker commit before stop the containers. After that, data stats are kept.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHubhttps://github.com/Juniper/open-nti/issues/154#issuecomment-302309928, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AF27-gJbNPzTFRX47AdnKupzc9hhpNQ5ks5r6-AagaJpZM4NenEE. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHubhttps://github.com/Juniper/open-nti/issues/154#issuecomment-302315575, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AF27-i8GAvp7IIQPaTUvHcAa1YAit6z3ks5r6-iUgaJpZM4NenEE.

3fr61n commented 7 years ago

Regarding the input-snmp issue I committed #fe154b0

zhanghaofeng commented 7 years ago

Great. But I never made this input-snmp work ☹ Maybe we can do a live troubleshooting together? I can demo the problems in this module.

From: Efrain [mailto:notifications@github.com] Sent: Thursday, May 18, 2017 6:26 PM To: Juniper/open-nti open-nti@noreply.github.com Cc: Haofeng Zhang hfzhang@juniper.net; State change state_change@noreply.github.com Subject: Re: [Juniper/open-nti] Make start-persistent doesn't keep data (#154)

Regarding the input-snmp issue I committed #fe154b0

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHubhttps://github.com/Juniper/open-nti/issues/154#issuecomment-302364887, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AF27-jmkr8G-iFK21gWhDvTTIwMRF93Bks5r7BylgaJpZM4NenEE.

3fr61n commented 7 years ago

Did you have the chance to take a look for http://open-nti.readthedocs.io/en/latest/snmpcollectionagent.html

zhanghaofeng commented 7 years ago

Yes. Several times. I see 2 issues here:

  1. “ADD templates /data/templates/” in input-snmp/Dockerfile doesn’t take effect. No matter how I change the template file in input-snmp/template/telegraf.tmp, the change won’t reflect to the opennti-input-snmp container

  2. I find a way to workaround above problem. But the SNMP polling result cannot go to influxdb database.

From: Efrain [mailto:notifications@github.com] Sent: Thursday, May 18, 2017 8:04 PM To: Juniper/open-nti open-nti@noreply.github.com Cc: Haofeng Zhang hfzhang@juniper.net; State change state_change@noreply.github.com Subject: Re: [Juniper/open-nti] Make start-persistent doesn't keep data (#154)

Did you have the chance to take a look for http://open-nti.readthedocs.io/en/latest/snmpcollectionagent.html

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHubhttps://github.com/Juniper/open-nti/issues/154#issuecomment-302384434, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AF27-mVbaoCAzD3ivLjeysIoeyNwFqthks5r7DO3gaJpZM4NenEE.