Juniper / open-nti

Open Network Telemetry Collector build with open source tools
Apache License 2.0
232 stars 93 forks source link

Influx db to persist / hold state? #233

Closed bodleytunes closed 6 years ago

bodleytunes commented 6 years ago

What is the correct way to run the containers so that influx stores the data after restarting the containers?

I tried make persistent and then make start and it's not holding data after a restart of all the containers.

Do you have to modify any of the files to point to an external volume?

Also when I run make start persistent, only a few of the containers start up (ie not as many as running make start), is that expected?

Regards,

Jon.

psagrera commented 6 years ago

Hi Jon

when you start openNTI in persistence mode , i.e , make start-persistent, a new directory called db is created

root@ubuntu:~/open-nti# tree db/influxdb/ db/influxdb/ |-- meta | -- meta.db -- wal |-- docker_internal | -- autogen |-- 3 | -- _00001.wal |-- _internal |-- monitor | -- 2 |-- _00001.wal |-- juniper | -- four_weeks |-- 1 | -- _00003.wal -- snmp -- autogen -- 4 `-- _00001.wal

All data is stored there, so if you want to stop the containers and hold all the data, you will have to start again in persistency mode (i.e make start-persistent )

You are right, in persistency mode kapacitor/chronograf and snmp was missing

I've committed that to fix it: https://github.com/Juniper/open-nti/commit/b7de0df3551f0bfe1a5ba2d45c20ac765e57568f

Regards

bodleytunes commented 6 years ago

Ok thanks I'll try that again

Cheers!

Jon.

bodleytunes commented 6 years ago

OK Works a treat now!

One last quick question, it doesn't seem to remember my dashboards in Grafana.

I added the json export files to the ./dashboards folder

image

But it doesn't seem to have them in the list after a stop/start-persistent?

Is there something else that needs to be done to make them persist?

Cheers, Jon.

psagrera commented 6 years ago

Hi Jon,

I've committed that:

https://github.com/Juniper/open-nti/commit/5ae665076b3de959e79b55fe8ede8addab431582

With this commit all dashboards under ./dashboards folder should be present (bear in mind that dashboards provisioning automatically cannot be directly edited).

Read

Making changes to a provisioned dashboard section from:

http://docs.grafana.org/administration/provisioning/

Pull latest version and then execute make build-main + make start

Regards.

bodleytunes commented 6 years ago

Cheers! Jon.

On Tue, 7 Aug 2018 at 18:18 psagrera notifications@github.com wrote:

Hi Jon,

I've committed that:

5ae6650 https://github.com/Juniper/open-nti/commit/5ae665076b3de959e79b55fe8ede8addab431582

With this commit all dashboards under ./dashboards folder should be present (bear in mind that dashboards provisioning automatically cannot be directly edited).

Read

Making changes to a provisioned dashboard section from:

http://docs.grafana.org/administration/provisioning/

Pull latest version and then execute make build-main + make start

Regards.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/233#issuecomment-411134212, or mute the thread https://github.com/notifications/unsubscribe-auth/AAly54_nzgvXLkXbuamAcL3rQI0J4cdjks5uOcvqgaJpZM4Vx3R9 .

bodleytunes commented 6 years ago

ok a ton of new dashboards have a appeared but its not showing mine in the list? :(

I have to manually import it and paste in the json again. stop then make-persistent, and nothing just the big list.

image

Is there something wrong with the way I'm exporting the panel which is stopping it from being imported.?

psagrera commented 6 years ago

That's what I've done and it works fine:

1) add json files inside dashboards directory

root@ubuntu:~/open-nti/dashboards# vi test.json

"Copy&Paste"

2) Stop containers root@ubuntu:~/open-nti# make stop

3) Start containers

root@ubuntu:~/open-nti# make start-persistent

and the new one is there

screen shot 2018-08-08 at 15 30 46

Regards

psagrera commented 6 years ago

BTW, you can remove from dashboard directory all that you don't need :)

bodleytunes commented 6 years ago

OK I'll have another go

On Wed, 8 Aug 2018 at 14:35 psagrera notifications@github.com wrote:

BTW, you can remove from dashboard directory all that you don't need :)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Juniper/open-nti/issues/233#issuecomment-411407437, or mute the thread https://github.com/notifications/unsubscribe-auth/AAly5_ofrFdEspVMITAMz-A4EhmWbPEzks5uOuksgaJpZM4Vx3R9 .