sipcapture / homer7-docker

HOMER 7 Docker Images
92 stars 86 forks source link

Purging data on a regular basis? #85

Closed lukeescude closed 4 years ago

lukeescude commented 4 years ago

Hello, is it possible to set up some kind of weekly/nightly purge?

I had 100% of my host's hard disk fill up with SIP packets. I figured homer7 would have had some kind of cleanup capabilities built-in.

adubovikov commented 4 years ago

Here you are: https://github.com/sipcapture/heplify-server/blob/master/example/homer7_config/heplify-server.toml#L31-L35

adubovikov commented 4 years ago

the hepliy-server will run an internal crontjob that will drop your old partition

lukeescude commented 4 years ago

Hmm I have added the the following to the docker-compose file:

  - "HEPLIFYSERVER_DBDROPDAYS=3"
  - "HEPLIFYSERVER_DISCARDMETHOD=[OPTIONS]"

And it does not appear to be working after restarting - OPTIONS packets still show up. Is there a particular location this config needs to go?

lmangani commented 4 years ago

try adding this:

DBDropOnStart         = true
lukeescude commented 4 years ago

I will give that a try!

Out of curiosity, whenever Homer restarts, most of my Kamailio proxies must be restarted in order to start sending HEP again... it’s like kamailio doesn’t automatically reconnect to Homer. Is this an issue on the kamailio side or homer side?

It has happened with Kamailio 4.x and 5.x with Homer 5.x and 7.x all combinations.

adubovikov commented 4 years ago

no, it should send traffic without restart, this is UDP and kamailio uses"sendto" function that doesn't require connect

lukeescude commented 4 years ago

Strange, I wonder why I must restart kamailio then... I will experiment with that and see if I can find out why.

lukeescude commented 4 years ago

The HDD is still filling up - I have data lasting 7+ days even though I've set HEPLIFYSERVER_DBDROPDAYS=3 in the configuration.

Is there a different place to put that, besides the docker-compose file?

lmangani commented 4 years ago

Could you confirm the heplify-server version from within the container please?

lukeescude commented 4 years ago

Hmmm, how do I do that?

lmangani commented 4 years ago

try pulling the latest container and see if it updates. docker pull sipcapture/heplify-server:master

lukeescude commented 4 years ago

Will do - I will perform this task later in the day, since I do not want to interrupt homer during the work day.

Thank you!

lukeescude commented 4 years ago

Hello, it does not appear like the new image helps at all - HDD still at 75% utilization, same as before the new image went up with docker-compose up -d

Any other tests?

lukeescude commented 4 years ago

It is interesting... Every time I restart the heplify-server, Grafana logs a large influx of packets.

See attached.

Screen Shot 2020-07-28 at 11 03 15 AM
lukeescude commented 4 years ago

Interestingly, having the following doesn't function either:

Heplify still continues to log OPTIONS packets instead of discarding them. At least, they show up in Grafana.

adubovikov commented 4 years ago

Granfa makes stats, but this option doesn't write OPTIONS message into PostgressDB

lukeescude commented 4 years ago

In that case, these options might be working perfectly fine.

I will let it go for a few days and report back if necessary.

lukeescude commented 4 years ago

Well, I stand corrected - There are still OPTIONS packets showing up in the Homer search screens.

I believe the docker-compose file options are being ignored when heplify boots up.