heplify-server to elasticsearch index with YYYYMMDD

[JonConnor]

currently heplify-server posts the sip messages into elasticsearch with index 'heplify-server'. Could this be enhanced to use the index with a date stamp? This will allow to clean up old data records in elasticsearch based on a retention policy i define.

eg. heplify-server-YYYYMMDD.

[negbie]

Sure will look into it tomorrow.

[negbie]

Implemented. Indexes will look like heplify-server-2018-07-03. On start and every 12 hours I will create an index for the same and the next 2 days. If they are already created you will find this in the log:

2018/07/03 19:47:17.249524 elasticsearch.go:108: INFO index heplify-server-2018-07-03 already created 2018/07/03 19:47:17.250259 elasticsearch.go:108: INFO index heplify-server-2018-07-04 already created 2018/07/03 19:47:17.252697 elasticsearch.go:108: INFO index heplify-server-2018-07-05 already created

[negbie]

New release online https://github.com/sipcapture/heplify-server/releases

Please test and report. Thanks!

[JonConnor]

Thanks, I should have mentioned I'm using the docker setup. Are you planning to push it there as well?

[negbie]

Yes. Clone this repo again or change the image path inside the compose file to image: sipcapture/heplify-server:latest

I've created an automated build for the sipcapture repo on dockerhub.

[negbie]

Keep in mind that when you use promtargets you maybe need to adjust some grafana dashboards because I introduced a new tag "direction". For example change:

rate(heplify_method_response{target_name=~"$tn", method="INVITE", response="200"}[1m]) to: sum without(direction)(rate(heplify_method_response{target_name=~"$tn", method="INVITE", response="200"}[1m]))

when you don't need the splitting by direction.

[JonConnor]

thanks looks good. Will let it run a few days and then close the ticket if no issues seen.

Thanks!

[JonConnor]

looking closer, it appears heplify_server is now sending non-SIP to elastic search as well. I have heplify-clients running on different sip servers and sending anything from 5060-9000 to heplify-server. After the update, non-SIP messages (eg SNMPTraps that are on port 8162) are being sent from heplify-server to Elasticsearch.

[lmangani]

@negbie should I prepare a paStash recipe to dispatch heplify-server pushouts? seems like the most viable and flexible way for this kind of request

[negbie]

@lmangani if you like so. It could be useful if some people want to use paStash's filter capabilites. What kind of input for paStash would you prefer heplify-server should send to? input_tcp.js, input_udp.js, input_unix.js, input_ws.js. I think tls over tcp would be the best.

[negbie]

@JonConnor will look into it this evening.

[lmangani]

@negbie whats the most convenient format for heplify to send it low cost? I can receive anything, including queues, etc.

[negbie]

@lmangani I guess in that scenario the most would use it besides heplify-server and in that case plain udp piped to a specified localhost port would be the fastest option.

[negbie]

@lmangani I thought about this and I would go for tls over tcp. I would like to do it this way so we can securely pipe the traffic over the network to paStash or mby another heplify-server which is not on the same host. With this approach you can securely chain multiple heplify-server together. I will add two flags PipeAddr and PipeFilter. PipeAddr is simply the pipe endpoint address. With PipeFilter you can filter your traffic by any string and pipe only on match. Like special nodes, or some special customers and so on.

[negbie]

@JonConnor latest docker image should fix this.

[JonConnor]

Looks good, will close this out in a few days if no other issue seen. Thanks again!

[negbie]

Seems to be working fine so I close this now ;)