sipcapture / heplify-server

HEP Capture Server for HOMER
https://sipcapture.org
GNU Affero General Public License v3.0
183 stars 85 forks source link

Homer displays empty SIP CSeq column #328

Closed systemcrash closed 4 years ago

systemcrash commented 4 years ago

Docker pulled in heplify-server 1.19.

I assume the problem lies in heplify-server (and not in the homer7 webapp which displays the data)

INVITE (and other) packet shows:

CSeq: 1 INVITE

SIP CSeq column:

empty

Same thing for

SIP From Tag column

systemcrash commented 4 years ago

Homer is HOMER 7.7.031 (prod)

negbie commented 4 years ago

@systemcrash this is because from tag isn't included in the default data_header. Since you use docker you can add this env var to the heplify-server container:

I just see that I currently have no cseq in my template code. Let me include the cseq for version 1.20

systemcrash commented 4 years ago

Good catch. Those fields are only filled and propagated when new packets ingress?

I restarted with this:

      - "HEPLIFYSERVER_SIPHEADER=callid,method,ruri_user,ruri_domain,from_user,from_domain,from_tag,to_user,to_domain,to_tag,contact_user,cseq"

in the compose file and still receive empty columns for existing material. :

2020/03/05 15:06:43.836348 server.go:83: INFO start heplify-server 1.19 with config.HeplifyServer{HEPAddr:"0.0.0.0:9060", HEPTCPAddr:"", HEPTLSAddr:"0.0.0.0:9060", ESAddr:"", ESDiscovery:true, ESUser:"", ESPass:"", LokiURL:"http://xxx.xxx.xx:3100/api/prom/push", LokiBulk:400, LokiTimer:2, LokiBuffer:100000, LokiHEPFilter:[]int{1, 5, 100}, ForceHEPPayload:[]int(nil), PromAddr:"0.0.0.0:9096", PromTargetIP:"", PromTargetName:"", DBShema:"homer7", DBDriver:"postgres", DBAddr:"172.16.16.10:5432", DBUser:"root", DBPass:"homerSeven", DBDataTable:"homer_data", DBConfTable:"homer_config", DBBulk:400, DBTimer:4, DBBuffer:400000, DBWorker:8, DBRotate:true, DBPartLog:"2h", DBPartIsup:"6h", DBPartSip:"2h", DBPartQos:"6h", DBDropDays:5, DBDropDaysCall:0, DBDropDaysRegister:0, DBDropDaysDefault:0, DBDropOnStart:false, Dedup:false, DiscardMethod:[]string(nil), AlegIDs:[]string(nil), CustomHeader:[]string(nil), SIPHeader:[]string{"ruri_user", "ruri_domain", "from_user", "from_domain", "to_user", "callid", "method", "user_agent"}, LogDbg:"", LogLvl:"info", LogStd:true, LogSys:false, Config:"./heplify-server.toml", ConfigHTTPAddr:"", ConfigHTTPPW:"", Version:false}
negbie commented 4 years ago

Hmm your config:

heplify-server.log "ruri_user", "ruri_domain", "from_user", "from_domain", "to_user", "callid", "method", "user_agent"

Should be the same like the config. Anyway. I pushed some commits which do not require any configuration. Just wait when 1.21 is on dockerhub.

negbie commented 4 years ago

Check latest image from master.

systemcrash commented 4 years ago

Pulled in the lastest :master tag, restarted. Don't see those columns populated.

2020/03/06 00:10:44.958683 server.go:83: INFO start heplify-server 1.22 with config.HeplifyServer{HEPAddr:"0.0.0.0:9060", HEPTCPAddr:"", HEPTLSAddr:"0.0.0.0:9060", ESAddr:"", ESDiscovery:true, ESUser:"", ESPass:"", LokiURL:"http://xxx.xxx.xx:3100/api/prom/push", LokiBulk:400, LokiTimer:2, LokiBuffer:100000, LokiHEPFilter:[]int{1, 5, 100}, ForceHEPPayload:[]int(nil), PromAddr:"0.0.0.0:9096", PromTargetIP:"", PromTargetName:"", DBShema:"homer7", DBDriver:"postgres", DBAddr:"172.16.16.10:5432", DBUser:"root", DBPass:"homerSeven", DBDataTable:"homer_data", DBConfTable:"homer_config", DBBulk:400, DBTimer:4, DBBuffer:400000, DBWorker:8, DBRotate:true, DBPartLog:"2h", DBPartIsup:"6h", DBPartSip:"2h", DBPartQos:"6h", DBDropDays:5, DBDropDaysCall:0, DBDropDaysRegister:0, DBDropDaysDefault:0, DBDropOnStart:false, Dedup:false, DiscardMethod:[]string(nil), AlegIDs:[]string(nil), CustomHeader:[]string(nil), SIPHeader:[]string{"callid", "method", "ruri_user", "ruri_domain", "from_user", "from_domain", "from_tag", "to_user", "to_domain", "to_tag", "contact_user", "cseq"}, LogDbg:"", LogLvl:"info", LogStd:true, LogSys:false, Config:"./heplify-server.toml", ConfigHTTPAddr:"", ConfigHTTPPW:"", Version:false}

See lots of rows like e.g.


2020/03/05 23:57:53.129032 rotator.go:290: DBG [rotator] db query:
CREATE INDEX IF NOT EXISTS hep_proto_1_default_20200306_1400_method ON hep_proto_1_default_20200306_1400 ((data_header->>'method'));

but don't seem to see anything in the debug logs for data_header->>'cseq'

Normal? Are DB rows created for the cseq and to_tag fields?

negbie commented 4 years ago

Hmm strange. data_header is json so you don't need to create new columns for new header fields. SIPHeader looks ok aswell. Keep in mind this will only be for new data not old one. Please set -logdbg sql and check whats being sent to postgres.