sipcapture / homer

HOMER - 100% Open-Source SIP, VoIP, RTC Packet Capture & Monitoring
https://sipcapture.org
GNU Affero General Public License v3.0
1.58k stars 239 forks source link

DB Error - Homer7 from "Homer-Installer" #597

Open idanam-del opened 1 year ago

idanam-del commented 1 year ago

hey, i run the homer stack of postgres 13, heplify-server and homer-app provided by the homer installer script.

I have an issue when i search in the UI by the parameter captureId, it doesnt show anything. When i does it here's the following error from the postgres log. Weird thing it worked before and it somehow broke in the last few days.

STATEMENT:  SELECT * FROM "hep_proto_1_call"  WHERE (create_date between $1 AND $2 AND ( (protocol_header->>'captureId')::int = $3 )) LIMIT 2000
 ERROR:  could not map dynamic shared memory segment
 ERROR:  could not map dynamic shared memory segment
LOG:  background worker "parallel worker" (PID 262853) exited with exit code 1
LOG:  background worker "parallel worker" (PID 262854) exited with exit code 1

Here are the DB settings in the heplify-server.toml file. From my google searches its seems to indicate a performance issue so what should i tweak in the DB settings?

DBBulk                = 200
DBTimer               = 4
DBBuffer              = 400000
DBWorker              = 8
DBRotate              = true

Another Error when searching by source ip and destination ip

 ERROR:  syntax error at or near "AND" at character 79
STATEMENT:  SELECT * FROM "hep_proto_1_call"  WHERE (create_date between $1 AND $2 AND (  AND protocol_header->>'dstIp'= $3 )) LIMIT 2000
ERROR:  syntax error at or near "AND" at character 79
 STATEMENT:  SELECT * FROM "hep_proto_1_call"  WHERE (create_date between $1 AND $2 AND (  AND protocol_header->>'srcIp'= $3 )) LIMIT 2000

Thank you

adubovikov commented 1 year ago

looks like the field has been ignored, can you show the API request ? (F12 -> Network)

idanam-del commented 1 year ago

@adubovikov

I've resolved the above issue by setting the capture id parameter to 'string' instead of 'integer' and it seems to go away for now.

But something else evolved and now you cannot search the 'x-uuid' by itself or you will face the same problem - Of not showing any correlated results.

The Postgres log shows a different error this time:


2023-06-22 12:13:28.710 UTC [219616] homer_user@homer_data ERROR:  syntax error at or near "AND" at character 79

2023-06-22 12:13:28.710 UTC [219616] homer_user@homer_data STATEMENT:  SELECT * FROM "hep_proto_1_call"  WHERE (create_date between $1 AND $2 AND (  AND data_header->>'X-uuid'= $3 )) LIMIT 2000

The API request goes through and doesn't comeback with any errors just this:

{"data":[],"keys":[],"total":0}

adubovikov commented 1 year ago

@idanam-del I would like to see the API request, not the response

idanam-del commented 1 year ago

@adubovikov Oh sure. Here it is:

curl 'https://HOMER-IP/api/v3/search/call/data' \
  -H 'authority: HOMER-IP' \
  -H 'accept: application/json, text/plain, */*' \
  -H 'accept-language: en-GB,en-US;q=0.9,en;q=0.8' \
  -H 'authorization: Bearer  XXXX' \
  -H 'content-type: application/json' \
  -H 'cookie: grafana_session=52a18d0b7b275c950e2ad9ca83e74373' \
  -H 'origin: https://HOMER-IP' \
  -H 'referer: https://HOMER-IP/dashboard/home' \
  -H 'sec-ch-ua: "Not.A/Brand";v="8", "Chromium";v="114", "Google Chrome";v="114"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "Windows"' \
  -H 'sec-fetch-dest: empty' \
  -H 'sec-fetch-mode: cors' \
  -H 'sec-fetch-site: same-origin' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36' \
  --data-raw '{"config":{"protocol_id":{"name":"SIP","value":1},"protocol_profile":{"name":"call","value":"call"},"searchbutton":false,"title":"CALL 2 SIP SEARCH"},"param":{"transaction":{},"limit":200,"orlogic":false,"search":{"1_call":[{"name":"limit","value":"2000","func":null,"type":"integer","hepid":1},{"name":"data_header.X-uuid","value":"feac4033-308c-4f12-aa45-3eac78e3a7c7","func":null,"type":"string","hepid":1}]},"location":{},"timezone":{"value":-180,"name":"Local"}},"timestamp":{"from":1687676190000,"to":1687676490000}}' \
  --compressed \
  --insecure