Non-deterministic DNS DGA alerts

[simonemainardi]

Different machines yields different results for DGA-domain flow risk.

Domains reported in the alerts are different from machine to machine.

A reason could be that all the packets of a DNS-flow are processed by DPI and this causes last_query to change without syncronization with reference to the thread that generates the alert.

[simonemainardi]

e.g.

[!]  Test Failure: Unexpected output from the test 'alert_flow_risk_dga_01'. Please check conflicts/alert_flow_risk_dga_01.out
        "description": "Suspicious DGA Domain [7cd501a621c362 |         "description": "Suspicious DGA Domain [317301e6da8351
          "active_url": "/lua/flow_details.lua?flow_key=32995 |           "active_url": "/lua/flow_details.lua?flow_key=32995
          "cli_port": "35966",                    |           "cli_port": "46961",
          "description": "Suspicious DGA Domain [7cd501a621c3 |           "description": "Suspicious DGA Domain [317301e6da83
        "row_id": "1",                        |         "row_id": "2",
        "description": "Suspicious DGA Domain [b54101e6da8351 |         "description": "Suspicious DGA Domain [b11c01a621c362
          "active_url": "/lua/flow_details.lua?flow_key=32995 |           "active_url": "/lua/flow_details.lua?flow_key=32995
          "cli_port": "46961",                    |           "cli_port": "35966",
          "description": "Suspicious DGA Domain [b54101e6da83 |           "description": "Suspicious DGA Domain [b11c01a621c3
        "row_id": "2",                        |         "row_id": "1",

[simonemainardi]

I can confirm this is due to the changing DNS query.

What happens is the following:

1. a DGA-query is set: 03/Jun/2021 14:49:50 [Flow.cpp:778] [DNS] 7cd501a621c362010a.skullseclabs.org
2. protocolDetected callbacks are executed: 03/Jun/2021 14:49:50 [Flow.cpp:2930] Protocol detected
3. Another query is detected 03/Jun/2021 14:49:50 [Flow.cpp:778] [DNS] b11c01a621c362010a.skullseclabs.org
4. The DGA-notification is generated using query at 3. rather than at 1.

[simonemainardi]

To address this, we can decide to only keep the first DNS query when multiple queries are seen over the same DNS flow. This is the behavior currently used for HTTP. Please comment. Full discussion at https://discord.com/channels/726054161547919440/726060121905561671/849997482808573962

[simonemainardi]

Issue has been fixed as follows:

1. The DNS query is always updated to keep the most recent for every flow. However, for the sake of simplicity and to avoid complex or costly synchronization mechanisms, we have chosen to update the DNS query at most one time per second ( 4ccca54)
2. For flows with the DGA-domain risk, the DGA query is chosen to be the one at the time the callback protocolDetected is called on the flow. This is kept constant and won't change during the lifetime of the flow, ensuring a deterministic behavior (c066e62).