nDPId: incomplete "flow_event_schema.json" schema definition

[verzulli]

During analysis of the UDP-stream generated by nDPId (as for ref), I got following JSON, related to an HTTPS request:

  {
    "flow_event_id": 7,
    "flow_event_name": "detection-update",
    "flow_id": 54994,
    "flow_state": "info",
    "flow_packets_processed": 6,
    [...]
    "l3_proto": "ip4",
    "src_ip": "192.168.0.128",
    "dst_ip": "***.***.***.***",
    "src_port": 45396,
    "dst_port": 443,
    "l4_proto": "tcp",
    "ndpi": {
      "flow_risk": {
        "15": {
          "risk": "TLS (probably) Not Carrying HTTPS",
          "severity": "Low",
          "risk_score": { "total": 760, "client": 680, "server": 80 }
        }
      },
      "confidence": { "6": "DPI" },
      "proto": "TLS",
      "breed": "Safe",
      "category": "Web"
    },
    "tls": {
      "version": "TLSv1.2",
      "client_requested_server_name": "www.********.com",
      "ja3": "398430069e0a8ecfbc8db0778d658d77",
      "ja3s": "fbe78c619e7ea20046131294ad087f05",
      "unsafe_cipher": 0,
      "cipher": "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
      "tls_supported_versions": "TLSv1.3,TLSv1.2"
    }
  }

Such a JSON contains following confidence object:

  "confidence": { "6": "DPI" }

In the example JSON schema file included in the nDPId sources, the very same confidence attribute is declared this way:

  "confidence": {
      "type": "string",
      "enum": [
          "0",
          "1",
          "2",
          "3",
          "4"
      ]
  }

and it's missing the value "6" is missing.

Schema definition should be updated to include also "6" value, as well as others missing (5?, 7?)

[utoni]

True. I need to update the schema files since libnDPI changed a lot which affects the generated JSON output. Will do ASAP.

[utoni]

I need to wait till ntop/nDPI #1725 get's merged.

[verzulli]

True. I need to update the schema files since libnDPI changed a lot which affects the generated JSON output. Will do ASAP.

If you point me (roughly...) to where could I get details about "current" JSON protocol (...in nDPI, I guess), I can update the doc myself.

[utoni]

Most of the libnDPI serializastion happens in ndpi_dpi2json in src/lib/ndpi_utils.c.

[utoni]

I've updated the JSON schema (hopefully w/o missing anything).

[utoni]

I think this issue is fixed as well. If not, reopen please.