search

ntop / ntopng

Web-based Traffic and Security Network Traffic Monitoring
http://www.ntop.org
GNU General Public License v3.0
6.17k stars 648 forks source link

nprobe collector #219

Closed anartahamim closed 8 years ago

anartahamim commented 8 years ago

I run nbrobe with:

nprobe --zmq "tcp://*:5556" -i none -n none --collector-port 2055 -b 2

When I send netflow packets to nprobe, I see the following errors:

11/Oct/2015 14:11:35 [engine.c:2361] New Flow: [cbt] 17.34.51.85:13124 -> 17.34.51.68:4386 [00:00:00:00:00:00 -> 00:00:00:00:00:00][vlan 65535][tos 0][ifIdx: 0 -> 0][subflowId: 0/0x0000][idx=13265] 11/Oct/2015 14:11:36 [util.c:3865] [ZMQ] {"8":"17.34.51.85","12":"17.34.51.68","15":"0.0.0.0","10":0,"14":0,"2":266,"1":48828,"22":981,"21":981,"7":13124,"11":4386,"6":0,"4":7,"5":0,"16":714,"17":714,"9":0,"13":0,"42":842} 11/Oct/2015 14:11:36 [engine.c:2541] Emitting Flow: [->][cbt] 17.34.51.85:13124 -> 17.34.51.68:4386 [266 pkt/48828 bytes][ifIdx 0->0][0.0 sec][init Unknown][AS: 714 -> 714] 11/Oct/2015 14:11:36 [engine.c:2361] New Flow: [cbt] 17.34.51.85:13124 -> 17.34.51.68:4386 [00:00:00:00:00:00 -> 00:00:00:00:00:00][vlan 65535][tos 0][ifIdx: 0 -> 0][subflowId: 0/0x0000][idx=13265] 11/Oct/2015 14:11:37 [util.c:3865] [ZMQ] {"8":"17.34.51.85","12":"17.34.51.68","15":"0.0.0.0","10":0,"14":0,"2":266,"1":48828,"22":981,"21":981,"7":13124,"11":4386,"6":0,"4":7,"5":0,"16":714,"17":714,"9":0,"13":0,"42":843} 11/Oct/2015 14:11:37 [engine.c:2541] Emitting Flow: [->][cbt] 17.34.51.85:13124 -> 17.34.51.68:4386 [266 pkt/48828 bytes][ifIdx 0->0][0.0 sec][init Unknown][AS: 714 -> 714]

what can I do to fix those errors? Also, when I send netflow data I see the flows in the flows view only for a short period of time. How can I see them longer?

lucaderi commented 8 years ago

What is the error you are talking about? It looks good to me

You can change the flow idleness in "Flow Idle Timeout" (menu preferences of ntopng)

anartahamim commented 8 years ago

Hi Luca,

We have a very important demo coming soon and I will very much appreciate your help. We bought the license for the NProbe, but still having problems.

I am using nprobe with zmq like this: nprobe --zmq "tcp://*:5556" -i none -n none --collector-port 2055 -b 2

In the CFlow packet I am passing the following fields:

FIRST_SWITCHED; template->fields[1].type = LAST_SWITCHED; template->fields[2].type = PROTOCOL; template->fields[3].type = IPV4_SRC_ADDR; template->fields[4].type = IPV4_DST_ADDR; template->fields[5].type = L4_SRC_PORT; template->fields[6].type = L4_DST_PORT; template->fields[7].type = IN_BYTES; template->fields[8].type = IN_PKTS; template->fields[9].type = FLOWS; template->fields[10].type = APPLICATION_TAG; template->fields[11].type = APPLICATION_NAME;

But I see poor views in the web interface of the ntop:

In the flows tab Application is Unknown In the Host tab there is not Traffic shown, in Peers L7 protocol is Unknown, in Protocols the Application Protocol is Unknown and no Talkers.

The pcap of the CFlow is attached.

Can you please tell me which fields I need to add in order to get a good view of the traffic (not Unknown data). Or do I need to do something else?

Thanks, Anat Rahamim, EZChip

From: Luca Deri [mailto:notifications@github.com] Sent: Monday, October 12, 2015 12:05 PM To: ntop/ntopng Cc: Anaty Rahamim Bar Kat Subject: Re: [ntopng] nprobe collector (#219)

What is the error you are talking about? It looks food to me

You can change the flow idleness in "Flow Idle Timeout" (menu preferences of ntopng)

— Reply to this email directly or view it on GitHubhttps://github.com/ntop/ntopng/issues/219#issuecomment-147336401.

lucaderi commented 8 years ago
  1. please send me a pcap file that contains some unknown applications so that we can see what is going on
  2. Do flows reach ntopng?
  3. I need the original packets that were used to generate the flows
  4. The configuration is ok, just the protocol is not detected and I need to see if this is a new protocol for which we do not have support, or if it’s a bug
anartahamim commented 8 years ago

Hi Luca,

The pcap file of the flow is attached (netflow_app_EPG.pcap) The original packets which generated the flow are attached (frames_in_side0_eng0_10GE_01.pcap) But when I generate the flow, for now, I set the protocol and application params hard coded in the following manner:

//type=PROTOCOL data_flow->protocol = 7;

            //type=IPV4_SRC_ADDR
            data_flow->src_ip = ((ezipv4_header*)packet->L3_header)->source_ip;

            //type=IPV4_DST_ADDR
            data_flow->dst_ip = ((ezipv4_header*)packet->L3_header)->destination_ip;

//type=L4_SRC_PORT data_flow->src_port = ((eztcp_header*)packet->L4_header)->source_port;

            //type=L4_DST_PORT
            data_flow->dst_port = ((eztcp_header*)packet->L4_header)->dest_port;

            //type=IN_BYTES
            data_flow->in_bytes = flow->num_bytes;//packet->frame.frame_length;

            //type=IN_PKTS
            data_flow->in_pkts = flow->num_packets;

            //type=FLOWS
            data_flow->flows = 1;

            //type=APPLICATION_TAG
            data_flow->application_tag = 8;

            //type=APPLICATION_NAME
            uint16_t application_name_len = 10
            ezdp_mem_copy(data_flow->application_name,"EPG",3);

The flow reaches ntopng. I see the it in a poor view. See below the screenshots.

Please tell me what fields do I need to pass in the flow in order to see full view in the ntop. Can you please send me an example of a flow pcap file which generates full view in the ntop?

I do not use the issue track because I can't attach pcap files there.

[cid:image004.png@01D1058C.153796E0]

[cid:image005.png@01D1058C.153796E0]

[cid:image006.png@01D1058C.153796E0] [cid:image001.png@01D1058C.9CC19E30] Thanks very much for your help, Anat Rahamim, EZChip

From: Luca Deri [mailto:notifications@github.com] Sent: Monday, October 12, 2015 5:56 PM To: ntop/ntopng Cc: Anaty Rahamim Bar Kat Subject: Re: [ntopng] nprobe collector (#219)

  1. please send me a pcap file that contains some unknown applications so that we can see what is going on
  2. Do flows reach ntopng?
  3. I need the original packets that were used to generate the flows
  4. The configuration is ok, just the protocol is not detected and I need to see if this is a new protocol for which we do not have support, or if it’s a bug

— Reply to this email directly or view it on GitHubhttps://github.com/ntop/ntopng/issues/219#issuecomment-147423634.

lucaderi commented 8 years ago

pcap is not attached, nor images. Please try again

anartahamim commented 8 years ago

I sent the original mail directly to you (with the attached files)

From: Luca Deri [mailto:notifications@github.com] Sent: Tuesday, October 13, 2015 10:12 AM To: ntop/ntopng Cc: Anaty Rahamim Bar Kat Subject: Re: [ntopng] nprobe collector (#219)

pcap is not attached, nor images. Please try again

— Reply to this email directly or view it on GitHubhttps://github.com/ntop/ntopng/issues/219#issuecomment-147626276.

lmangani commented 8 years ago

Moved to mail thread, closing.