Citrix Netscaler: No templates for flowset-ids below 260

[HenryTheSir]

• Version: logstash-codec-netflow (3.5.2) - also with 3.7.0
• Operating System: Ubuntu 16.04 LTS
• Config File (if you have sensitive info, please remove it): input { udp { host => "elks" port => 4739 codec => netflow { versions => [10] target => ipfix cache_save_path => "/tmp/" } type => ipfix } }
• Sample Data:
• Steps to Reproduce: Configure Citrix Netscaler to send Appflow Data to Logstash Port, get the first data, some Templates get decoded and saved to the cache file, some are not decoded and the dataflows can not be decoded:

[2017-10-02T14:12:15,793][WARN ][logstash.codecs.netflow ] Can't (yet) decode flowset id 258 from observation domain id 1000, because no template to decode it with has been received. This message will usually go away after 1 minute.

But if one traces the incoming data for ~60 Secs there are Packets with the template Data and wireshark can decode them with not problem.

In the Cache file are only templates for the following IDs:

If necessary I can provide Wireshark trace.

The one thing I saw until now is that Wireshark is warning at the Template Packet for ID 258:

[jorritfolmer]

If you can email me a pcap, I’ll take a look.

[bluefangs]

@jorritfolmer

I'm attaching a similar pcap file that contains both IDS (256 and 1024) I've also observed that the ones with id 256 are ignored while the id with 1024 get processed just fine. One difference I've noted is that I don't even get the log message that the OP has stated with id256 (can't yet decode flowset....)

@HenryTheSir - Can you try to replay my attached pcap file to see if the issue is the same?

Thanks.

not-working-pcap.pcap.tar.gz

[HenryTheSir]

Hi @bluefangs , your problems seems quite different to mine.

@jorritfolmer : I send the .pcap to you.

[marian-craciunescu]

@bluefangs your template can be decoded because you have actually 2 unkown fields in the template. Field 12 and 13 FIELD 12: Unkown (57590) with length 2 FIELD 13: Unkown (57591) with length 16

Any ideas what they are? in order to fix your problem you need to add them in netflow.yaml

[bluefangs]

@marian-craciunescu - Thanks for the suggestion. Those represent application-id and application-name. Rather strange that it is actually defined in the netflow yaml file :

94:
- :string
- :application_description
95:
- :application_id
- :application_id
96:
- :string
- :application_name

On the side note, does it matter if the fields are numbered 12 and 13 on the packet while in the yaml file it falls under 95 and 96?

---

Havning said that, I think you can disregard the pcap I've provided in this case since mine is a bit different. I'll look into my specific case further and maybe raise a new issue should there be a need.

Thanks!

[jorritfolmer]

@HenryTheSir Thanks for your pcap! I've looked at all the templates that the Netscaler emits, and it appears there are A LOT of fields we lack a definition for.

Do you have access to any documentation? A quick Google turns up empty...

These are the unknown fields:

In the Netscaler PEN 5951 we miss definitions of:

• field 358, length 4
• field 392, length 2
• field 403, length 1
• field 434, length 1
• field 451, length 8
• field 464, length 4
• field 476, length 1
• field 477, length 4
• field 479, length 65535
• field 487, length 4
• field 491, length 4
• field 492, length 4
• field 509, length 4
• field 512, length 1
• field 522, length 4
• field 536, length 4

[jorritfolmer]

And also lack these in PEN 5951:

• field 359, length 4
• field 364, length 4
• field 384, length 16
• field 400, length 2
• field 435, length 1
• field 452, length 8
• field 463, length 4
• field 475, length 65535
• field 488, length 4
• field 510, length 4
• field 511, length 4
• field 523, length 4
• field 532, length 8
• field 541, length 4

[jorritfolmer]

And:

• field 360, length 4
• field 365, length 4
• field 380, length 1
• field 385, length 16
• field 388, length 1
• field 401, length 2
• field 452, length 8
• field 458, length 4
• field 466, length 4
• field 489, length 4
• field 507, length 4
• field 518, length 8
• field 533, length 8
• field 537, length 2

[jorritfolmer]

And:

• field 361, length 4
• field 366, length 8
• field 377, length 4
• field 386, length 65535
• field 399, length 4
• field 402, length 1
• field 447, length 4
• field 455, length 8
• field 459, length 4
• field 480, length 1
• field 493, length 8
• field 508, length 4
• field 519, length 8
• field 538, length 2

Plus:

• field 362, length 4
• field 363, length 16
• field 393, length 4
• field 448, length 4
• field 460, length 4
• field 467, length 8
• field 481, length 1
• field 494, length 8
• field 513, length 2
• field 520, length 8
• field 534, length 8

Also:

• field 367, length 16
• field 377, length 4
• field 387, length 65535
• field 389, length 65535
• field 394, length 4
• field 408, length 1
• field 461, length 4
• field 468, length 8
• field 481, length 1
• field 494, length 8
• field 514, length 2
• field 521, length 8
• field 535, length 8

[jorritfolmer]

No idea what these Netscaler guys have been smoking, but the list goes on and on. I'm pretty sure there must be fields like:

• bytes sent on wednesday
• bytes sent on wednesday, but on a leap year.
• bytes sent on sunday, but only when it was raining

I've pushed a commit with the fields above (all netscalerUnknown). This should at least let you decode the templates below 270.

[HenryTheSir]

Hi @jorritfolmer ,

there is only the public available docu from citrix. But we could raise a case and hope to get more information.

One maybe silly question:

I changed my config -> ipfix_definitions => "/tmp/ipfix_template.prepared"

And the "/tmp/ipfix_template.prepared" contains a 1-1 copy of this file: https://github.com/logstash-plugins/logstash-codec-netflow/blob/master/lib/logstash/codecs/netflow/ipfix.yaml

but I get again the Unsupported enterprise field error

[2017-10-10T08:38:25,149][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>524, :enterprise=>5951, :length=>2} [2017-10-10T08:38:25,195][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>368, :enterprise=>5951, :length=>1} [2017-10-10T08:38:25,196][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>378, :enterprise=>5951, :length=>4} [2017-10-10T08:38:25,198][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>382, :enterprise=>5951, :length=>65535} [2017-10-10T08:38:25,205][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>410, :enterprise=>5951, :length=>1} [2017-10-10T08:38:25,206][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>469, :enterprise=>5951, :length=>4} [2017-10-10T08:38:25,207][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>462, :enterprise=>5951, :length=>4} [2017-10-10T08:38:25,207][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>395, :enterprise=>5951, :length=>4} [2017-10-10T08:38:25,207][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>482, :enterprise=>5951, :length=>1} [2017-10-10T08:38:25,207][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>515, :enterprise=>5951, :length=>2} [2017-10-10T08:38:25,209][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>497, :enterprise=>5951, :length=>8} [2017-10-10T08:38:25,209][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>495, :enterprise=>5951, :length=>8} [2017-10-10T08:38:47,083][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>368, :enterprise=>5951, :length=>1} [2017-10-10T08:38:47,084][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>378, :enterprise=>5951, :length=>4} [2017-10-10T08:38:47,086][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>382, :enterprise=>5951, :length=>65535} [2017-10-10T08:38:47,088][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>524, :enterprise=>5951, :length=>2} [2017-10-10T08:38:47,101][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>410, :enterprise=>5951, :length=>1} [2017-10-10T08:38:47,101][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>469, :enterprise=>5951, :length=>4} [2017-10-10T08:38:47,103][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>462, :enterprise=>5951, :length=>4} [2017-10-10T08:38:47,103][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>395, :enterprise=>5951, :length=>4} [2017-10-10T08:38:47,103][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>482, :enterprise=>5951, :length=>1} [2017-10-10T08:38:47,103][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>515, :enterprise=>5951, :length=>2} [2017-10-10T08:38:47,105][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>497, :enterprise=>5951, :length=>8} [2017-10-10T08:38:47,105][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>495, :enterprise=>5951, :length=>8} [2017-10-10T08:38:47,174][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>524, :enterprise=>5951, :length=>2} [2017-10-10T08:38:47,223][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>368, :enterprise=>5951, :length=>1} [2017-10-10T08:38:47,223][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>378, :enterprise=>5951, :length=>4} [2017-10-10T08:38:47,225][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>382, :enterprise=>5951, :length=>65535} [2017-10-10T08:38:47,239][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>410, :enterprise=>5951, :length=>1} [2017-10-10T08:38:47,239][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>469, :enterprise=>5951, :length=>4} [2017-10-10T08:38:47,240][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>462, :enterprise=>5951, :length=>4} [2017-10-10T08:38:47,241][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>395, :enterprise=>5951, :length=>4} [2017-10-10T08:38:47,241][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>482, :enterprise=>5951, :length=>1} [2017-10-10T08:38:47,241][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>515, :enterprise=>5951, :length=>2} [2017-10-10T08:38:47,242][WARN ][logstash.codecs.netflow ] Unsupported enterprise field {:type=>497, :enterprise=>5951, :length=>8}

Did I made a mistake?

[jorritfolmer]

No I don't think you made a mistake. Flows using tempates in the rage 250-260 and 261-270 should now decode.

The unsupported warnings are most likely from the higher numbered templates, I stopped adding the fields yesterday. There are likely loads more. I'll add them when I have more time available.

[HenryTheSir]

Thanks import now is working!

[jorritfolmer]

Thanks for the update!