Empty Sampling Rate in IPFIX in v2.0.0

[mieczkowski]

Describe the bug

In version 2.0.0 with no mapping config (or with provided in cmd/goflow2/mapping) sampling_rate in IPFIX (maybe in netflow_v9 too?) is always 0.

With previous version (v1.3.5) it is filled with correct value.

In sflow everything is ok.

To Reproduce

mapping.yaml:

formatter:
  fields:
    - type
    - time_received_ns
    - sampling_rate
    - packets
# with or without mapping for sampling_rate
ipfix:
  mapping:
    - field: 34
      destination: sampling_rate

./bin/goflow2 -listen "netflow://:5054?count=5" -mapping mapping.yaml
(...)
{"type":"IPFIX","time_received_ns":1692717004888748779,"sampling_rate":0,"packets":4}
{"type":"IPFIX","time_received_ns":1692717004889875193,"sampling_rate":0,"packets":1}
{"type":"IPFIX","time_received_ns":1692717004891125195,"sampling_rate":0,"packets":1}
{"type":"IPFIX","time_received_ns":1692717004892344462,"sampling_rate":0,"packets":1}
{"type":"IPFIX","time_received_ns":1692717004931292813,"sampling_rate":0,"packets":1}
{"type":"IPFIX","time_received_ns":1692717004933758147,"sampling_rate":0,"packets":1}
{"type":"IPFIX","time_received_ns":1692717004934781487,"sampling_rate":0,"packets":3}

After checkout to v1.3.5 and run without mapping:

{"Type":"IPFIX","TimeReceived":1692717105,"SamplingRate":500,"Bytes":7500,"Packets":5 ... }

Expected behavior sampling_rate in IPFIX should be filled

GoFlow2:

• Version: 2.0.0
• Environment: docker or go run
• OS: Debian 11.7

[mieczkowski]

My guess: decoders/netflow/netflow.go +397

case IPFIXOptionsTemplateRecord:
      records, err := DecodeOptionsDataSet(version, dataReader, templatec.Scopes, templatec.Options)
      if err != nil {
          return flowSet, &FlowError{version, "DataSet", obsDomainId, fsheader.Id, err}
      }

      spew.Dump(records)

In v1.3.5 there are values (and our sampling rate), but in v2.0.0 there is filled list with zero values

([]netflow.OptionsDataRecord) (len=1 cap=1) {
 (netflow.OptionsDataRecord) {
  ScopesValues: ([]netflow.DataField) (len=1 cap=1) {
   (netflow.DataField) {
    PenProvided: (bool) false,
    Type: (uint16) 144,
    Pen: (uint32) 0,
    Value: ([]uint8) (len=4 cap=8924) {
     00000000  0a 11 00 00                                       |....|
    }
   }
  },
  OptionsValues: ([]netflow.DataField) (len=10 cap=10) {
   (netflow.DataField) {
    PenProvided: (bool) false,
    Type: (uint16) 0,
    Pen: (uint32) 0,
    Value: (interface {}) <nil>
   },
   (netflow.DataField) {
    PenProvided: (bool) false,
    Type: (uint16) 0,
    Pen: (uint32) 0,
    Value: (interface {}) <nil>
   },
   (netflow.DataField) {
    PenProvided: (bool) false,
    Type: (uint16) 0,
    Pen: (uint32) 0,
    Value: (interface {}) <nil>
   },
(...)

versus 1.3.5:

([]netflow.OptionsDataRecord) (len=1 cap=1) {
 (netflow.OptionsDataRecord) {
  ScopesValues: ([]netflow.DataField) (len=1 cap=1) {
   (netflow.DataField) {
    PenProvided: (bool) false,
    Type: (uint16) 144,
    Pen: (uint32) 0,
    Value: ([]uint8) (len=4 cap=60) {
     00000000  00 00 00 02                                       |....|
    }
   }
  },
  OptionsValues: ([]netflow.DataField) (len=10 cap=10) {
   (netflow.DataField) {
    PenProvided: (bool) false,
    Type: (uint16) 41,
    Pen: (uint32) 0,
    Value: ([]uint8) (len=8 cap=56) {
     00000000  00 00 00 00 00 01 ea ef                           |........|
    }
   },
   (netflow.DataField) {
    PenProvided: (bool) false,
    Type: (uint16) 42,
    Pen: (uint32) 0,
    Value: ([]uint8) (len=8 cap=48) {
     00000000  00 00 00 00 00 01 ed 31                           |.......1|
    }
   },
(...)

[lspgn]

Thank you for the report! Will have a look asap.

Might be related to #113 if you need to use the custom mapping. In that case, could you run GoFlow2 with -produce=raw? ~But also could be the decoding error you identified. Are you able to share a template+data or a packet capture?~

Figured it out. Fix is in #207, are you able to test the bugfix/sampling branch?

[mieczkowski]

Wow, thank you for quick response and fix. I can confirm that this change works :)

{"type":"IPFIX","time_received_ns":1692776265077063604,"sampling_rate":500,"packets":1,"bytes":52}
{"type":"IPFIX","time_received_ns":1692776265077063604,"sampling_rate":500,"packets":1,"bytes":40}
{"type":"IPFIX","time_received_ns":1692776265077063604,"sampling_rate":500,"packets":1,"bytes":40}
{"type":"IPFIX","time_received_ns":1692776265077063604,"sampling_rate":500,"packets":3,"bytes":224}