Unable to see traffic data with softflowd

[gaby]

@lspgn I'm having this exact issue. I'm sending data using softflowd. When using a sampling rate of 0 which would get everything the graph is flat. I tried changing the sample rate and data doesnt show up even though running the queries above return results.

The only way I was able to get data in was using v5 with the legacy listener. When doing that the goflow internal dashboard would remain empty.

My cmd is something like: softflowd -d -s 0 -A mili -i eth0 -v 9

Originally posted by @gaby in https://github.com/netsampler/goflow2/issues/151#issuecomment-1550538069

[lspgn]

Hello @gaby You don't need to open a new issue, I'm happy to re-open an older one if necessary. Unfortunately I do not use softflowd so I do not know if the tool provides the option templates necessary for any flow collector to set the sampling rate.

Could you specify which version of GoFlow2 you are using?

Have you tried the solutions listed here?

Regarding your question:

I tried changing the sample rate and data doesnt show up even though running the queries above return results.

Could you give more information about the query?

[gaby]

@lspgn I'm using the latest goflow2 with the ClickHouse/Kafka/Grafana pipeline.

I tried the solutions listed on the previous issue. Changing the query still returns 0 for netflow v9.

Actually this one:

curl -XPOST --data 'SELECT sampling_rate,count() FROM flows_raw GROUP BY sampling_rate' http://localhost:8123
0   56541584

You mentioned that means there's no options template sent. So maybe softflowd has the same issue.

I wanted to use akvorado as suggested but it requires sending SNMP which I don't have. That's the only thing holding me from using it. Discussion: https://github.com/akvorado/akvorado/discussions/658 @vincentbernat

[lspgn]

I tried the solutions listed on the previous issue. Changing the query still returns 0 for netflow v9.

My advice would be to remove the *sampling_rate from the queries inside the Grafana dashboard. This should assume 1.

You mentioned that means there's no options template sent. So maybe softflowd has the same issue.

That's quite possible. Mikrotik provides the sampling rate inside the data template instead of the option template. But if you can provide a packet capture that would be great.

[gaby]

Removing sampling rate from dashboards made no difference

[lspgn]

hello, just following up on this, I am still missing information to be able to investigate: would you be able to provide a packet capture?

additionally, could you try the following in a mapping.yaml:

netflowv9:
  mapping:
    - field: 34 # samplingInterval provided within the template
      destination: sampling_rate
      endian: little

And would you be able to compile the latest commit of the main branch (v2)?

go run cmd/goflow2/main.go -mapping ./mapping.yaml

[gaby]

I'm currently on vacation, will report back in 3 weeks. Thanks!

[lspgn]

Hi @gaby Let me know if you still need help

[gaby]

@lspgn I could never get it to work with the data from softflowd, going to migrate to Elastic Filebeat. It supports ingesting Sflow and outputs JSON data. Thanks!