network/?? measurements only contain type pod, no measurements for type node

[damianoneill]

When collecting stats with Heapster, metrics written to influxdb only contain network measurements for type pod. Subsequently in Grafana the cluster dashboard contains no datapoints on the Network Graphs.

> show measurements
name: measurements
name
----
cpu/limit
cpu/node_allocatable
cpu/node_capacity
cpu/node_reservation
cpu/node_utilization
cpu/request
cpu/usage
cpu/usage_rate
filesystem/available
filesystem/limit
filesystem/usage
log/events
memory/limit
memory/major_page_faults
memory/major_page_faults_rate
memory/node_allocatable
memory/node_capacity
memory/node_reservation
memory/node_utilization
memory/page_faults
memory/page_faults_rate
memory/request
memory/usage
memory/working_set
network/rx
network/rx_errors
network/rx_errors_rate
network/rx_rate
network/tx
network/tx_errors
network/tx_errors_rate
network/tx_rate
uptime

$> show tag values on "k8s" with key = "type"
...
name: network/rx
key  value
---  -----
type pod

name: network/rx_errors
key  value
---  -----
type pod

name: network/rx_errors_rate
key  value
---  -----
type pod

name: network/rx_rate
key  value
---  -----
type pod

name: network/tx
key  value
---  -----
type pod

name: network/tx_errors
key  value
---  -----
type pod

name: network/tx_errors_rate
key  value
---  -----
type pod

name: network/tx_rate
key  value
---  -----
type pod

Output of heapster --version:

Running within a privileged docker container.

heapster-v1.3.0

Output of kubectl version:

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.4", GitCommit:"7243c69eb523aa4377bce883e7c0dd76b84709a1", GitTreeState:"clean", BuildDate:"2017-03-07T23:53:09Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"269f928217957e7126dc87e6adfa82242bfe5b1e", GitTreeState:"clean", BuildDate:"2017-07-03T15:31:10Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}

[damianoneill]

$ curl -s http://xxx:10255/stats/ | jq '.stats[0].network'
{
  "name": "eno1",
  "rx_bytes": 24109667713,
  "rx_packets": 78524878,
  "rx_errors": 0,
  "rx_dropped": 12,
  "tx_bytes": 3019947673,
  "tx_packets": 25560068,
  "tx_errors": 0,
  "tx_dropped": 0,
  "interfaces": [
    {
      "name": "eno1",
      "rx_bytes": 24109667713,
      "rx_packets": 78524878,
      "rx_errors": 0,
      "rx_dropped": 12,
      "tx_bytes": 3019947673,
      "tx_packets": 25560068,
      "tx_errors": 0,
      "tx_dropped": 0
    },
    {
      "name": "flannel.1",
      "rx_bytes": 0,
      "rx_packets": 0,
      "rx_errors": 0,
      "rx_dropped": 0,
      "tx_bytes": 0,
      "tx_packets": 0,
      "tx_errors": 0,
      "tx_dropped": 8
    }
  ],
  "tcp": {
    "Established": 0,
    "SynSent": 0,
    "SynRecv": 0,
    "FinWait1": 0,
    "FinWait2": 0,
    "TimeWait": 0,
    "Close": 0,
    "CloseWait": 0,
    "LastAck": 0,
    "Listen": 0,
    "Closing": 0
  },
  "tcp6": {
    "Established": 0,
    "SynSent": 0,
    "SynRecv": 0,
    "FinWait1": 0,
    "FinWait2": 0,
    "TimeWait": 0,
    "Close": 0,
    "CloseWait": 0,
    "LastAck": 0,
    "Listen": 0,
    "Closing": 0
  }
}

[cezarygerard]

Hi @damianoneill

Thank you for submitting the issue. Do you see the same behaviour on clusters with newer versions of Kubernetes?

[damianoneill]

Hi @cezarygerard the challenge is we are using redhat atomic and therefore the version of Kubernetes is managed by this container operating system.

[DirectXMan12]

@damianoneill forgive me if I've missed something here, but what's the expected result? Network can only be measured on a per-pod basis (with cadvisor, at least), since it looks at the interfaces in the network namespace, and all containers in a pod share a netns.

[fejta-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten