[procdockerstatsd] ERR procdockerstatsd[92237]: 'docker stats --no-stream -a' returned null output

[vivekrnv]

Description

In rare cases, procdockerstatsd returned the folllowing error:

Feb  4 07:11:05.231043 r-boxer-sw01 INFO procdockerstatsd[151258]: panic: send on closed channel
Feb  4 07:11:05.231195 r-boxer-sw01 INFO procdockerstatsd[151258]: goroutine 51 [running]:
Feb  4 07:11:05.231352 r-boxer-sw01 INFO procdockerstatsd[151258]: github.com/docker/cli/cli/command/container.runStats.func1(0xc00010e300, 0xc00010e360)
Feb  4 07:11:05.231424 r-boxer-sw01 INFO procdockerstatsd[151258]: #011/go/src/github.com/docker/cli/cli/command/container/stats.go:78 +0x291
Feb  4 07:11:05.231489 r-boxer-sw01 INFO procdockerstatsd[151258]: created by github.com/docker/cli/cli/command/container.runStats
Feb  4 07:11:05.231569 r-boxer-sw01 INFO procdockerstatsd[151258]: #011/go/src/github.com/docker/cli/cli/command/container/stats.go:152 +0x1245
Feb  4 07:11:05.237333 r-boxer-sw01 ERR procdockerstatsd[92237]: Error running command 'docker stats --no-stream -a'
Feb  4 07:11:05.237507 r-boxer-sw01 ERR procdockerstatsd[92237]: 'docker stats --no-stream -a' returned null output

The following issue is already reported in docker: https://github.com/docker/cli/issues/3482

This is fixed by https://github.com/docker/cli/pull/3597 but is not available in the docker version used in bullseye.

[arlakshm]

This problem could have impact on the telemetry streaming cpu stats

[arlakshm]

Need help in understanding how this failure affect telemetry

[vivekrnv]

This command is run by procdockerstatsd once every two mins, https://github.com/sonic-net/sonic-host-services/blob/4a2ef99630ee02e104643bf356cafd6a1228b882/scripts/procdockerstatsd#L176 , so even if it fails once, the next time it succeeds, it'll populate the STATE_DB properly. So i think it shouldn't impact the telemetry all that much