search

dell / csm

Dell Container Storage Modules (CSM)
Apache License 2.0
71 stars 16 forks source link

[FEATURE]: Run each metrics gatherer in a separate goroutine #1007

Open donatwork opened 1 year ago

donatwork commented 1 year ago

Description

Found issues in the karavi-metrics-powerstore, and the code is replicated to the other platforms. https://github.com/dell/csm-metrics-powerstore/blob/dec4da3c9825c3f0eddd6edd68bdc9609eff3996/internal/entrypoint/run.go#L116C7-L180C7

  1. The select statement in the Run method is not working as expected. The worker methods fire off too soon and are not deterministic. The idiom used in Go is to run each worker in a separate goroutine. Some small refactoring is needed.
  2. If the first item is implemented, then the semaphore used to gate the number of requests to the array/client lib needs to be shared among all goroutines. Currently each worker method creates a channel and uses it as a semaphore. Nothing wrong with that but that channel is local. Needs to work across multiple goroutines.

Logs

Clearly the methods are not being run according to the default intervals.

time="2023-10-09T21:22:29Z" level=info msg="function duration" duration="1.715µs" function=gatherVolumeMetrics time="2023-10-09T21:22:29Z" level=info msg="function duration" duration="1.582µs" function=pushVolumeMetrics time="2023-10-09T21:22:29Z" level=info msg="function duration" duration=3.37172ms function=ExportVolumeStatistics time="2023-10-09T21:22:29Z" level=info msg="function duration" duration="2.103µs" function=gatherFileSystemMetrics time="2023-10-09T21:22:29Z" level=info msg="function duration" duration="1.362µs" function=pushFileSystemMetrics time="2023-10-09T21:22:29Z" level=info msg="function duration" duration=1.652448ms function=ExportFileSystemStatistics time="2023-10-09T21:22:49Z" level=info msg="function duration" duration="1.678µs" function=gatherVolumeMetrics

Screenshots

No response

Additional Environment Information

No response

Steps to Reproduce

Install Observability and observe the logs of the csm/karavi-metrics-.

Expected Behavior

The gather methods should be running in the intervals configured for the module.

CSM Driver(s)

n/a

Installation Type

n/a

Container Storage Modules Enabled

No response

Container Orchestrator

n/a

Operating System

n/a

csmbot commented 1 year ago

@donatwork: Thank you for submitting this issue!

The issue is currently awaiting triage. Please make sure you have given us as much context as possible.

If the maintainers determine this is a relevant issue, they will remove the needs-triage label and respond appropriately.

We want your feedback! If you have any questions or suggestions regarding our contributing process/workflow, please reach out to us at container.storage.modules@dell.com.

gallacher commented 1 year ago

link: 18465