search

testcontainers / testcontainers-go

Testcontainers for Go is a Go package that makes it simple to create and clean up container-based dependencies for automated integration/smoke tests. The clean, easy-to-use API enables developers to programmatically define containers that should be run as part of a test and clean up those resources when the test is done.
https://golang.testcontainers.org
MIT License
3.63k stars 500 forks source link

[Feature]: Expose containers stats #1826

Open pablochacin opened 1 year ago

pablochacin commented 1 year ago

Problem

In some use cases, it would be convenient to have access to the container's stats to measure the resource (memory, CPU) consumption to detect regressions or establish a comparison between alternative implementations or configurations.

Solution

Docker provides this information in the stats command. The proposed solution is to expose this information from the Container interface as Container.Stats()

The raw information obtained from docker is quite extensive as seen in the example below, with many elements only available in Linux systems but not in Windows. Moreover, the units used for the reported metrics can vary between platforms. For example, Linux uses nanoseconds for measuring CPU time, while Windows uses 100s of nanoseconds.

container's stats in JSON format. Click to visualize ```json "stats": { "read": "2023-10-27T12:25:06.019724365Z", "pre_read": "0001-01-01T00:00:00Z", "pids_stats": { "current": 1059, "limit": 76608 }, "blkio_stats": { "io_service_bytes_recursive": [ { "major": 253, "minor": 2, "op": "read", "value": 0 }, { "major": 253, "minor": 2, "op": "write", "value": 4096 }, { "major": 259, "minor": 0, "op": "read", "value": 1744896 }, { "major": 259, "minor": 0, "op": "write", "value": 0 }, { "major": 253, "minor": 0, "op": "read", "value": 1744896 }, { "major": 253, "minor": 0, "op": "write", "value": 12197888 }, { "major": 253, "minor": 1, "op": "read", "value": 1744896 }, { "major": 253, "minor": 1, "op": "write", "value": 12505088 } ], "io_serviced_recursive": [], "io_queued_recursive": [], "io_service_time_recursive": [], "io_wait_time_recursive": [], "io_merged_recursive": [], "io_time_recursive": [], "sectors_recursive": [] }, "num_procs": 0, "storage_stats": { "read_count_normalized": 0, "read_size_bytes": 0, "write_count_normalized": 0, "write_size_bytes": 0 }, "cpu_stats": { "cpu_usage": { "total_usage": 25415300269000, "percpu_usage": [], "usage_in_kernelmode": 7710788995000, "usage_in_usermode": 17704511274000 }, "system_usage": 14395911570000000, "online_cpu_s": 20, "throttling_data": { "periods": 0, "throttled_periods": 0, "throttled_time": 0 } }, "pre_cpu_stats": { "cpu_usage": { "total_usage": 0, "percpu_usage": [], "usage_in_kernelmode": 0, "usage_in_usermode": 0 }, "system_usage": 0, "online_cpu_s": 0, "throttling_data": { "periods": 0, "throttled_periods": 0, "throttled_time": 0 } }, "memory_stats": { "usage": 3990077440, "max_usage": 0, "stats": { "pglazyfreed": 0, "pgmajfault": 341, "shmem": 275202048, "slab": 170641456, "anon_thp": 29360128, "file": 2074447872, "file_dirty": 4096, "sock": 12288, "file_mapped": 401707008, "file_writeback": 0, "pgactivate": 87325, "pgfault": 17291097, "pgrefill": 2955, "pgscan": 170011, "workingset_refault": 0, "active_file": 194707456, "anon": 1692143616, "workingset_nodereclaim": 0, "inactive_anon": 1813409792, "kernel_stack": 17350656, "unevictable": 0, "active_anon": 155107328, "inactive_file": 1606324224, "pgdeactivate": 2646, "slab_reclaimable": 144779624, "thp_collapse_alloc": 2, "thp_fault_alloc": 24, "pglazyfree": 436, "slab_unreclaimable": 25861832, "workingset_activate": 0, "pgsteal": 113971 }, "failcnt": 0, "limit": 67073630208, "commit": 0, "commit_peak": 0, "private_working_set": 0 } }, "read": "2023-10-27T12:25:06.019724365Z", "pre_read": "0001-01-01T00:00:00Z", "pids_stats": { "current": 1059, "limit": 76608 }, "blkio_stats": { "io_service_bytes_recursive": [ { "major": 253, "minor": 2, "op": "read", "value": 0 }, { "major": 253, "minor": 2, "op": "write", "value": 4096 }, { "major": 259, "minor": 0, "op": "read", "value": 1744896 }, { "major": 259, "minor": 0, "op": "write", "value": 0 }, { "major": 253, "minor": 0, "op": "read", "value": 1744896 }, { "major": 253, "minor": 0, "op": "write", "value": 12197888 }, { "major": 253, "minor": 1, "op": "read", "value": 1744896 }, { "major": 253, "minor": 1, "op": "write", "value": 12505088 } ], "io_serviced_recursive": [], "io_queued_recursive": [], "io_service_time_recursive": [], "io_wait_time_recursive": [], "io_merged_recursive": [], "io_time_recursive": [], "sectors_recursive": [] }, "num_procs": 0, "storage_stats": { "read_count_normalized": 0, "read_size_bytes": 0, "write_count_normalized": 0, "write_size_bytes": 0 }, "cpu_stats": { "cpu_usage": { "total_usage": 25415300269000, "percpu_usage": [], "usage_in_kernelmode": 7710788995000, "usage_in_usermode": 17704511274000 }, "system_usage": 14395911570000000, "online_cpu_s": 20, "throttling_data": { "periods": 0, "throttled_periods": 0, "throttled_time": 0 } }, "pre_cpu_stats": { "cpu_usage": { "total_usage": 0, "percpu_usage": [], "usage_in_kernelmode": 0, "usage_in_usermode": 0 }, "system_usage": 0, "online_cpu_s": 0, "throttling_data": { "periods": 0, "throttled_periods": 0, "throttled_time": 0 } }, "memory_stats": { "usage": 3990077440, "max_usage": 0, "stats": { "inactive_anon": 1813409792, "kernel_stack": 17350656, "unevictable": 0, "active_anon": 155107328, "inactive_file": 1606324224, "pgdeactivate": 2646, "slab_reclaimable": 144779624, "thp_collapse_alloc": 2, "thp_fault_alloc": 24, "pglazyfree": 436, "slab_unreclaimable": 25861832, "workingset_activate": 0, "pgsteal": 113971, "pglazyfreed": 0, "pgmajfault": 341, "shmem": 275202048, "slab": 170641456, "anon_thp": 29360128, "file": 2074447872, "file_dirty": 4096, "sock": 12288, "file_mapped": 401707008, "file_writeback": 0, "pgactivate": 87325, "pgfault": 17291097, "pgrefill": 2955, "pgscan": 170011, "workingset_refault": 0, "active_file": 194707456, "anon": 1692143616, "workingset_nodereclaim": 0 }, "failcnt": 0, "limit": 67073630208, "commit": 0, "commit_peak": 0, "private_working_set": 0 }, "name": "/test-e2e-control-plane", "id": "6271ed069235600bcbdbe70505db90f299197c69a49b097dfa5b6e941b917172", "networks": { "eth0": { "rx_bytes": 80197969, "rx_packets": 19779, "rx_errors": 0, "rx_dropped": 0, "tx_bytes": 2127699, "tx_packets": 5507, "tx_errors": 0, "tx_dropped": 0, "endpoint_id": "", "instance_id": "" } } } ```

Therefore, we suggest returning a custom type with the information provided by the stats command and containing only fields that are available in both Linux and Windows platforms and normalized to one unit (for example, nanoseconds):

type ContainerStats struct {
            Timestamp               time.Time
            CPUUsageTotal       uint64
            CPUUsageInKernel uint64
            CPUUsageUser       uint64
            CPUPercentage      float64
            MemoryUsage         uint64               // in Windows, it corresponds to Commit memory
            MemoryMaxUsage   uint64               // in Windows, it corresponds to PeakCommit memory
            MemoryPercentage float64
            NetworkRxBytes      uint64
            NetworkTxBytes      uint64
            BlockIOReadBytes  uint64               // BlockIO metrics must be calculated from different sources in Windows and Linux
            BlockIOWriteBytes  uint64                
}

Benefit

Offers the ability to monitor a container's resource usage.

Alternatives

We don't see any alternative that is convenient for the test developer.

Would you like to help contributing this feature?

Yes

ervitis commented 6 months ago

hi @pablochacin! I started using testcontainers-go and I would be very interested in using this feature