search

IBM / k8s-storage-perf

This git repo will host the playbooks for collecting performance metrics for a Kubernetes persistent storage for IBM Cloud Paks
Apache License 2.0
9 stars 18 forks source link

S390 support for k8s storage performance tool. #9

Open krishna-k-ibm opened 1 year ago

krishna-k-ibm commented 1 year ago

Problem Description: The storage performance tool is not running successfully on S390 cluster. This came up while working on this customer issue.

Reproducibility: Consistent

Type of Environment: Pre-existing and/or Embed

S390

Steps to reproduce:

  1. Install the per-requisites as mentioned here https://github.com/IBM/k8s-storage-perf
  2. Run the playbook
  3. Tool is timed out for S390 clusters

Actual result:

image.png

Expected result:

Tool should be successful.

Additional info or Screenshots: (log files, diagnostics, system/env information etc, use the tool mentioned below)

For questions pertaining to this template, please contact shahrukh.suhail@ibm.com or svoruga@us.ibm.com

Information on assigning Severity: here

Information on Cluster Serviceability & Troubleshooting tool: here

Diagnostics and logs files for all pods can be fetched by: ./icp4d_tools.sh --collect=smart

Also, here is the logs from the failed pods.

image
pgodowski commented 1 year ago

A stop-gap recommendation received directly from this tool owners is to use dd command instead; sample command and expected output to meet the acceptable storage perf criteria below:

Disk latency test

dd if=/dev/zero of=/PVC_mount_path/testfile bs=4096 count=1000 oflag=dsync

The value must be comparable to or better than: 2.5 MB/s.

Disk throughput test

dd if=/dev/zero of=/PVC_mount_path/testfile bs=1G count=1 oflag=dsync

The value must be comparable to or better than: 209 MB/s.

bxu1999 commented 1 year ago

Thanks for the feedback. Will look into adding the dd command as an alternative for running the performance tests.