result.csv contains extra lines of old environment

[cdingIBM]

We tried to run the perf test too to test some openshift environment. It seems the results.csv's detailed section contains environment that we used a couple of years ago. In order to remove these "ghost" lines, we switched to new openshift environment and also tried to delete the existing storage-perf project , delete all the .csv and .log from the github directory, but these ghost lines seem to persist there. From the following output, the wisc-ocp4-dev is the old environment but the results seem to show up in Detailed Measurement section. The existence of these ghost lines would cause confusion that seqrd is being performed, which is disabled in params.yaml.

Cluster Name,PVC,Storage Type,Environment,Test Name,Thread Count,write Mb/s,Requirement
wisc-rd2,pvc-sysbench-rwo,ocs,vmware,rndwr_4k_8,8,1.2,Recommended to meet the requirement of 11 Mb/s or higher
wisc-rd2,pvc-sysbench-rwx,ocs,vmware,seqwr_1g_2,2,168.0,Recommended to meet the requirement of 128 Mb/s or higher
wisc-rd2,pvc-sysbench-rwo,ocs,vmware,seqwr_1g_2,2,129.8,Recommended to meet the requirement of 128 Mb/s or higher
wisc-rd2,pvc-sysbench-rwx,ocs,vmware,rndwr_4k_8,8,1.2,Recommended to meet the requirement of 11 Mb/s or higher
,,,,,,,,,,,,,,
,,,,,,,,,,,,,,
Detailed Measurements,,,,,,,,,,,,,,
Cluster Name,PVC,Storage Type,Environment,Test Name,Thread Count,write Mb/s,Writes/s,read Mb/s,Reads/s,Total Time,Latency Min,Latency Avg,Latency Max,Latency 95th
wisc-rd2,pvc-sysbench-rwo,ocs,vmware,rndwr_4k_8,8,1.2,310.1,0.0,0.0,10.2,6.7,24.6,250.5,50.4
wisc-ocp4-dev,pvc-sysbench-rwx,ocs,vmware,seqrd_1g_48,48,0.0,0.0,15151.9,177.6,10.1,37.1,283.1,2942.4,1057.2
wisc-ocp4-dev,pvc-sysbench-rwo,ocs,vmware,rndrd_4k_48,48,0.0,0.0,4779.4,1223538.5,10.0,0.0,0.0,71.9,0.1
wisc-rd2,pvc-sysbench-rwx,ocs,vmware,seqwr_1g_2,2,168.0,0.2,0.0,0.0,12.2,11578.8,11906.4,12233.9,12188.5
wisc-rd2,pvc-sysbench-rwo,ocs,vmware,seqwr_1g_2,2,129.8,0.1,0.0,0.0,16.1,16130.8,16130.8,16130.8,16107.5
wisc-rd2,pvc-sysbench-rwx,ocs,vmware,rndwr_4k_8,8,1.2,321.3,0.0,0.0,10.0,0.0,23.9,216.4,47.6
wisc-ocp4-dev,pvc-sysbench-rwo,ocs,vmware,seqrd_1g_48,48,0.0,0.0,15498.8,181.6,10.1,46.0,285.3,2944.0,1068.6
wisc-ocp4-dev,pvc-sysbench-rwx,ocs,vmware,rndrd_4k_48,48,0.0,0.0,4399.4,1126240.7,10.0,0.0,0.0,204.3,0.0
result.csv (END)

This is the command we used to run the performance test.

ansible-playbook main.yml --extra-vars "@./params_rd2.yml" | tee output.log

Here is the params_rd2.yaml we used. Please note I removed the 4 lines about ocp credentials and url.

storageClass_ReadWriteMany: "ocs-storagecluster-cephfs"

run_storage_perf: true

arch: amd64  # amd64, ppc64le

############################ STORAGE PERFORMANCE PARAMETERS START #######################
# vars file for roles/storage-perf-test
storage_perf_namespace: storageperf  # openshift namespace/project where jobs will be executed, it will be created by playbook if not already existing.
logfolder: '.logs'
imageurl: quay.io/ibm-cp4d-public/xsysbench:1.1

cluster_infrastructure: vmware # optional labels eg ibmcloud, aws, azure, vmware
cluster_name: wisc-rd2 # optional labels 
storage_type: ocs # optional label eg portworx, ocs, <storage vendor>

# To run the performace jobs on a dedicated compute nodes, set the node label which meet the criteria.
# The idea is to gather performance data when the jobs are running remotely from a storage node.
# A cluster administrator can label a node by running this query with appropriate label key/value: 
# oc label node <node name> "<label_key>=<label_value>" --overwrite
dedicated_compute_node:
   label_key: "<optional>"
   label_value: "<optional>"

rwx_storagesize: 10Gi
rwo_storagesize: 10Gi

#sysbench random read
sysbench_random_read: false
rread_threads: 8            # 1,4,8,16
rread_fileTotalSize: 128m
rread_fileNum: 128
rread_fileBlockSize: 4k     # 4k,8k,16k

#sysbench random write
sysbench_random_write: true
rwrite_threads: 8           # 1,4,8,16
rwrite_fileTotalSize: 4096m
rwrite_fileNum: 4
rwrite_fileBlockSize: 4k    # 4k,8k,16k

#sysbench sequential read
sysbench_sequential_read: false
sread_threads: 2            # 1,2
sread_fileTotalSize: 4096m
sread_fileNum: 4
sread_fileBlockSize: 1g     # 512m,1g

#sysbench sequential write
sysbench_sequential_write: true
swrite_threads: 2           # 1,2
swrite_fileTotalSize: 4096m
swrite_fileNum: 4
swrite_fileBlockSize: 1g    # 512m,1g

############################ STORAGE PERFORMANCE PARAMETERS END #########################

[bxu1999]

The result.csv is generated based on all the json files under the directory .logs inside the project folder. So can you check the contents of those json files to see if they contain any lines for wisc-ocp4-dev? Does not seem to be possible if you had cleaned up all the old stuffs. Thanks.

[bxu1999]

Could not reproduce this issue. Close it. Reopen it if there is any more info.