sousinha97
commented
1 year ago Hello David,
This was an issue on quisby side, we only support packet / block sizes of 4KiB / 1024 KiB, but obviously this should not be the case, so I have noted this, will push with other changes.
No issue on dataset / pbench side.
On Thu, Oct 26, 2023 at 9:25 PM David Butenhof @.***> wrote:
26 09:26:29 [INFO] ** Starting preprocessing of data ** 26 09:26:29 [INFO] Input Type: InputType.STREAM 26 09:26:29 [INFO] Benchmark Type: BenchmarkName.FIO 26 09:26:29 [INFO] Dataset Name: fio__2023.10.26T09.19.27 26 09:26:29 [INFO] Start extraction... 26 09:26:29 [INFO] Extracting required data... 26 09:26:29 [INFO] IO_DEPTH: <> | NDISKS: <> | NJOBS: <> 26 09:26:29 [WARNING] Invalid row data. Ignoring... 26 09:26:29 [WARNING] Invalid row data. Ignoring... 26 09:26:29 [INFO] Grouping data to a specified format 26 09:26:29 [ERROR] Failed to fetch FIO data... 26 09:26:29 [INFO] !!! PREPROCESSING FAILED !!!
Quisby processing failure. Exception: '1KiB' suggests a KeyError from a getitem. There seems to be a read-1KiB column but no 1KiB.
This is a new fio test run Pravin pushed this morning. The result.csv file is:
iteration_number,iteration_name,iops_sec:client_hostname:localhost-9,iops_sec:client_hostname:localhost-8,iops_sec:client_hostname:localhost-7,iops_sec:client_hostname:localhost-6,iops_sec:client_hostname:localhost-5,iops_sec:client_hostname:localhost-4,iops_sec:client_hostname:localhost-3,iops_sec:client_hostname:localhost-2,iops_sec:client_hostname:localhost-16,iops_sec:client_hostname:localhost-15,iops_sec:client_hostname:localhost-14,iops_sec:client_hostname:localhost-13,iops_sec:client_hostname:localhost-12,iops_sec:client_hostname:localhost-11,iops_sec:client_hostname:localhost-10,iops_sec:client_hostname:localhost-1,iops_sec:client_hostname:all,clat:client_hostname:localhost-9,clat:client_hostname:localhost-8,clat:client_hostname:localhost-7,clat:client_hostname:localhost-6,clat:client_hostname:localhost-5,clat:client_hostname:localhost-4,clat:client_hostname:localhost-3,clat:client_hostname:localhost-2,clat:client_hostname:localhost-16,clat:client_hostname:localhost-15,clat:client_hostname:localhost-14,clat:client_hostname:localhost-13,clat:client_hostname:localhost-12,clat:client_hostname:localhost-11,clat:client_hostname:localhost-10,clat:client_hostname:localhost-1,clat:client_hostname:all,lat:client_hostname:localhost-9,lat:client_hostname:localhost-8,lat:client_hostname:localhost-7,lat:client_hostname:localhost-6,lat:client_hostname:localhost-5,lat:client_hostname:localhost-4,lat:client_hostname:localhost-3,lat:client_hostname:localhost-2,lat:client_hostname:localhost-16,lat:client_hostname:localhost-15,lat:client_hostname:localhost-14,lat:client_hostname:localhost-13,lat:client_hostname:localhost-12,lat:client_hostname:localhost-11,lat:client_hostname:localhost-10,lat:client_hostname:localhost-1,lat:client_hostname:all,slat:client_hostname:localhost-9,slat:client_hostname:localhost-8,slat:client_hostname:localhost-7,slat:client_hostname:localhost-6,slat:client_hostname:localhost-5,slat:client_hostname:localhost-4,slat:client_hostname:localhost-3,slat:client_hostname:localhost-2,slat:client_hostname:localhost-16,slat:client_hostname:localhost-15,slat:client_hostname:localhost-14,slat:client_hostname:localhost-13,slat:client_hostname:localhost-12,slat:client_hostname:localhost-11,slat:client_hostname:localhost-10,slat:client_hostname:localhost-1,slat:client_hostname:all 1,read-1KiB, 122.8000, 122.8000, 123.2000, 122.9000, 122.8000, 122.6000, 123.0000, 122.6000, 122.7000, 122.9000, 122.8000, 123.1000, 123.0000, 122.8000, 123.0000, 122.8000, 1966.0000, 992700000.0000, 992500000.0000, 990100000.0000, 991300000.0000, 992700000.0000, 994000000.0000, 990800000.0000, 992800000.0000, 993900000.0000, 992100000.0000, 993000000.0000, 990300000.0000, 991400000.0000, 992800000.0000, 991700000.0000, 992300000.0000, 1035000000.0000, 1001000000.0000, 1001000000.0000, 998300000.0000, 999500000.0000, 1001000000.0000, 1002000000.0000, 999000000.0000, 1001000000.0000, 1002000000.0000, 1000000000.0000, 1001000000.0000, 998400000.0000, 999500000.0000, 1001000000.0000, 999900000.0000, 1001000000.0000, 1044000000.0000, 8182000.0000, 8179000.0000, 8161000.0000, 8171000.0000, 8182000.0000, 8193000.0000, 8168000.0000, 8183000.0000, 8192000.0000, 8178000.0000, 8188000.0000, 8161000.0000, 8171000.0000, 8182000.0000, 8174000.0000, 8180000.0000, 8152000.0000, 2,read-2KiB, 62.0000, 62.0800, 62.3200, 62.2800, 62.2700, 62.0700, 62.2000, 62.0400, 62.1600, 62.2100, 62.3600, 62.1600, 62.1800, 62.2600, 62.3400, 62.1500, 993.6000, 1958000000.0000, 1958000000.0000, 1955000000.0000, 1953000000.0000, 1953000000.0000, 1957000000.0000, 1954000000.0000, 1959000000.0000, 1958000000.0000, 1954000000.0000, 1951000000.0000, 1956000000.0000, 1957000000.0000, 1956000000.0000, 1951000000.0000, 1958000000.0000, 2048000000.0000, 1975000000.0000, 1974000000.0000, 1971000000.0000, 1969000000.0000, 1969000000.0000, 1973000000.0000, 1970000000.0000, 1975000000.0000, 1974000000.0000, 1970000000.0000, 1967000000.0000, 1972000000.0000, 1973000000.0000, 1972000000.0000, 1967000000.0000, 1974000000.0000, 2064000000.0000, 16170000.0000, 16160000.0000, 16110000.0000, 16110000.0000, 16100000.0000, 16140000.0000, 16110000.0000, 16160000.0000, 16150000.0000, 16120000.0000, 16080000.0000, 16120000.0000, 16140000.0000, 16130000.0000, 16080000.0000, 16160000.0000, 16120000.0000,
Can you tell whether this is a Quisby problem, or a dataset problem? (Although we would like to be able to gracefully handle any fio run without loose exceptions we can't really interpret, which I think means catching all of these inside the pquisby library and providing a diagnosis.)
— Reply to this email directly, view it on GitHub https://github.com/sousinha1997/Quisby/issues/30, or unsubscribe https://github.com/notifications/unsubscribe-auth/AXBEZZAWYXMTBTU7MQPBZXTYBKBXNAVCNFSM6AAAAAA6ROM4OOVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE3DGOBXGQ3TCNY . You are receiving this because you are subscribed to this thread.Message ID: @.***>
dbutenhof
Quisby processing failure. Exception: '1KiB'suggests aKeyErrorfrom a getitem. There seems to be aread-1KiBcolumn but no1KiB.This is a new fio test run Pravin pushed this morning. The
result.csvfile is:Can you tell whether this is a Quisby problem, or a dataset problem? (Although we would like to be able to gracefully handle any fio run without loose exceptions we can't really interpret, which I think means catching all of these inside the pquisby library and providing a diagnosis.)