Closed phillxnet closed 2 months ago
When checking here:
https://www.kernel.org/doc/Documentation/ABI/testing/procfs-diskstats
it seems that additional fields have been added since Kernel 5.5 and depending on the way the parsing occurs that might throw it for a loop (pun intended).
Kernel 5.5+ appends two more fields for flush requests:
== =====================================
19 flush requests completed successfully
20 time spent flushing
== =====================================
Nice find, @Hooverdan96 !
Looks like we should update our unit tests there and then our parsing.
The following should most likely be moved to a dedicated issue but before that, I wanted to briefly get your feedback on considering it further: should we consider and investigate whether the python psutil
library would help us better than having to do our own parsing and try to have it keep up-to-date with kernel versions? I would presume that library would be better at that than we can spend time on, but it remains to be looked at. We would also need to check whether it provides all the metrics we currently use. Thoughts?
using psutil
might be the no-brainer (a bit more effort, but removing the kernel/OpenSUSE flavor dependency).
I think this is where the parsing occurs (only showing the upper code snippet)
Quick Research: here is the disk readout from 'psutil'
https://psutil.readthedocs.io/en/latest/#psutil.disk_io_counters
giving these fields:
Return system-wide disk I/O statistics as a named tuple including the following fields:
read_count: number of reads
write_count: number of writes
read_bytes: number of bytes read
write_bytes: number of bytes written
Platform-specific fields:
read_time: (all except NetBSD and OpenBSD) time spent reading from disk (in milliseconds)
write_time: (all except NetBSD and OpenBSD) time spent writing to disk (in milliseconds)
busy_time: (Linux, FreeBSD) time spent doing actual I/Os (in milliseconds)
read_merged_count (Linux): number of merged reads (see [iostats doc](https://www.kernel.org/doc/Documentation/iostats.txt))
write_merged_count (Linux): number of merged writes (see [iostats doc](https://www.kernel.org/doc/Documentation/iostats.txt))
in the Rockstor parsing it seems to map these fields:
sooo, unless we restructure the widget, I think psutil
might not cover the requirement ...
ok, some more analysis. I attempted a mapping, and there are a few I couldn't really map (or find that they're derived/calculated). So, any refactoring might not be so bad ...
<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">
In psutil | Maps to in Rockstor diskstats | Not mapped in psutil | Comments -- | -- | -- | -- | | | read_count: number of reads | "reads_completed": data[0], | | write_count: number of writes | "writes_completed": data[4], | | read_bytes: number of bytes read | - | | write_bytes: number of bytes written | - | | read_time: (all except NetBSD and OpenBSD) time spent reading from disk (in milliseconds) | "ms_reading": data[3], | | write_time: (all except NetBSD and OpenBSD) time spent writing to disk (in milliseconds) | "ms_writing": data[7], | | busy_time: (Linux, FreeBSD) time spent doing actual I/Os (in milliseconds) | "ms_ios": data[9], | | read_merged_count (Linux): number of merged reads (see [iostats doc](https://www.kernel.org/doc/Documentation/iostats.txt)) | "reads_merged": data[1], | | write_merged_count (Linux): number of merged writes (see [iostats doc](https://www.kernel.org/doc/Documentation/iostats.txt)) | "writes_merged": data[5], | | | | "name": disk, | derived | | "sectors_read": data[2], | | | "sectors_written": data[6], | | | "ios_progress": data[8], | | | "weighted_ios": data[10], | | | "ts": str(datetime.utcnow().replace(tzinfo=utc).isoformat() | derived
On our pending 5.0.9-0 installers we have no activity shown in the Dashboard disk activity widget. This is only for a Tumblweed base and affects all our installer machine/arch targets: i.e. TW on x86_64, ARM64EFI, Pi4. Example command output from a failing TW.x86_64 installer instance:
Example command output from a working 15.6.x86_64 installer instance:
It is assumed currently that we have a parsing issue afoot, and that there is something in the TW output that is throwing our disk activity data retrieval in all TW targets.