[Bug] Storage information not accurate on BTRFS RAID (Type 5)

zwimer commented 8 months ago

Description of the bug

I have a BTRFS raid 5 array. On split view: Most of the HDDs hit bug #935 but the one that does not lists that it is 47.6TiB / 16 TiB full. On non-split view: The pie chart shows 47.6 TiB / 67.3 TiB.

The first one is showing the used of the entire raid array / capacity of the individual HDD. The second is showing used of the entire raid array / sum(capacity of each HDD in the raid array).

Neither shows used / capacity of the raid array.

I'm not sure how to handle this exactly, as df -Thl itself reports 47.6 TiB / 67.3 TiB; I know btrfs fi usage /mnt/foo shows the correct amount of 49 TiB. But even if this is a won't fix, I think at least the split view display value is incorrect.

How to reproduce

Create a BTRFS raid 5 array using 4 HDDs
Run dash

Relevant log output

dashdot  |   storage: [
dashdot  |     {
dashdot  |       size: 2000398934016,
dashdot  |       disks: [ { device: 'nvme0n1', brand: 'Samsung', type: 'NVMe' } ]
dashdot  |     },
dashdot  |     {
dashdot  |       size: 18000207937536,
dashdot  |       disks: [ { device: 'sda', brand: 'Western Digital', type: 'HD' } ]
dashdot  |     },
dashdot  |     {
dashdot  |       size: 18000207937536,
dashdot  |       disks: [ { device: 'sdb', brand: 'Western Digital', type: 'HD' } ]
dashdot  |     },
dashdot  |     {
dashdot  |       size: 18000207937536,
dashdot  |       disks: [ { device: 'sdc', brand: 'Western Digital', type: 'HD' } ]
dashdot  |     },
dashdot  |     {
dashdot  |       size: 18000207937536,
dashdot  |       disks: [ { device: 'sdd', brand: 'Western Digital', type: 'HD' } ]
dashdot  |     },
dashdot  |     {
dashdot  |       size: 8589934592,
dashdot  |       disks: [ { device: 'zram0', brand: 'zram0', type: 'SSD' } ]
dashdot  |     }
dashdot  |   ],

Info output of dashdot cli

yarn run v1.22.19
$ node dist/apps/cli/main.js info
node:internal/modules/cjs/loader:1080
  throw err;
  ^

Error: Cannot find module 'systeminformation'
Require stack:
- /app/dist/apps/cli/apps/cli/src/main.js
- /app/dist/apps/cli/main.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1077:15)
    at Module._resolveFilename (/app/dist/apps/cli/main.js:32:36)
    at Module._load (node:internal/modules/cjs/loader:922:27)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at require (node:internal/modules/cjs/helpers:121:18)
    at Object.<anonymous> (/app/dist/apps/cli/apps/cli/src/main.js:26:18)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/app/dist/apps/cli/apps/cli/src/main.js',
    '/app/dist/apps/cli/main.js'
  ]
}

Node.js v18.17.1
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

What browsers are you seeing the problem on?

Chrome

Where is your instance running?

Linux Server

Additional context

In docker

MauriceNino commented 8 months ago

Hi there! The minor difference in the values is probably due to different units used (GiB vs GB). About the split-view: That is definitely wrong.

Can you please update your application and then provide the output of the following command?

docker exec CONTAINER yarn cli raw-data --storage

zwimer commented 8 months ago

yarn run v1.22.19

Output:

```js const disks = [ { device: '/dev/sda', type: 'HD', name: 'WDC WD181KFGX-68', vendor: 'Western Digital', size: 18000207937536, bytesPerSector: null, totalCylinders: null, totalHeads: null, totalSectors: null, totalTracks: null, tracksPerCylinder: null, sectorsPerTrack: null, firmwareRevision: '0A83', serialNum: '', interfaceType: 'SATA', smartStatus: 'unknown', temperature: null }, { device: '/dev/sdb', type: 'HD', name: 'WDC WD181KFGX-68', vendor: 'Western Digital', size: 18000207937536, bytesPerSector: null, totalCylinders: null, totalHeads: null, totalSectors: null, totalTracks: null, tracksPerCylinder: null, sectorsPerTrack: null, firmwareRevision: '0A83', serialNum: '', interfaceType: 'SATA', smartStatus: 'unknown', temperature: null }, { device: '/dev/sdc', type: 'HD', name: 'WDC WD181KFGX-68', vendor: 'Western Digital', size: 18000207937536, bytesPerSector: null, totalCylinders: null, totalHeads: null, totalSectors: null, totalTracks: null, tracksPerCylinder: null, sectorsPerTrack: null, firmwareRevision: '0A83', serialNum: '', interfaceType: 'SATA', smartStatus: 'unknown', temperature: null }, { device: '/dev/sdd', type: 'HD', name: 'WDC WD181KFGX-68', vendor: 'Western Digital', size: 18000207937536, bytesPerSector: null, totalCylinders: null, totalHeads: null, totalSectors: null, totalTracks: null, tracksPerCylinder: null, sectorsPerTrack: null, firmwareRevision: '0A83', serialNum: '', interfaceType: 'SATA', smartStatus: 'unknown', temperature: null }, { device: '/dev/nvme0n1', type: 'NVMe', name: 'Samsung SSD 980 PRO 2TB ', vendor: 'Samsung', size: 2000398934016, bytesPerSector: null, totalCylinders: null, totalHeads: null, totalSectors: null, totalTracks: null, tracksPerCylinder: null, sectorsPerTrack: null, firmwareRevision: '', serialNum: 'S6B0NL0T615860L', interfaceType: 'PCIe', smartStatus: 'unknown', temperature: null } ] const sizes = [ { fs: '/dev/nvme0n1p3', type: 'btrfs', size: 1998694907904, used: 236039712768, available: 1760865275904, use: 11.82, mount: '/', rw: true }, { fs: 'efivarfs', type: 'efivarfs', size: 262144, used: 54272, available: 202752, use: 21.12, mount: '/mnt/host/sys/firmware/efi/efivars', rw: false }, { fs: '/dev/nvme0n1p2', type: 'ext4', size: 1020702720, used: 388988928, available: 561250304, use: 40.94, mount: '/mnt/host/boot', rw: true }, { fs: '/dev/nvme0n1p1', type: 'vfat', size: 627900416, used: 18206720, available: 609693696, use: 2.9, mount: '/mnt/host/boot/efi', rw: true }, { fs: '/dev/sdc1', type: 'btrfs', size: 72000827555840, used: 51600711196672, available: 2409158074368, use: 95.54, mount: '/mnt/host/mnt/cryo', rw: true } ] const blocks = [ { name: 'nvme0n1', type: 'disk', fsType: '', mount: '', size: 2000398934016, physical: 'SSD', uuid: '', label: '', model: 'Samsung SSD 980 PRO 2TB', serial: 'S6B0NL0T615860L ', removable: false, protocol: 'nvme', group: '', device: '/dev/nvme0n1' }, { name: 'sda', type: 'disk', fsType: 'btrfs', mount: '', size: 18000207937536, physical: 'HDD', uuid: 'a5850a6f-adba-4e34-8fe4-6f191813d5cd', label: '', model: 'WDC WD181KFGX-68', serial: '', removable: false, protocol: 'sata', group: '', device: '/dev/sda' }, { name: 'sdb', type: 'disk', fsType: 'btrfs', mount: '', size: 18000207937536, physical: 'HDD', uuid: 'a5850a6f-adba-4e34-8fe4-6f191813d5cd', label: '', model: 'WDC WD181KFGX-68', serial: '', removable: false, protocol: 'sata', group: '', device: '/dev/sdb' }, { name: 'sdc', type: 'disk', fsType: '', mount: '', size: 18000207937536, physical: 'HDD', uuid: '', label: '', model: 'WDC WD181KFGX-68', serial: '', removable: false, protocol: 'sata', group: '', device: '/dev/sdc' }, { name: 'sdd', type: 'disk', fsType: '', mount: '', size: 18000207937536, physical: 'HDD', uuid: '', label: '', model: 'WDC WD181KFGX-68', serial: '', removable: false, protocol: 'sata', group: '', device: '/dev/sdd' }, { name: 'zram0', type: 'disk', fsType: 'swap', mount: '[SWAP]', size: 8589934592, physical: 'SSD', uuid: '7bcab061-21a3-4241-b6e3-bfe482fa018d', label: 'zram0', model: '', serial: '', removable: false, protocol: '', group: '', device: '/dev/zram0' }, { name: 'nvme0n1p1', type: 'part', fsType: 'vfat', mount: '/mnt/host/boot/efi', size: 629145600, physical: '', uuid: '3671-6189', label: '', model: '', serial: '', removable: false, protocol: 'nvme', group: '', device: '/dev/nvme0n1' }, { name: 'nvme0n1p2', type: 'part', fsType: 'ext4', mount: '/mnt/host/boot', size: 1073741824, physical: '', uuid: '7b45994d-fe45-406d-b7f8-1ec605f6dcd7', label: '', model: '', serial: '', removable: false, protocol: 'nvme', group: '', device: '/dev/nvme0n1' }, { name: 'nvme0n1p3', type: 'part', fsType: 'btrfs', mount: '/etc/hosts', size: 1998694907904, physical: '', uuid: 'b795618d-1f46-4658-b196-69c7d4348e40', label: 'fedora_localhost-live', model: '', serial: '', removable: false, protocol: 'nvme', group: '', device: '/dev/nvme0n1' }, { name: 'sdc1', type: 'part', fsType: 'btrfs', mount: '/mnt/host/mnt/cryo', size: 18000205840384, physical: '', uuid: 'a5850a6f-adba-4e34-8fe4-6f191813d5cd', label: '', model: '', serial: '', removable: false, protocol: '', group: '', device: '/dev/sdc' }, { name: 'sdd1', type: 'part', fsType: 'btrfs', mount: '', size: 18000205840384, physical: '', uuid: 'a5850a6f-adba-4e34-8fe4-6f191813d5cd', label: '', model: '', serial: '', removable: false, protocol: '', group: '', device: '/dev/sdd' } ] ```

zwimer commented 8 months ago

I'm not sure how to handle this exactly, as df -Thl itself reports 47.6 TiB / 67.3 TiB; I know btrfs fi usage /mnt/foo shows the correct amount of 49 TiB. But even if this is a won't fix, I think at least the split view display value is incorrect.

I don't know how to fix this exactly as it also shows up incorrectly in df. I think it's because this is a raid array as mentioned above. The problem is there are 4 HDD's Raid 5'd into one array:

$ btrfs fi df /mnt/my_mount | grep RAID5
Data, RAID5: total=48.96TiB, used=46.79TiB

MauriceNino commented 8 months ago

Okay I see. Unfortunately, there is not much I can fix right now, because the mounted path is directly assigned to the /dev/sdc drive. The best fix for this would be to correctly detect the raid array, but that is somewhat impossible from the given data, because there is normally a raid block in there, to match the data.

Something like this:

{
  name: 'md0',
  type: 'raid1',
  fsType: 'vfat',
  mount: '/mnt/host/boot/efi',
  size: 268369920,
  physical: '',
  uuid: 'F539-5EE9',
  label: 'archlinux-latest-64-minimal:0',
  model: '',
  serial: '',
  removable: false,
  protocol: '',
  group: '',
},

This is missing on your system and I don't know why. I would suggest opening an issue on the systeminformation repository, managed by @sebhildebrandt. Maybe he can add the functionality to support your raid type.

In the meantime, I would suggest using the following variables to get a somewhat desired result:

DASHDOT_FS_DEVICE_FILTER: 'sda,sdb,sdd'
DASHDOT_OVERRIDE_STORAGE_SIZES: 'sdc=72000831750144'

It will look like there are only 2 drives in your system (which is kind of correct, but it will omit the raid info), but they should show the correct sizes.

zwimer commented 8 months ago

@MauriceNino I normally would open an issue but I don't know anything about systeminformation or what it is so I don't feel too comfortable making a bug report. If you do so though, feel free to tag me and I can provide help in the form of df -hl or the like.

MauriceNino commented 7 months ago

@zwimer I have created an issue for you. If you can please comment all the information that might help on there, that would be nice.

What would help:

A command to get the exact size and usage of your RAID (in bytes/bit, not human-readable)
A command to get the name/label of your RAID
Anything you deem important information

MauriceNino / dashdot