Closed madajczyk closed 3 years ago
This is a great idea! I actually went down the route of filtering out removable disks, since I wanted to ignore thumbdrives, but I should have thought about external hard drives connected over USB.
Let me think about this a bit. Eventually I plan on adding support for multi-host collector device grouping (and filtering) so I want to make sure that any include
functionality I add is compatible with that deployment model.
It is not just removable disks. Some array controllers need parameters. so
Smartctl open device: /dev/sdf failed: DELL or MegaRaid controller, please try adding '-d megaraid,N'
optional driver flags will need to be passed especially if it is to be run in server environments (and lets face it multiple disk scenarios means servers)
Otherwise it is a great little tool and I have already installed it on non controller card servers. Cheers!
Scrutiny uses smartctl --scan
to try and auto-detect the correct device type. It has worked for some users with RAID controllers, but yeah in other cases smartctl just doesn't know what to do.
A config file is definitely necessary, I'm working on this off-and-on while waiting for a couple of users to verify the notifications code in a different branch is working. Once that's done I can dedicate some time to this issue.
S'ok. I understand. Thanks for a great project and look forward to seeing improvements when they land. No worries.
Cheers.
EDIT:
FWIW I think I can see where the confusion comes in. 2 sets of devices are listed.
smartctl --scan
/dev/sda -d scsi # /dev/sda, SCSI device
/dev/sdb -d scsi # /dev/sdb, SCSI device
/dev/sdc -d scsi # /dev/sdc, SCSI device
/dev/sdd -d scsi # /dev/sdd, SCSI device
/dev/sde -d scsi # /dev/sde, SCSI device
/dev/sdf -d scsi # /dev/sdf, SCSI device
/dev/bus/0 -d megaraid,14 # /dev/bus/0 [megaraid_disk_14], SCSI device
/dev/bus/0 -d megaraid,15 # /dev/bus/0 [megaraid_disk_15], SCSI device
/dev/bus/0 -d megaraid,18 # /dev/bus/0 [megaraid_disk_18], SCSI device
/dev/bus/0 -d megaraid,19 # /dev/bus/0 [megaraid_disk_19], SCSI device
/dev/bus/0 -d megaraid,20 # /dev/bus/0 [megaraid_disk_20], SCSI device
/dev/bus/0 -d megaraid,21 # /dev/bus/0 [megaraid_disk_21], SCSI device
I suspect that some extra parsing will be needed in these edge cases.
Scrutiny uses
smartctl --scan
to try and auto-detect the correct device type. It has worked for some users with RAID controllers, but yeah in other cases smartctl just doesn't know what to do. A config file is definitely necessary, I'm working on this off-and-on while waiting for a couple of users to verify the notifications code in a different branch is working. Once that's done I can dedicate some time to this issue.
Definitely looks like lots of inconsistencies/special cases - I have an H240 card in HBA mode, and it appears that the only way --scan
identifies any drives is by specifying a driver (eg. smartctl --scan -d sat
). smartctl itself actually recommends using -d ccis,N
for these devices, but both seem to work at the drive device level.
For this case it'd be useful to specify devices and/or drivers in the config file (similar to how you've proposed)...
smarctl --scan
<no results>
smartctl --scan -d sat
/dev/sda -d scsi # /dev/sda, SCSI device
/dev/sdb -d scsi # /dev/sdb, SCSI device
/dev/sdc -d scsi # /dev/sdc, SCSI device
/dev/sdd -d scsi # /dev/sdd, SCSI device
/dev/sde -d scsi # /dev/sde, SCSI device
I'm in a similar boat here - I have a 3ware 9650 RAID card, and it requires the following form to identify the device to smartctl:
smartctl -d 3ware,<N> /dev/twa0
where <N>
is the index of the drive on the RAID card
smartctl --scan
doesn't automatically populate the information needed for this:
/dev/sda -d scsi # /dev/sda, SCSI device
/dev/sdb -d scsi # /dev/sdb, SCSI device
/dev/sdc -d scsi # /dev/sdc, SCSI device
/dev/sdd -d scsi # /dev/sdd, SCSI device
/dev/sde -d scsi # /dev/sde, SCSI device
/dev/sdf -d scsi # /dev/sdf, SCSI device
/dev/sdg -d scsi # /dev/sdg, SCSI device
/dev/sdh -d scsi # /dev/sdh, SCSI device
Hey,
I'm working on this issue right now. Just wanted to give you all a preview of what the collector
config file will look like:
# Commented Scrutiny Configuration File
#
# The default location for this file is /scrutiny/config/collector.yaml.
# In some cases to improve clarity default values are specified,
# uncommented. Other example values are commented out.
#
# When this file is parsed by Scrutiny, all configuration file keys are
# lowercased automatically. As such, Configuration keys are case-insensitive,
# and should be lowercase in this file to be consistent with usage.
######################################################################
# Version
#
# version specifies the version of this configuration file schema, not
# the scrutiny binary. There is only 1 version available at the moment
version: 1
# This block allows you to override/customize the settings for devices detected by
# Scrutiny via `smartctl --scan`
# See the "--device=TYPE" section of https://linux.die.net/man/8/smartctl
# type can be a 'string' or a 'list'
devices:
# example for forcing device type detection for a single disk
- device: /dev/sda
type: 'sat'
# example to show how to ignore a specific disk/device.
- device: /dev/sda
ignore: true
# examples showing how to force smartctl to detect disks inside a raid array/virtual disk
- device: /dev/bus/0
type:
- megaraid,14
- megaraid,15
- megaraid,18
- megaraid,19
- megaraid,20
- megaraid,21
- device: /dev/twa0
type:
- 3ware,0
- 3ware,1
- 3ware,2
- 3ware,3
- 3ware,4
- 3ware,5
Feedback is welcome & appreciated :)
If the devices are being explicitly specified, I think they should be under their own key, to be used instead of or in addition to --scan
results.
devices:
- device: /dev/sda
type: 'scsi'
- device: /dev/sdb
type: 'scsi'
- device: /dev/sdc
type: 'sat'
- device: /dev/sdd
type: 'sat'
I have more testing to verify this makes sense across different versions of smartctl
, but wanted to get you some quick feedback.
that's a good point. I'll change the keyname.
Hey everyone,
I just released a beta version of the Scrutiny docker image with support for overriding the collector device detection.
The instructions for how to create the collector config file, and the new docker image tag are available in the PR description:
https://github.com/AnalogJ/scrutiny/pull/88
All feedback (success & failure) is appreciated :)
I tested it out with my original use case, and the external drive lit up right away:
The configuration file was dead simple to use. In the PR, I also appreciated how you set up the example.collector.yaml file with clear instructions that covered all of the use cases discussed here. Thank you very much!!
Hello. My config is: devices:
and i am getting error: failed to deploy a stack: services.scrutiny.devices.0 must be a string
and i am getting error: failed to deploy a stack: services.scrutiny.devices.0 must be a string
Had the same Problem. The Device overwrite config ist not in the docker-compose.yaml instead the device config musst be in the config folder in the file collector.yaml
First, I'd like to say that Scrutiny is such a great idea. Thank for you for creating this!!
Feature Request It would be awesome if there was a way to pass the device type to Scrutiny so that it can collect details on drives that the smartctl utility can't identify on it's own (due to USB bridge identification limits, etc).
Context
I deployed this in Docker to a little home server I run, and mapped in 3 devices in the config file as such:
When I opened the Scrutiny UI after doing a scan, only the first two internal drives were visible. The last (/dev/sdc) is an external USB drive. I poked around and saw that it can be problematic for smartctl to get the details for external drives. However, if I could find and pass the correct device type for the USB bridge, it might work.
I was able to identify the correct value to pass for my particular drive, which allowed the retrieval of SMART data from my external drive from terminal. If I could provide configuration detail that the device is of type sat, hopefully I'll be able to monitor that external drive as well with Scrutiny. Below are the outputs of various commands I ran as I tested my theory.
sudo smartctl --scan
sudo smartctl -i /dev/sdc
sudo smartctl -i /dev/sdc -d sat