search

amarao / sdled

Control fault/locate indicators in disk slots in enclosures (SES devices)
GNU General Public License v2.0
49 stars 20 forks source link

Missing "link" from dev to slot #5

Open NdK73 opened 6 years ago

NdK73 commented 6 years ago

I have a disk server with an LSI MegaRaid SAS controller in JBOD mode:

[   12.716078] megaraid_sas 0000:02:00.0: FW restarted successfully from megasas_init_fw!
[   44.204956] megaraid_sas 0000:02:00.0: FW now in Ready state
[   44.205266] megaraid_sas 0000:02:00.0: firmware supports msix        : (96)
[   44.205269] megaraid_sas 0000:02:00.0: current msix/online cpus      : (12/12)
[   44.205272] megaraid_sas 0000:02:00.0: RDPQ mode     : (enabled)
[   44.205503] megaraid_sas 0000:02:00.0: Current firmware maximum commands: 4064        LDIO threshold: 0
[   44.356968] megaraid_sas 0000:02:00.0: Init cmd success
[   44.384952] megaraid_sas 0000:02:00.0: firmware type : Extended VD(240 VD)firmware
[   44.384957] megaraid_sas 0000:02:00.0: controller type       : MR(2048MB)
[   44.384959] megaraid_sas 0000:02:00.0: Online Controller Reset(OCR)  : Enabled
[   44.384961] megaraid_sas 0000:02:00.0: Secure JBOD support   : Yes
[   44.413121] megaraid_sas 0000:02:00.0: INIT adapter done
[   44.414667] megaraid_sas 0000:02:00.0: pci id                : (0x1000)/(0x005d)/(0x15d9)/(0x0a09)
[   44.414669] megaraid_sas 0000:02:00.0: unevenspan support    : no
[   44.414672] megaraid_sas 0000:02:00.0: firmware crash dump   : no
[   44.414674] megaraid_sas 0000:02:00.0: jbod sync map         : yes
[   44.414678] scsi host0: Avago SAS based MegaRAID driver

The controller is attached to two enclosures:

[   44.415337] scsi 0:0:0:0: Enclosure         LSI      SAS3x28          0601 PQ: 0 ANSI: 5
[   45.065359] scsi 0:0:1:0: Enclosure         LSI      SAS3x40          0601 PQ: 0 ANSI: 5

There are disks on both enclosures, but something seems missing (reduced number of lines):

# encled
ENCLOSURE/SLOT  DEV FAULT_LED   LOCATION_LED
----------------------------------------------------
0:0:1:0/Slot16  ----    fault_off   locate_off
0:0:1:0/Slot06  ----    fault_off   locate_off
0:0:1:0/Slot14  ----    fault_off   locate_off
0:0:1:0/Slot04  ----    fault_off   locate_off
0:0:1:0/Slot22  ----    fault_off   locate_off

and

# sdled
sdled is deprecated. Use encled
DEVICE NAME      STATUS
/dev/sdy          N/A
/dev/sdf          N/A
/dev/sdab         N/A
/dev/sdo          N/A

Is there some way to go from /dev/sd* to enc:slot ? Something I should enable?

Tks.

amarao commented 6 years ago

You have a configuration I never saw in my life. Would you please provide me some debug output? I'll try to understand what's wrong.

# (slashes at the end are important)
find /sys/class/enclosure/
find /sys/bus/scsi/
find /sys/block/sdXXXX/

where sdXXXX is a device name (f.e. sdaz, sdf) for any device which should have led entry in the encled output but was omitted.

Thanks!

NdK73 commented 6 years ago

It's the first "brick" of this kind I configure, so I'm no expert... Thought it would be way easier... Here's the results:

# find /sys/class/enclosure/
/sys/class/enclosure/
/sys/class/enclosure/0:0:1:0
/sys/class/enclosure/0:0:0:0
# find /sys/bus/scsi/
/sys/bus/scsi/
/sys/bus/scsi/drivers_probe
/sys/bus/scsi/devices
/sys/bus/scsi/devices/0:2:18:0
/sys/bus/scsi/devices/0:2:39:0
/sys/bus/scsi/devices/host0
/sys/bus/scsi/devices/target0:2:4
/sys/bus/scsi/devices/0:2:1:0
/sys/bus/scsi/devices/target0:2:11
/sys/bus/scsi/devices/0:2:20:0
/sys/bus/scsi/devices/target1:0:0
/sys/bus/scsi/devices/0:2:11:0
/sys/bus/scsi/devices/target0:2:2
/sys/bus/scsi/devices/0:2:4:0
/sys/bus/scsi/devices/0:2:32:0
/sys/bus/scsi/devices/target0:2:38
/sys/bus/scsi/devices/1:0:0:0
/sys/bus/scsi/devices/0:2:14:0
/sys/bus/scsi/devices/target0:2:18
/sys/bus/scsi/devices/0:2:7:0
/sys/bus/scsi/devices/0:2:35:0
/sys/bus/scsi/devices/target0:2:0
/sys/bus/scsi/devices/target0:2:36
/sys/bus/scsi/devices/target0:0:1
/sys/bus/scsi/devices/host5
/sys/bus/scsi/devices/target0:2:9
/sys/bus/scsi/devices/0:2:17:0
/sys/bus/scsi/devices/0:2:38:0
/sys/bus/scsi/devices/target0:2:16
/sys/bus/scsi/devices/0:2:0:0
/sys/bus/scsi/devices/target2:0:0
/sys/bus/scsi/devices/target0:2:34
/sys/bus/scsi/devices/host3
/sys/bus/scsi/devices/target0:2:7
/sys/bus/scsi/devices/target0:2:14
/sys/bus/scsi/devices/0:2:10:0
/sys/bus/scsi/devices/0:2:3:0
/sys/bus/scsi/devices/target0:2:32
/sys/bus/scsi/devices/host1
/sys/bus/scsi/devices/target0:2:5
/sys/bus/scsi/devices/target0:2:12
/sys/bus/scsi/devices/0:2:13:0
/sys/bus/scsi/devices/0:2:6:0
/sys/bus/scsi/devices/0:0:1:0
/sys/bus/scsi/devices/0:2:34:0
/sys/bus/scsi/devices/2:0:0:0
/sys/bus/scsi/devices/target0:2:20
/sys/bus/scsi/devices/target0:2:3
/sys/bus/scsi/devices/0:2:16:0
/sys/bus/scsi/devices/target0:2:10
/sys/bus/scsi/devices/target0:2:39
/sys/bus/scsi/devices/0:2:9:0
/sys/bus/scsi/devices/0:2:37:0
/sys/bus/scsi/devices/target0:2:19
/sys/bus/scsi/devices/target0:2:1
/sys/bus/scsi/devices/0:2:19:0
/sys/bus/scsi/devices/target0:2:37
/sys/bus/scsi/devices/0:2:2:0
/sys/bus/scsi/devices/host6
/sys/bus/scsi/devices/target0:2:17
/sys/bus/scsi/devices/target0:2:35
/sys/bus/scsi/devices/0:2:12:0
/sys/bus/scsi/devices/0:2:5:0
/sys/bus/scsi/devices/0:0:0:0
/sys/bus/scsi/devices/0:2:33:0
/sys/bus/scsi/devices/target0:0:0
/sys/bus/scsi/devices/host4
/sys/bus/scsi/devices/target0:2:8
/sys/bus/scsi/devices/target0:2:15
/sys/bus/scsi/devices/target0:2:33
/sys/bus/scsi/devices/0:2:15:0
/sys/bus/scsi/devices/0:2:8:0
/sys/bus/scsi/devices/0:2:36:0
/sys/bus/scsi/devices/host2
/sys/bus/scsi/devices/target0:2:6
/sys/bus/scsi/devices/target0:2:13
/sys/bus/scsi/uevent
/sys/bus/scsi/drivers
/sys/bus/scsi/drivers/ses
/sys/bus/scsi/drivers/ses/bind
/sys/bus/scsi/drivers/ses/unbind
/sys/bus/scsi/drivers/ses/0:0:1:0
/sys/bus/scsi/drivers/ses/module
/sys/bus/scsi/drivers/ses/0:0:0:0
/sys/bus/scsi/drivers/ses/uevent
/sys/bus/scsi/drivers/sd
/sys/bus/scsi/drivers/sd/0:2:18:0
/sys/bus/scsi/drivers/sd/0:2:39:0
/sys/bus/scsi/drivers/sd/0:2:1:0
/sys/bus/scsi/drivers/sd/0:2:20:0
/sys/bus/scsi/drivers/sd/bind
/sys/bus/scsi/drivers/sd/0:2:11:0
/sys/bus/scsi/drivers/sd/0:2:4:0
/sys/bus/scsi/drivers/sd/0:2:32:0
/sys/bus/scsi/drivers/sd/1:0:0:0
/sys/bus/scsi/drivers/sd/0:2:14:0
/sys/bus/scsi/drivers/sd/0:2:7:0
/sys/bus/scsi/drivers/sd/0:2:35:0
/sys/bus/scsi/drivers/sd/0:2:17:0
/sys/bus/scsi/drivers/sd/0:2:38:0
/sys/bus/scsi/drivers/sd/0:2:0:0
/sys/bus/scsi/drivers/sd/unbind
/sys/bus/scsi/drivers/sd/0:2:10:0
/sys/bus/scsi/drivers/sd/0:2:3:0
/sys/bus/scsi/drivers/sd/0:2:13:0
/sys/bus/scsi/drivers/sd/0:2:6:0
/sys/bus/scsi/drivers/sd/0:2:34:0
/sys/bus/scsi/drivers/sd/2:0:0:0
/sys/bus/scsi/drivers/sd/0:2:16:0
/sys/bus/scsi/drivers/sd/0:2:9:0
/sys/bus/scsi/drivers/sd/0:2:37:0
/sys/bus/scsi/drivers/sd/module
/sys/bus/scsi/drivers/sd/0:2:19:0
/sys/bus/scsi/drivers/sd/0:2:2:0
/sys/bus/scsi/drivers/sd/0:2:12:0
/sys/bus/scsi/drivers/sd/0:2:5:0
/sys/bus/scsi/drivers/sd/0:2:33:0
/sys/bus/scsi/drivers/sd/uevent
/sys/bus/scsi/drivers/sd/0:2:15:0
/sys/bus/scsi/drivers/sd/0:2:8:0
/sys/bus/scsi/drivers/sd/0:2:36:0
/sys/bus/scsi/drivers_autoprobe
# find /sys/block/sdg/
/sys/block/sdg/
/sys/block/sdg/device
/sys/block/sdg/badblocks
/sys/block/sdg/range
/sys/block/sdg/capability
/sys/block/sdg/subsystem
/sys/block/sdg/inflight
/sys/block/sdg/ext_range
/sys/block/sdg/power
/sys/block/sdg/power/runtime_active_kids
/sys/block/sdg/power/runtime_suspended_time
/sys/block/sdg/power/autosuspend_delay_ms
/sys/block/sdg/power/runtime_enabled
/sys/block/sdg/power/runtime_active_time
/sys/block/sdg/power/control
/sys/block/sdg/power/async
/sys/block/sdg/power/runtime_usage
/sys/block/sdg/power/runtime_status
/sys/block/sdg/dev
/sys/block/sdg/ro
/sys/block/sdg/sdg1
/sys/block/sdg/sdg1/start
/sys/block/sdg/sdg1/subsystem
/sys/block/sdg/sdg1/inflight
/sys/block/sdg/sdg1/power
/sys/block/sdg/sdg1/power/runtime_active_kids
/sys/block/sdg/sdg1/power/runtime_suspended_time
/sys/block/sdg/sdg1/power/autosuspend_delay_ms
/sys/block/sdg/sdg1/power/runtime_enabled
/sys/block/sdg/sdg1/power/runtime_active_time
/sys/block/sdg/sdg1/power/control
/sys/block/sdg/sdg1/power/async
/sys/block/sdg/sdg1/power/runtime_usage
/sys/block/sdg/sdg1/power/runtime_status
/sys/block/sdg/sdg1/dev
/sys/block/sdg/sdg1/ro
/sys/block/sdg/sdg1/partition
/sys/block/sdg/sdg1/stat
/sys/block/sdg/sdg1/size
/sys/block/sdg/sdg1/alignment_offset
/sys/block/sdg/sdg1/trace
/sys/block/sdg/sdg1/trace/end_lba
/sys/block/sdg/sdg1/trace/act_mask
/sys/block/sdg/sdg1/trace/start_lba
/sys/block/sdg/sdg1/trace/enable
/sys/block/sdg/sdg1/trace/pid
/sys/block/sdg/sdg1/holders
/sys/block/sdg/sdg1/holders/md2
/sys/block/sdg/sdg1/uevent
/sys/block/sdg/sdg1/discard_alignment
/sys/block/sdg/stat
/sys/block/sdg/events_poll_msecs
/sys/block/sdg/events_async
/sys/block/sdg/removable
/sys/block/sdg/size
/sys/block/sdg/events
/sys/block/sdg/alignment_offset
/sys/block/sdg/queue
/sys/block/sdg/queue/hw_sector_size
/sys/block/sdg/queue/max_segment_size
/sys/block/sdg/queue/physical_block_size
/sys/block/sdg/queue/discard_max_bytes
/sys/block/sdg/queue/rotational
/sys/block/sdg/queue/iosched
/sys/block/sdg/queue/iosched/target_latency_us
/sys/block/sdg/queue/iosched/target_latency
/sys/block/sdg/queue/iosched/slice_sync_us
/sys/block/sdg/queue/iosched/low_latency
/sys/block/sdg/queue/iosched/fifo_expire_async
/sys/block/sdg/queue/iosched/slice_sync
/sys/block/sdg/queue/iosched/group_idle_us
/sys/block/sdg/queue/iosched/back_seek_max
/sys/block/sdg/queue/iosched/slice_idle
/sys/block/sdg/queue/iosched/slice_async_us
/sys/block/sdg/queue/iosched/slice_idle_us
/sys/block/sdg/queue/iosched/group_idle
/sys/block/sdg/queue/iosched/slice_async_rq
/sys/block/sdg/queue/iosched/fifo_expire_sync
/sys/block/sdg/queue/iosched/back_seek_penalty
/sys/block/sdg/queue/iosched/slice_async
/sys/block/sdg/queue/iosched/quantum
/sys/block/sdg/queue/write_same_max_bytes
/sys/block/sdg/queue/max_sectors_kb
/sys/block/sdg/queue/discard_zeroes_data
/sys/block/sdg/queue/read_ahead_kb
/sys/block/sdg/queue/discard_max_hw_bytes
/sys/block/sdg/queue/nomerges
/sys/block/sdg/queue/max_segments
/sys/block/sdg/queue/rq_affinity
/sys/block/sdg/queue/iostats
/sys/block/sdg/queue/dax
/sys/block/sdg/queue/minimum_io_size
/sys/block/sdg/queue/io_poll
/sys/block/sdg/queue/max_hw_sectors_kb
/sys/block/sdg/queue/add_random
/sys/block/sdg/queue/optimal_io_size
/sys/block/sdg/queue/nr_requests
/sys/block/sdg/queue/scheduler
/sys/block/sdg/queue/discard_granularity
/sys/block/sdg/queue/logical_block_size
/sys/block/sdg/queue/max_integrity_segments
/sys/block/sdg/queue/write_cache
/sys/block/sdg/bdi
/sys/block/sdg/trace
/sys/block/sdg/trace/end_lba
/sys/block/sdg/trace/act_mask
/sys/block/sdg/trace/start_lba
/sys/block/sdg/trace/enable
/sys/block/sdg/trace/pid
/sys/block/sdg/holders
/sys/block/sdg/slaves
/sys/block/sdg/uevent
/sys/block/sdg/integrity
/sys/block/sdg/integrity/write_generate
/sys/block/sdg/integrity/device_is_integrity_capable
/sys/block/sdg/integrity/tag_size
/sys/block/sdg/integrity/read_verify
/sys/block/sdg/integrity/protection_interval_bytes
/sys/block/sdg/integrity/format
/sys/block/sdg/discard_alignment
NdK73 commented 6 years ago

The only "mappings" I can find are:

[   47.056175] sd 0:2:4:0: [sdg] Attached SCSI disk

in dmesg and

lrwxrwxrwx 1 root root  9 mag 22 08:27 pci-0000:02:00.0-scsi-0:2:4:0 -> ../../sdg

in /dev/disk/by-path . Too bad drives in enclosure 0 are seen after disks in enclosure 1, and the slots (third number in SCSI ID) gets assigned with no mention of the enclosure Luckily "holes" in slots are preserved in SCSI IDs, but not in device names. So if I hotplug another disk in slot 21 of enclosure 1, it will receive /dev/sdaf till next reboot, but a stable SCSI ID of 0:2:21:0 . If instead I plug it in slot 0 of enclosure 0, it still gets /dev/sdaf but a SCSI ID of 0:2:24:0 .

flixr commented 4 months ago

Same problem here... any updates/ideas?