Check for USB Device Serial Numbers if SMART/ATA doesn't yield results

juliadin commented 3 months ago

Scenario

at my workplace we cycle USB drives a lot as transports for potentially sensitive information. Currently, the nwipe reports and GUI do not contain a lot of information on - for example - SanDisk USB flash drives. They do have identifiable serial numbers though on the USB level that are available in sysfs.

Difficulties

I have not found a clean way to figure out which block device belongs to which usb port, attached device and thereby serial number and it might differ among manufacturers of flash drives.
Not all flash drives that contain serial numbers do contain sensible ones. some just report 0123456789ABCDEF or something - the question though is if this information is worse than showing N/A

Information found so far

considering a SanDisk Cruzer USB flash drive:

if /sys/bus/usb/devices/4-2/serial is the serial of usb device on bus 4, port 2 (in this case 120 character hex string + newline)
then /sys/bus/usb/devices/4-2\:1.0/ is a function of the device
then /sys/bus/usb/devices/4-2\:1.0/host* is/are the SCSI hosts
then /sys/bus/usb/devices/4-2\:1.0/host*/target* are the SCSI targets attached to that host containing information possibily making it possible to link via the SCSI Subsystem address (HOST:BUS:DEVICE:LUN)
because /sys/class/block/sda/device/ points to the same target

Request

Would it be possible to fetch this and make reports potentially more usable for overwriting flash drives?

Disclaimer

(I am aware the the usual wear leveling disclaimers of SSDs still apply and that overwriting and wiping them with nwipe might not be enough to fully erase all information on the drives. I do work with scenarios where destroying most of the data would be nice to keep a papertrail about as well though and therefore it would still be useful)

PartialVolume commented 3 months ago

@juliadin Yes, I can add that. I've checked it out with a Sandisk Ultra, Corsair Padlock3 and a Kingston Data Traveler 3.0. The paths obviously differ but I can link the block device to the serial number found at /sys/bus/usb/devices/4-1/serial. And the serial number looks good.

For other cheap USB devices that I have tested, the same serial number field contains non alphanumeric garbage so the code will need to parse the serial number and discard it if it contains any non alphanumeric characters, but yes, I can see this will work for the Sandisk, Corsair and Kingston USB flash drives I have. And the serial number is exactly 20 characters too.

PartialVolume commented 3 months ago

Added to projects

juliadin commented 3 months ago

I am happy to hear that :). Even though the Sandisk serial numher here is more like 120 characters with two devices differeing in the last few characters after a long block of 000... It might be required to shorten it to the last few bytes if it is longer than expected from ATA/SAS/NVMe to prevent cluttering of UI/Reports with exremely long strings.

dmesg from plugging in a trusty SanDisk Ultra:

[21856.793229] usb 6-1.3.4: new SuperSpeed USB device number 8 using xhci_hcd
[21856.814110] usb 6-1.3.4: New USB device found, idVendor=0781, idProduct=5581, bcdDevice= 1.00
[21856.814114] usb 6-1.3.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[21856.814115] usb 6-1.3.4: Product: Ultra
[21856.814116] usb 6-1.3.4: Manufacturer: SanDisk
[21856.814117] usb 6-1.3.4: SerialNumber: 0501b34e424d1cd6d7a86afdaee1c89fc37d09067cac161eaf9b29a7c986042d2bf100000000000000000000ad745fa000910010815581062f8a6cb84
[21856.814557] usb-storage 6-1.3.4:1.0: USB Mass Storage device detected

PartialVolume commented 3 months ago

This also works for MMC devices, e.g microSD cards too.

 /sys/bus/mmc/devices/mmc0:e624$ ls
block  csd   driver  erase_size  hwrev   name  oemid  preferred_erase_size  scr     ssr        type
cid    date  dsr     fwrev       manfid  ocr   power  rca                   serial  subsystem  uevent
>: /sys/bus/mmc/devices/mmc0:e624$ more serial
0x12e7dbed

Although oddly displayed as hex so converting to decimal ... 317185005 which sort of vaguely looks like the number printed on the card ??0508505DJQ Maybe just coincidence. But even so probably better than just N/A

juliadin commented 3 months ago

That's brilliant. I had hoped that this would be possible but considered it different enough not to request both at once. I am not sure if convertig would be such a good idea though.

It might make it harder for people to identify which card they have wiped when the information accessible through the interface/report vs. the commandline and /sysfs, don't you think?

PartialVolume commented 3 months ago

It might be required to shorten it to the last few bytes if it is longer than expected from ATA/SAS/NVMe to prevent cluttering of UI/Reports with exremely long strings.

Yes, I could truncate that to the last 20 characters which is the length we generally use.

PartialVolume commented 3 months ago

It might make it harder for people to identify which card they have wiped when the information accessible through the interface/report vs. the command line and /sysfs, don't you think?

Yes, you are correct. Reporting it as seen without conversion would also be much easier as different manufactures may do different things.

juliadin commented 3 months ago

If I can help in any way please let me know, otherwise I will check some mmc/emmc devices and USB flash drives I have handy and supply test data if you like.

PartialVolume commented 3 months ago

If I can help in any way please let me know, otherwise I will check some mmc/emmc devices and USB flash drives I have handy and supply test data if you like.

Yes, that would be useful. Thanks.

juliadin commented 3 months ago

I collect the data using

grep -a -H '^' /sys/bus/{usb,mmc}/devices/**/{manufacturer,product,serial,name,idVendor,idProduct} 2>/dev/null| sort

/sys/bus/mmc/devices/mmc0:0001/name:H8G4a
/sys/bus/mmc/devices/mmc0:0001/serial:0x57a5f654
/sys/bus/usb/devices/2-1/idProduct:5583
/sys/bus/usb/devices/2-1/idVendor:0781
/sys/bus/usb/devices/2-1/manufacturer: USB
/sys/bus/usb/devices/2-1/product: SanDisk 3.2Gen1
/sys/bus/usb/devices/2-1/serial:0101c9f9bb0e473bec91ddba85db5a9c67a81feb9c033b26cd6bf521a694e10edeed00000000000000000000805672b4ff0449008355810713b08655

Cheap philips 32G USB2.0 Stick (FM32FD70B) without name in USB data but with name in SCSI data (is correctly identified by nwipe)

/sys/bus/usb/devices/5-1.1.3.4/idProduct:5678
/sys/bus/usb/devices/5-1.1.3.4/idVendor:346d
/sys/bus/usb/devices/5-1.1.3.4/serial:2456151147328746038

juliadin commented 3 months ago

Found some more. Seems I don't have a device with a native SD/MMC bus handy but most USB flash drives I have available seem to report no serial at all or a sensible one (even if it is probably not really unique in some cases)

/sys/bus/usb/devices/6-2/idProduct:5583
/sys/bus/usb/devices/6-2/idVendor:0781
/sys/bus/usb/devices/6-2/manufacturer: USB
/sys/bus/usb/devices/6-2/product: SanDisk 3.2Gen1
/sys/bus/usb/devices/6-2/serial:0501a3c15c21d5f32f9c81a9ac7724b3bdf58fd317d28d7b55bba4a6c40bd947dca9000000000000000000000a568bc3ff0f1510835581077228f4b3

/sys/bus/usb/devices/5-2/idProduct:5406
/sys/bus/usb/devices/5-2/idVendor:0781
/sys/bus/usb/devices/5-2/manufacturer:SanDisk Corporation
/sys/bus/usb/devices/5-2/product:U3 Cruzer Micro
/sys/bus/usb/devices/5-2/serial:0000060328086981

/sys/bus/usb/devices/6-2/idProduct:1666
/sys/bus/usb/devices/6-2/idVendor:0951
/sys/bus/usb/devices/6-2/manufacturer:Kingston
/sys/bus/usb/devices/6-2/product:DataTraveler 3.0
/sys/bus/usb/devices/6-2/serial:1831BFBB3E7AF74079240431

/sys/bus/usb/devices/5-2/idProduct:6100
/sys/bus/usb/devices/5-2/idVendor:0781
/sys/bus/usb/devices/5-2/manufacturer:SanDisk 
/sys/bus/usb/devices/5-2/product:Gemini
/sys/bus/usb/devices/5-2/serial:000000000001

/sys/bus/usb/devices/6-2/idProduct:5581
/sys/bus/usb/devices/6-2/idVendor:0781
/sys/bus/usb/devices/6-2/manufacturer: USB
/sys/bus/usb/devices/6-2/product: SanDisk 3.2Gen1
/sys/bus/usb/devices/6-2/serial:04012ddcd8732ced7321b4984b84c11385e4716b65176cae3d3486fa9b5fb85494290000000000000000000058054b17001c6518815581076528fd62

/sys/bus/usb/devices/5-2/idProduct:5530
/sys/bus/usb/devices/5-2/idVendor:0781
/sys/bus/usb/devices/5-2/manufacturer:SanDisk
/sys/bus/usb/devices/5-2/product:Cruzer
/sys/bus/usb/devices/5-2/serial:200608764204F581B2F6

/sys/bus/usb/devices/6-2/idProduct:1000
/sys/bus/usb/devices/6-2/idVendor:090c
/sys/bus/usb/devices/6-2/manufacturer:SMI Corporation
/sys/bus/usb/devices/6-2/product:USB DISK
/sys/bus/usb/devices/6-2/serial:AA00000000015473

/sys/bus/usb/devices/5-2/idProduct:5567
/sys/bus/usb/devices/5-2/idVendor:0781
/sys/bus/usb/devices/5-2/manufacturer:SanDisk
/sys/bus/usb/devices/5-2/product:Cruzer Blade
/sys/bus/usb/devices/5-2/serial:4C530000300228210551

/sys/bus/usb/devices/5-2/idProduct:55a1
/sys/bus/usb/devices/5-2/idVendor:0781
/sys/bus/usb/devices/5-2/manufacturer:SanDisk
/sys/bus/usb/devices/5-2/product:Cruzer Spark
/sys/bus/usb/devices/5-2/serial:03001831021722163230

/sys/bus/usb/devices/6-2/idProduct:0916
/sys/bus/usb/devices/6-2/idVendor:1f75
/sys/bus/usb/devices/6-2/manufacturer:innostor
/sys/bus/usb/devices/6-2/product:USB 3.0
/sys/bus/usb/devices/6-2/serial:201207220696

/sys/bus/usb/devices/6-2/idProduct:1000
/sys/bus/usb/devices/6-2/idVendor:090c
/sys/bus/usb/devices/6-2/manufacturer:SMI Corporation
/sys/bus/usb/devices/6-2/product:USB DISK
/sys/bus/usb/devices/6-2/serial:AA00000000016060

/sys/bus/usb/devices/5-2/idProduct:1f73
/sys/bus/usb/devices/5-2/idVendor:1c79
/sys/bus/usb/devices/5-2/manufacturer:Unigen Corporation
/sys/bus/usb/devices/5-2/product:PQS1000B8-JN
/sys/bus/usb/devices/5-2/serial:UNIGEN-000005837

/sys/bus/usb/devices/5-2/idProduct:1f73
/sys/bus/usb/devices/5-2/idVendor:1c79
/sys/bus/usb/devices/5-2/manufacturer:Unigen Corporation
/sys/bus/usb/devices/5-2/product:PQS4000B8-JNP
/sys/bus/usb/devices/5-2/serial:UNIGEN-000003507

mdcato commented 3 months ago

Julia, Will you share how you created the output below?

/sys/bus/usb/devices/6-2/idProduct:5583

/sys/bus/usb/devices/6-2/idVendor:0781

/sys/bus/usb/devices/6-2/manufacturer: USB

/sys/bus/usb/devices/6-2/product: SanDisk 3.2Gen1

It obviously follows the sysfs format, but I don’t know the “magic” to get it in the readable format you have below. I’ve tried hdparm -I, lsusb, usbview, etc, but they’re not as concise as what you provided. Since one of the tenants of effective communication is consistency, I want to follow your lead (unless @PartialVolume prefers something else). Thanks!

Mike Cato

From: Julia Mono @.> Sent: Wednesday, March 27, 2024 12:43 To: martijnvanbrummelen/nwipe @.> Cc: Subscribed @.***> Subject: Re: [martijnvanbrummelen/nwipe] Check for USB Device Serial Numbers if SMART/ATA doesn't yield results (Issue #560)

Found some more. Seems I don't have a device with a native SD/MMC bus handy but most USB flash drives I have available seem to report no serial at all or a sensible one (even if it is probably not really unique in some cases)

/sys/bus/usb/devices/6-2/idProduct:5583

/sys/bus/usb/devices/6-2/idVendor:0781

/sys/bus/usb/devices/6-2/manufacturer: USB

/sys/bus/usb/devices/6-2/product: SanDisk 3.2Gen1

/sys/bus/usb/devices/6-2/serial:0501a3c15c21d5f32f9c81a9ac7724b3bdf58fd317d28d7b55bba4a6c40bd947dca9000000000000000000000a568bc3ff0f1510835581077228f4b3

/sys/bus/usb/devices/5-2/idProduct:5406

/sys/bus/usb/devices/5-2/idVendor:0781

/sys/bus/usb/devices/5-2/manufacturer:SanDisk Corporation

/sys/bus/usb/devices/5-2/product:U3 Cruzer Micro

/sys/bus/usb/devices/5-2/serial:0000060328086981

/sys/bus/usb/devices/6-2/idProduct:1666

/sys/bus/usb/devices/6-2/idVendor:0951

/sys/bus/usb/devices/6-2/manufacturer:Kingston

/sys/bus/usb/devices/6-2/product:DataTraveler 3.0

/sys/bus/usb/devices/6-2/serial:1831BFBB3E7AF74079240431

/sys/bus/usb/devices/5-2/idProduct:6100

/sys/bus/usb/devices/5-2/idVendor:0781

/sys/bus/usb/devices/5-2/manufacturer:SanDisk

/sys/bus/usb/devices/5-2/product:Gemini

/sys/bus/usb/devices/5-2/serial:000000000001

/sys/bus/usb/devices/6-2/idProduct:5581

/sys/bus/usb/devices/6-2/idVendor:0781

/sys/bus/usb/devices/6-2/manufacturer: USB

/sys/bus/usb/devices/6-2/product: SanDisk 3.2Gen1

/sys/bus/usb/devices/6-2/serial:04012ddcd8732ced7321b4984b84c11385e4716b65176cae3d3486fa9b5fb85494290000000000000000000058054b17001c6518815581076528fd62

/sys/bus/usb/devices/5-2/idProduct:5530

/sys/bus/usb/devices/5-2/idVendor:0781

/sys/bus/usb/devices/5-2/manufacturer:SanDisk

/sys/bus/usb/devices/5-2/product:Cruzer

/sys/bus/usb/devices/5-2/serial:200608764204F581B2F6

/sys/bus/usb/devices/6-2/idProduct:1000

/sys/bus/usb/devices/6-2/idVendor:090c

/sys/bus/usb/devices/6-2/manufacturer:SMI Corporation

/sys/bus/usb/devices/6-2/product:USB DISK

/sys/bus/usb/devices/6-2/serial:AA00000000015473

/sys/bus/usb/devices/5-2/idProduct:5567

/sys/bus/usb/devices/5-2/idVendor:0781

/sys/bus/usb/devices/5-2/manufacturer:SanDisk

/sys/bus/usb/devices/5-2/product:Cruzer Blade

/sys/bus/usb/devices/5-2/serial:4C530000300228210551

/sys/bus/usb/devices/5-2/idProduct:55a1

/sys/bus/usb/devices/5-2/idVendor:0781

/sys/bus/usb/devices/5-2/manufacturer:SanDisk

/sys/bus/usb/devices/5-2/product:Cruzer Spark

/sys/bus/usb/devices/5-2/serial:03001831021722163230

/sys/bus/usb/devices/6-2/idProduct:0916

/sys/bus/usb/devices/6-2/idVendor:1f75

/sys/bus/usb/devices/6-2/manufacturer:innostor

/sys/bus/usb/devices/6-2/product:USB 3.0

/sys/bus/usb/devices/6-2/serial:201207220696

/sys/bus/usb/devices/6-2/idProduct:1000

/sys/bus/usb/devices/6-2/idVendor:090c

/sys/bus/usb/devices/6-2/manufacturer:SMI Corporation

/sys/bus/usb/devices/6-2/product:USB DISK

/sys/bus/usb/devices/6-2/serial:AA00000000016060

/sys/bus/usb/devices/5-2/idProduct:1f73

/sys/bus/usb/devices/5-2/idVendor:1c79

/sys/bus/usb/devices/5-2/manufacturer:Unigen Corporation

/sys/bus/usb/devices/5-2/product:PQS1000B8-JN

/sys/bus/usb/devices/5-2/serial:UNIGEN-000005837

/sys/bus/usb/devices/5-2/idProduct:1f73

/sys/bus/usb/devices/5-2/idVendor:1c79

/sys/bus/usb/devices/5-2/manufacturer:Unigen Corporation

/sys/bus/usb/devices/5-2/product:PQS4000B8-JNP

/sys/bus/usb/devices/5-2/serial:UNIGEN-000003507

— Reply to this email directly, view it on GitHubhttps://github.com/martijnvanbrummelen/nwipe/issues/560#issuecomment-2023405059, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANGK2PWFSNSJ4OOX3OLQRK3Y2LZKNAVCNFSM6AAAAABFIUMHYCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRTGQYDKMBVHE. You are receiving this because you are subscribed to this thread.Message ID: @.**@.>>

juliadin commented 3 months ago

Sure. I already did in an earlier edit but I guess it didn’t produce an email so it might have slipped past you.

I use the one liner

grep -a -H '^' /sys/bus/{usb,mmc}/devices/**/{manufacturer,product,serial,name,idVendor,idProduct} 2>/dev/null| sort

and copy out the relevant lines. It should capture what I have above for all usb and mmc devices, and I used it in Linux using bash >4. I don’t know if other shells support the curly braces expansion in the same way.

Cheers, Julia

mdcato commented 3 months ago

I took the liberty of putting @juliadin's devices, and ones I have, in a Libre Calc spreadsheet, along with the matching .CSV. I hope this helps prevent hunting, and forms a base for additions as needed. USB-Devices.csv USB-Devices.ods

Firminator commented 2 months ago

It might be required to shorten it to the last few bytes if it is longer than expected from ATA/SAS/NVMe to prevent cluttering of UI/Reports with exremely long strings.

Yes, I could truncate that to the last 20 characters which is the length we generally use.

Yes that's a good approach as serial numbers usually change only on the last characters if you have drives from the same manufactured batch. Usually being the keyword here. There are probably edge case drives out there where this is reversed, i.e. characters/numbers change on the first few characters :chuckle:

martijnvanbrummelen / nwipe