(question) always HPA/DCO, i.e. also on USB-sticks?

[zWhdmB5T]

Dear all,

this is a humble question that I haven't found clear info on, so far. Sorry, if it is quite stupid. But I am interested in and would like to hear about, but haven't. For what HPA/DCO is, I have read https://support.blancco.com/pages/viewpage.action?pageId=15181398 but I am still confused.

I just wiped two USB-sticks using this programm, v.0.37.

After wiping success I got:

Disk Erasure Details
Start time: 2024/07/30 07:21:13
 End time: 2024/07/30 08:41:33
Duration: 01:20:20
 Status:
 ERASED
 See Warning !
Method: HMG IS5 Enhanced
 PRNG algorithm: Isaac64
Final Pass(Zeros/Ones/None): Zeros
 Verify Pass(Last/All/None): Verify Last
*Bytes Erased: 7736072192, (100.00%)
 Rounds(completed/requested): 1/1
HPA/DCO: Unknown
 HPA/DCO Size: Unknown
Errors(pass/sync/verify): 0/0/0
 Throughput: 9566 KB/sec
Information:
 Warning
 HPA/DCO data unavailable, can not determine hidden sector status.

What does this actually mean?

• does my stick support HPA/DC0 or does not?
• does it work by the programm or does it not?
• do I miss some specific tool on my machine?

Thank you for your input! Highly appreciated!

[zWhdmB5T]

Addition, here. Please note: I changed the wiping parameters — but that should not affect HPA/DCO. Please, see below for more details:

~> xdg-su -c /usr/bin/nwipe

(gnomesu:14024): Gtk-WARNING **: 10:25:21.572: gtk_window_set_titlebar() called on a realized window
..
Waiting for wipe thread to cancel for /dev/sda
[2024/07/30 10:25:26]    info: Nwipes config file /etc/nwipe/nwipe.conf exists
[2024/07/30 10:25:26]    info: Reading nwipe's config file /etc/nwipe/nwipe.conf
[2024/07/30 10:25:26]    info: Sucessfully written nwipe config to /etc/nwipe/nwipe.conf
[2024/07/30 10:25:26]    info: Nwipes customer file /etc/nwipe/nwipe_customers.csv exists
[2024/07/30 10:25:26]    info: nwipe 0.37
[2024/07/30 10:25:26]    info: Linux version 6.4.0-150600.23.14-default (geeko@
                               buildhost) (gcc (SUSE Linux) 7.5.0, GNU ld (GNU
                                Binutils; SUSE Linux Enterprise 15) 2.41.0.202
                               30908-150100.7.46) #1 SMP PREEMPT_DYNAMIC Wed J
                               ul  3 00:26:09 UTC 2024 (95fb0f8)
[2024/07/30 10:25:27] warning: Command not found. Install smartmontools !
[2024/07/30 10:25:27]  notice: Found /dev/sda,  USB    , Kingston DataTraveler 3.0, 7864 MB, S/N=
[2024/07/30 10:25:27]    info: /dev/sda, sector(logical)/block(physical) sizes 512/512
[2024/07/30 10:25:27]   error: SG_IO bad/missing sense data /sbin/hdparm --verbose -N /dev/sda 2>&1

[2024/07/30 10:25:27] warning: [UNKNOWN] We can't find the HPA line, has hdparm ouput unknown/changed? /dev/sda
[2024/07/30 10:25:27]    info: hdparm:DCO Real max sectors reported as 1 on /dev/sda
[2024/07/30 10:25:27]    info: NWipe: DCO Real max sectors reported as 1 on /dev/sda
[2024/07/30 10:25:27]    info: libata: apparent max sectors reported as 15360000 with sector size as 512/512 (logical/physical) on /dev/sda
[2024/07/30 10:25:27]    info: func:nwipe_read_dco_real_max_sectors(), DCO real max sectors = 0
[2024/07/30 10:25:27]    info:  
[2024/07/30 10:25:27] warning: Command not found. Install smartmontools !
[2024/07/30 10:25:27]  notice: Found /dev/nvme0n1, NVME    , NVMe Device,  512 GB, S/N=
[2024/07/30 10:25:27]    info: /dev/nvme0n1, sector(logical)/block(physical) sizes 512/512
[2024/07/30 10:25:27]    info:  
[2024/07/30 10:25:27]    info: Automatically enumerated 2 devices.
[2024/07/30 10:25:27] warning: Command not found. Install dmidecode !
[2024/07/30 10:25:27]  notice: Opened entropy source '/dev/urandom'.
[2024/07/30 10:25:27]  notice: hwmon: Module drivetemp loaded, drive temperatures available
[2024/07/30 10:25:27]    info: Temperature limits for /dev/sda, critical=N/A, max=N/A, highest=N/A, lowest=N/A, min=N/A, low critical=N/A. 
[2024/07/30 10:25:27]  notice: hwmon: nvme0n1 has temperature monitoring
[2024/07/30 10:25:27]    info: Temperature limits for /dev/nvme0n1, critical=N/A, max=N/A, highest=N/A, lowest=N/A, min=N/A, low critical=N/A. 
[2024/07/30 10:25:33]    info: Updated PDF_Certificate.PDF_Enable with value DISABLED in /etc/nwipe/nwipe.conf
[2024/07/30 10:27:43]  notice: Program options are set as follows...
[2024/07/30 10:27:43]  notice:   autonuke = 0 (off)
[2024/07/30 10:27:43]  notice:   autopoweroff = 0 (off)
[2024/07/30 10:27:43]  notice:   do not perform a final blank pass
[2024/07/30 10:27:43]  notice:   banner   = nwipe 0.37
[2024/07/30 10:27:43]  notice:   prng     = Isaac64
[2024/07/30 10:27:43]  notice:   method   = Fill With Zeros
[2024/07/30 10:27:43]  notice:   quiet    = 0
[2024/07/30 10:27:43]  notice:   rounds   = 1
[2024/07/30 10:27:43]  notice:   sync     = 100000
[2024/07/30 10:27:43]  notice:   verify   = 1 (last pass)
[2024/07/30 10:27:43]  notice: /dev/sda has serial number 
[2024/07/30 10:27:43]  notice: /dev/sda, sect/blk/dev 512/4096/7864320000
[2024/07/30 10:27:43]  notice: Invoking method 'Fill With Zeros' on /dev/sda
[2024/07/30 10:27:43]  notice: Starting round 1 of 1 on /dev/sda
[2024/07/30 10:27:43]  notice: Starting pass 1/1, round 1/1, on /dev/sda
[2024/07/30 10:59:36]  notice: 7864320000 bytes written to /dev/sda
[2024/07/30 10:59:36]  notice: Verifying pass 1 of 1, round 1 of 1, on /dev/sda
[2024/07/30 10:59:38]  notice: 7864320000 bytes read from /dev/sda
[2024/07/30 10:59:38]  notice: Verified pass 1 of 1, round 1 of 1, on '/dev/sda'.
[2024/07/30 10:59:38]  notice: Finished pass 1/1, round 1/1, on /dev/sda
[2024/07/30 10:59:38]  notice: Finished final round 1 of 1 on /dev/sda

******************************** Error Summary *********************************
!   Device | Pass Errors | Verifications Errors | Fdatasync I\O Errors
--------------------------------------------------------------------------------
       sda |           0 |                    0 |                    0
********************************************************************************

********************************* Drive Status *********************************
!   Device | Status | Thru-put | HH:MM:SS | Model/Serial Number
--------------------------------------------------------------------------------
       sda | Erased | 8205 KB/s | 00:31:55 | Kingston DataTrav/
--------------------------------------------------------------------------------
[2024/07/30 11:03:03] Total Throughput 8205 KB/s, Fill With Zeros, 1R+NB+VL
********************************************************************************

Creating PDF report in .

[2024/07/30 11:03:03]    info: Nwipe successfully completed. See summary table for details.

[PartialVolume]

HPA/DCO

HPA/DCO (Host Protected Area/Drive Configuration Overlay) are generally not supported by any USB flash drive I've ever come across. So Nwipe will always say (with some exceptions, which I'll discuss below) that it cannot determine whether there are any hidden sectors or not as the device will not respond to the low level commands that are issued to determine the status of the HPA/DCO. So in the case of almost every USB flash drive HPA? ??? or in the logs "[UNKNOWN] We can't find the HPA line, has hdparm ouput unknown/changed? /dev/sda" is as expected and nothing to worry about.

However, don't assume that the reason HPA/DCO can't be determined is because it's a USB device. A minority of USB adapters that allow you to plug a hard disk into your USB port do come with a chipset that allows ATA pass through. ATA pass through is where the chipset in the controller will pass ATA commands through to the harddisk. So some USB adapters, if connected to a spinning hard disk do respond to HPA/DCO commands and therefore nwipe will respond with a yes or no as to whether there are hidden sectors.

In regards to which USB adapters support ATA pass through. See this discussion https://github.com/PartialVolume/shredos.x86_64/discussions/128

dmidecode/smartmontools

Yes, you are missing smartmontools and dmidecode

[2024/07/30 10:25:27] warning: Command not found. Install smartmontools !
[2024/07/30 10:25:27] warning: Command not found. Install dmidecode !

You will get more information especially if you are wiping ordinary spinning discs if you install smartmontools. For instance the 2nd and subsequent pages of the PDF will contain smart data courtesy of smartmontools, however having said that smartmontools doesn't generally produce anything much for USB flash drives, however it does for SATA/SAS connected discs and disks connected to USB that are using a USB/SATA adapter that supports ATA pass through (See link above).

dmidecode provides host related information which you are missing in the logs.

I don't know which distro you are running but if a debian based distro then

sudo apt install smartmontools
sudo apt install dmidecode

Write Speed

I noticed that your write speed looked pretty abysmal at 8205 KB/s until I looked up the specs for a Kingston Traveller 8GB which reported a sustained write of a poor 3 MB/s so not that different. This hardware is fairly old I'm guessing?

Hope that helps.

[PartialVolume]

I just noticed from the log you are using SuSe, so I guess installing smartmontools and dmidecode would be something like

zypper install smartmontools
zypper install dmidecode or dmidecode-tool

but I'm just guessing as it's been a very long time since I used Suse.

[zWhdmB5T]

Dear @PartialVolume , thank you so much for sharing the details! Highly appreciated! Interesting to know about USB-sticks in special and the speciality of controllers…

[zWhdmB5T]

Dear @PartialVolume , well…

Yes, I am on SUSE, i.e. openSUSE Leap (15.6).

But look here:

~> whereis smartmontools
smartmontools: /usr/lib/smartmontools /usr/share/smartmontools

and

~> whereis dmidecode
dmidecode: /usr/sbin/dmidecode /usr/share/man/man8/dmidecode.8.gz

as well as

~> export | grep "PATH"
declare -x MANPATH="/usr/local/man:/usr/local/share/man:/usr/share/man"
declare -x PATH="/home/g/bin:/usr/local/bin:/usr/bin:/bin:/snap/bin:/usr/local/bin"
declare -x XDG_SEAT_PATH="/org/freedesktop/DisplayManager/Seat0"
declare -x XDG_SESSION_PATH="/org/freedesktop/DisplayManager/Session0"
declare -x XNLSPATH="/usr/share/X11/nls"

I wonder why /usr/sbin is not included (should it?)… — but this works from a user:

~> sudo nwipe --version
nwipe version 0.37

[PartialVolume]

Re

~> whereis smartmontools
smartmontools: /usr/lib/smartmontools /usr/share/smartmontools

You would need to type

~> whereis smartctl

To locate the executable. If you could run that and let us know the results.

Nwipe searches common locations for smartctl and dmidecode and it maybe that Suse stores it somewhere that's not amongst the paths we search or included in the environment path alias. So if that is the case, once I know where Suse stores the executables I can modify nwipe to look in these new locations as well.

[zWhdmB5T]

So, I have:

~> whereis dmidecode
dmidecode: /usr/sbin/dmidecode /usr/share/man/man8/dmidecode.8.gz

and this seems the only "true" hit for dmidecode referring to https://software.opensuse.org/search?baseproject=ALL&q=dmidecode.

And I have:

~> whereis smartctl
smartctl: /usr/sbin/smartctl /usr/share/man/man8/smartctl.8.gz

as well as

~> whereis smartmontools
smartmontools: /usr/lib/smartmontools /usr/share/smartmontools

[zWhdmB5T]

Oh, almost forgotten, about the poor write speed: yes, I sadly have several of this sticks and they are very slow: that is why I want to wipe them to give them away then.

[PartialVolume]

So, I have:

~> whereis dmidecode
dmidecode: /usr/sbin/dmidecode /usr/share/man/man8/dmidecode.8.gz

and this seems the only "true" hit for dmidecode referring to https://software.opensuse.org/search?baseproject=ALL&q=dmidecode.

And I have:

~> whereis smartctl
smartctl: /usr/sbin/smartctl /usr/share/man/man8/smartctl.8.gz

as well as

~> whereis smartmontools
smartmontools: /usr/lib/smartmontools /usr/share/smartmontools

I see the problem now. Because your 'PATH' environment variable does not include /usr/sbin/, smartctl is never found by nwipe as nwipe only looks in everything in the 'PATH' plus /sbin/ and /usr/bin/.

I therefore need to add the additional location of /usr/sbin/ to the code so it becomes ...

int nwipe_get_smart_data( nwipe_context_t* c )
{
    FILE* fp;

    char* pdata;
    char page_title[50];

    char smartctl_command[] = "smartctl -a %s";
    char smartctl_command2[] = "/sbin/smartctl -a %s";
    char smartctl_command3[] = "/usr/bin/smartctl -a %s";
    char smartctl_command3[] = "/usr/sbin/smartctl -a %s";

I'll make the changes and update nwipe for the next release.

[PartialVolume]

Same goes for dmidecode. Can you check something for me please.

If you type sudo /usr/sbin/dmidecode do you get a load of text output displayed?

One other thing I remember about Suse was that sudo wasn't used much, if at all. You become superuser (root) to run programs like nwipe that require full access. So instead of sudo you would type

su
 [enter root password]

then
nwipe

The root user (superuser) has different environment variables to a ordinary user account and I would expect /usr/sbin/ to then be located in the PATH variable when you are logged in as root.

Ubuntu switched to sudo as a means of allowing the user to escalate privileges as they wanted to disable root access for network logins. At least that's the way I remember it. So unless Suse has followed Ubuntu's way then I would imagine just logging in as su and not sudo would allow nwipe to locate smartctl & dmidecode.

I see you are starting nwipe with xdg-su xdg-su -c /usr/bin/nwipe. Is there any reason why you are using xdg-su and not just su?

[zWhdmB5T]

Same goes for dmidecode. Can you check something for me please.

If you type sudo /usr/sbin/dmidecode do you get a load of text output displayed?

~> sudo /usr/sbin/dmidecode --dump
# dmidecode 3.4
Getting SMBIOS data from sysfs.
SMBIOS 3.1.1 present.
Table at 0x39C35000.
…more…

[zWhdmB5T]

~> sudo smartctl --scan
/dev/nvme0 -d nvme # /dev/nvme0, NVMe device

[zWhdmB5T]

~> sudo smartctl --version
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.4.0-150600.23.14-default] (SUSE RPM)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

smartctl comes with ABSOLUTELY NO WARRANTY. This is free
software, and you are welcome to redistribute it under
the terms of the GNU General Public License; either
version 2, or (at your option) any later version.
See https://www.gnu.org for further details.

smartmontools release 7.4 dated 2023-08-01 at 10:59:45 UTC
smartmontools SVN rev 5530 dated 2023-08-01 at 11:00:21
smartmontools build host: x86_64-suse-linux-gnu
smartmontools build with: C++11, GCC 7.5.0
smartmontools configure arguments: [hidden in reproducible builds]
reproducible build SOURCE_DATE_EPOCH: 1695384000 (2023-09-22 14:00:00)

[zWhdmB5T]

~> sudo dmidecode --version
3.4

[zWhdmB5T]

Please note: One time, I used sudo /usr/bin/<cmd>, the other time sudo <cmd>: both worked.

~> export | grep -w 'PATH'
declare -x PATH="/home/g/bin:/usr/local/bin:/usr/bin:/bin:/snap/bin:/usr/local/bin"

Wondering a bit…

[zWhdmB5T]

I see you are starting nwipe with xdg-su xdg-su -c /usr/bin/nwipe. Is there any reason why you are using xdg-su and not just su?

~> xdg-su --help
xdg-su ? run a GUI program as another user (typically root) after prompting for
that user's password
…more…

I dont start nwipe from console/terminal, but from Xfce GUI starter. I have created a starter in my Xfce menu (GUI).

[PartialVolume]

I just noticed you used whereis to locate the executable. Is the which command available on your system, i.e which smartctl should return the path to smartctl. which is the command nwipe uses. Maybe that's the problem?

Can you try

sudo which smartctl
sudo which /usr/sbin/smartctl

Does the first command not return anything and the second command does?

[zWhdmB5T]

I just noticed you used whereis to locate the executable. Is the which command available on your system, i.e which smartctl should return the path to smartctl. which is the command nwipe uses. Maybe that's the problem?

Can you try

sudo which smartctl
sudo which /usr/sbin/smartctl

Does the first command not return anything and the second command does?

Here you are:

1. ~> which smartctl
   which: no smartctl in (/home/g/bin:/usr/local/bin:/usr/bin:/bin:/snap/bin:/usr/local/bin)

2. 
   ~> sudo which smartctl
   /usr/sbin/smartctl

3. 

~> which /usr/sbin/smartctl /usr/sbin/smartctl

4. 

~> sudo which /usr/sbin/smartctl /usr/sbin/smartctl

Being on openSUSE Leap (15.6), for me, it seems to be like this:
- `<cmd>` is looking for `<cmd>` in `PATH` where `PATH` does include several locations _but not_ `/usr/sbin`
- `sudo <cmd>` is looking for `<cmd>` in PATH where `PATH` does include several locations but not `/usr/sbin` (again…) _plus explicitly_ the location `/usr/sbin` (which makes some sense for the use of `sudo`…)

[zWhdmB5T]

Addition, please have a look.

~> echo $PATH
/home/g/bin:/usr/local/bin:/usr/bin:/bin:/snap/bin:/usr/local/bin
~> su -
Passwort: 
~ # echo $PATH
/sbin:/usr/sbin:/usr/local/sbin:/root/bin:/usr/local/bin:/usr/bin:/bin:/snap/bin:/usr/local/bin
~ # exit
logout

The PATH for my user and for root are different by system.

The tools dmidecode and smartctl mentioned above can be found in the location /usr/sbin mentioned above on my system (openSUSE Leap).

Thank you! Let's see, if it works in the next version of nwipe.

[PartialVolume]

Are you ok building nwipe from source, if so I can make the changes sometime today for you to test.

[PartialVolume]

Here's a link to the build instructions for fedora https://github.com/martijnvanbrummelen/nwipe?tab=readme-ov-file#fedora-prerequisites

Please let me know if any of these instructions are incorrect for openSUSE Leap. Thanks.

[zWhdmB5T]

No, I don't build any stuff locally. I always get the RPMs from openSUSE repository management. (And besides, a few Flatpaks, Snaps, AppImages, though I don't like them that much as RPMs). Sorry. So, no need to hurry. Just, at all. But, don't hurry.

[PartialVolume]

/use/sbin/ path issue fixed by commit 5506c76

[zWhdmB5T]

Nice, thank you!