Keyboard inoperable on Atom Pineview platform

[ZigZagJoe]

Issue: Boot into shredOS works fine, but USB support seems partially functional. I can see one USB device (the internal flash drive), but nothing else USB seems to work. I end up at the nwipe screen listing drives, but no action is possible. Pressing a key will flash the capslock LED at me on keypress and beep.

Plugging in a second USB keyboard does nothing - I assume it is not enumerating?

This is an old QNAP ts-439 pro ii NAS. Not a spritely critter, but it suffices for wiping drives.

Here's a lspci from partedmagic.

lspci.txt lspci-v.txt

CPU info: https://ark.intel.com/content/www/us/en/ark/products/49489/intel-atom-processor-d425-512k-cache-1-80-ghz.html

[PartialVolume]

Thanks for reporting, are you using the latest version of Shredos?

There were USB keyboard detection issues on earlier versions that were resolved.

[ZigZagJoe]

Yes, the most recent x64 version.

[PartialVolume]

What do the keyboards show up as in lsusb, when you run partedmagic? What are the make and model of the keyboards? Just checking that they are actual USB keyboards and not PS2 with a PS2 to USB adapter which is a known problem which I'm looking into.

[ZigZagJoe]

I tried two keyboards that are both native USB AFAIK. I lack the ability to get any input in, so boot logs or further debugging seems somewhat difficult....

One is a newer Dell oem chiclet board. The other is some Logitech board.

I'll get details from lsusb in parted tomorrow (and dmesg - maybe it will help).

[PartialVolume]

Thanks, I'm using standard build-in HID drivers, my Dell USB keyboard KB212B is working ok but I guess I'm missing some vendor specific modules.

[PartialVolume]

When you fire up partedmagic it will also be useful to know what they have in /lib/modules/some version number/kernel/drivers/ I've only got crypto thermal and zen, think I might be missing 'hid' from there at least.

[PartialVolume]

@ZigZagJoe I've dug a bit deeper and can confirm it looks like I'm missing some vendor specific drivers, a4tech, apple, belkin, cherry, chicony, cypress, ezkey, Kensington, logitech, Microsoft, Monterey. I'm busy over the next three days but by the end of the weekend or early next week I hope to release a new release with the missing HID drivers.

[ZigZagJoe]

Here is lsusb -t and -v from partedmagic, dmesg and a listing of its rootfs. I also have dmesg and rootfs from a tiny linux distro called bgrescue. This is with 3 keyboards attached: a new dell chiclet type, a logitech K120, and an older dell O EM board.

parted-dmesg.txt lparted-susb-v.txt parted-lsusb-t.txt bgrescue-rootfs.txt bgrescue-dmesg.txt parted-rootfs.txt.gz

[PartialVolume]

@ZigZagJoe Can you check out the latest release, I'm fairly confident this release will fix the problem with your keyboards. I've tested about ten USB keyboards from various vendors including logitech and Microsoft and they all work, including a USB to PS2 adapter that allowed you to connect a PS2 keyboard to the USB port.

https://github.com/PartialVolume/shredos.x86_64/releases/tag/v2020.05.007_x86-64_0.30.001

[ZigZagJoe]

I'll give it a shot on Monday.

Though, I'm not sure it's a hid driver issue - the same keyboards worked fine on another system running shredos. I wonder if it's related to the intel low speed usb controller / root hubs (uhci_hcd) since this chipset is older.

[PartialVolume]

I currently don't have UHCI HCD enabled, I'll add that support over the weekend so you can test it Monday.

[PartialVolume]

@ZigZagJoe New version now available https://github.com/PartialVolume/shredos.x86_64/releases/tag/v2020.05.008_x86-64_0.30.001

[ZigZagJoe]

Confirmed, it now works just fine. Just intel things.

I was able to get shredos building inside of WSL (debian) - let me know if you want the notes from that (packages to install and a couple tweaks).

I'm going to see if I can optimize nwipe a bit, there's no way it should be using as much CPU as it is for just a simple zero-fill routine. Trying to wring as much juice as I can out of this sad little NAS.

[PartialVolume]

I'm going to see if I can optimize nwipe a bit, there's no way it should be using as much CPU as it is for just a simple zero-fill routine. Trying to wring as much juice as I can out of this sad little NAS.

Depends what your using to measure CPU usage. I've found some desktops include IO wait in their calculation so it looks like the CPU is running at 100% most of the time while in fact it's barely using a couple of %. KDEs CPU usage widget is a case in point. Better to use top. The attached picture shows nwipe during a zero wipe.

[PartialVolume]

I was able to get shredos building inside of WSL (debian) - let me know if you want the notes from that (packages to install and a couple tweaks).

Yes definitely interested in seeing some notes on that. Thanks.

[PartialVolume]

My comments about CPU usage when I originally opened it as an issue in nwipe. https://github.com/martijnvanbrummelen/nwipe/issues/158

[ZigZagJoe]

Here's what I get doing a DOD Wipe. Throughput of appx 295mb/s across 4 drives. 2 drives is very slightly faster. Zero method is more or less the same %s - about 10mb/s faster. Doesn't seem to be a reporting issue.

WSL Debian Notes.txt

Your thoughts on larger blocksize are correct: I modified nwipe to multiply block size by 32 (resulting in 128K IO) along with slowing GUI frequency to 1HZ. This resulted in a maximum throughput of approximately 295mb/s from 245mb/s. Every bit counts on this crappy little toaster....

This was done by adding the following to nwipe.c: nwipe_log (NWIPE_LOG_INFO, "'%s' actual block size of %d", c2[i]->device_name,c2[i]->device_stat.st_blksize ); c2[i]->device_stat.st_blksize = c2[i]->device_stat.st_blksize * 32; nwipe_log (NWIPE_LOG_INFO, "'%s' using block size of %d", c2[i]->device_name,c2[i]->device_stat.st_blksize );

[PartialVolume]

Is that 53% from top running inside WSL?. I'm just wondering if that's what is causing the apparent high CPU usage. Do you get the same results when ShredOS is operating standalone without any host OS. Also top is showing 0% idle, where I'm seeing 50+%.

[ZigZagJoe]

No, this is ShredOS running on the NAS. It's a 10 year old NAS with a single core/dual thread atom. Slow as all get out.

WSL was used to build ShredOS.

[PartialVolume]

Ah ok. I've never run ShredOS on a atom processor. I do have a old EEE-PC 700 (intel Celeron-M underclocked) and 900 (Atom N270). I'll give those a try and see if they give similar results, which is very likely. Not sure how you are going to optimize nwipe's code much more. I suppose you could run with no-gui and see how much the GUI overhead effects CPU usage, however any optimisations you do find I'll be interested in.

[ZigZagJoe]

Yeah, this is definitely something that results from an awful CPU. Most aren't terrible enough for this to matter.

But: I find it very strange that it is unable to trivially max out the disk IO when writing a static pattern. The same hardware can easily saturate all 4 disks using the badblocks utility, it suggests some inefficiency somewhere along the line.

[PartialVolume]

@ZigZagJoe Thought I would have a play with the only single core processor I can get my hands on today, a Pentium 4. So I found that, as shown in top the CPU was running at about 25% wiping two hard drives with nwipe --method=zero --verify=off --noblank with the GUI running.

Next I did the same wipe but without GUI, i.e nwipe --method=zero --verify=off --noblank --nogui --autonuke /dev/sda /dev/sdb and top reported the CPU usage of nwipe as approximately 25%. So, without digging too deep, this would indicate that the improvement in performance is down to the larger block size reducing I/O overhead and increasing speed and nothing to do with inefficiency in GUI code, at least on my test system.

I'm thinking of maybe adding an extra option to nwipe so that you can define block size, so it's easier to play with the block size on low end hardware. How does that sound?

[PartialVolume]

Now this is interesting, in my previous comment I mentioned I was wiping two drives. So I thought I would just wipe one drive, expecting to see about half the amount of CPU usage, i.e. 12%. However wiping just /dev/sda I was only using 5% CPU tops. If I then wiped only /dev/sdb then it was using 20% CPU. /dev/sda is a 80GB maxtor 6Y080P0 while /dev/sdb is a 500GB Seagate ST3500418AS, the seagate is a much faster drive, 125MB/s vs 30MB/s for the older Maxtor. What surprised me about this was that even though the Seagate was faster and so would use slightly more CPU I didn't expect it to use so much CPU considering disc I/O is so much slower than the CPU. Also being a single core processor we do have 4 threads running on the one core. One thread for each disc wipe, one thread for the GUI and one thread for the signals. The GUI and signals threads barely use much CPU it would seem, but the wipe thread does look like it could benefit from some optimisation.

[PartialVolume]

Further observations, so in the previous comment I said wiping two drives used 20% CPU, wiping the same two drives, as they near 30% (Maxtor) and 50% (Seagate) the CPU usage has steadily climbed to 50-70% depending upon whether syncing or writing is happening. Simultaneously throughput is dropping slightly. I wasn't expecting such a big change in CPU usage as the wipe progressed.

[PartialVolume]

One final comment, my last test was with DOD short, which explains the jump in CPU usage from 20% to 50+% during the random data wipe. Still I think I will still look at how block size affects CPU usage and throughput. Another thing to go on the list !.

[ZigZagJoe]

My method of testing when the actual hardware is not available is to use either Virtualbox or Hyper-V with performance caps set. The Atom CPU gets about 2GB/s with the following command: dd if=/dev/zero of=/dev/null bs=1M count=10000 - adjust the VM's cap to match and it seems to track well enough.

I didn't see any performance gains above a block size of 128K. 32K seemed to get most of the gains that are possible with my test hardware.

Reducing the UI update speed to 1hz from 10hz seemed to help to some degree (slow framebuffer), but I didn't measure. It definitely made readings a lot less twitchy, though.

My testing agrees that signals and GUI aren't wasting much time. Definitely the wipe threads. if I go from 4 drives to 2, I still see the same total throughput, so we're bound by CPU somewhere in the process.

I'd like to profile a run with 4 drives to see if I can figure out what calls are eating the most time - I think that, for a static pattern, a thread should be spending most of the time waiting on IO if we're writing 128K chunks and doing little else.