PartialVolume / shredos.x86_64

Shredos Disk Eraser 64 bit for all Intel 64 bit processors as well as processors from AMD and other vendors which make compatible 64 bit chips. ShredOS - Secure disk erasure/wipe
Other
1.28k stars 52 forks source link

Black Screen Flashing Underscore #204

Closed Binary-Bear closed 5 months ago

Binary-Bear commented 5 months ago

Hi

I have a very old computer (about 12 years old) which I would like to use for wiping hard drives.

I use a 32GB USB2.0 flash drive (formatted exFAT) to boot from using YUMI (YUMI-exFAT-1.0.2.4)

All other installed ISO’s on this flash drive and setup work ok.

I can use shredos-2021.08.2_21_x86-64_0.32.023.iso which works.

If I try to use any newer versions of shredos I only get to a black screen with a flashing underscore at the top left corner. Nothing happens from then on.

I have edited the grub.cfg file like this:


set default="0"
set timeout="0"
set gfxpayload=800x600x16

menuentry "shredos" {
    linux /boot/bzImage console=tty3 loglevel=3 loadkeys=uk nomodeset noapic nwipe_options="--method=zero --verify=off --noblank --nousb --autopoweroff"
}

I have tried various versions of the above:

nomodeset noapic noapic nomodeset nomodeset noapic

I am unable to get any version of shredos later than “shredos-2021.08.2_21_x86-64_0.32.023.iso” to boot.

Please can you help me use the newer versions?

I notice that version v2023.08.2_25.0_x86-64_0.35 is only available as an .img which does not seem to work at all for me. Will there be an .iso version released?

Thank you.

PartialVolume commented 5 months ago

Last question first..

I notice that version v2023.08.2_25.0_x86-64_0.35 is only available as an .img which does not seem to work at all for me. Will there be an .iso version released?

Yes, if not the current version the next one, which will be soon. Unfortunately I'm going to have to upgrade the system I build shredOS on with a larger hard drive as I'm constantly running out of disc space and need to upgrade so I can build .ISOs along side the .IMGs

If I try to use any newer versions of shredos I only get to a black screen with a flashing underscore at the top left corner. Nothing happens from then on.

Q1. Do you see the ShredOS progress info reach 100% then the blank screen & flashing cursor? Q2. What's the make/model of the system Q3. What's the display card make/model Q4. What's the method of connecting to the monitor, DVI, display port, VGA. How many display connectors are present. Q5. Are there multiple display cards, i'e onboard graphics and PCie adapter?

The way I'd try diagnosing this one is first create a ShredOS stick using the very latest version. Boot it, then wait at least three minutes even if all you have is the flashing cursor. After three minutes pull the USB stick and see if there is a dmesg.txt file on the USB stick. If there is post it's contents here.

If you can't find a dmesg.txt file on the USB stick then I would attempt to use the telnet login for headless systems and see if you get a shredos login. See headless login

If you get a login we then know it's a display issue. If you can login you can then assuming you are running the latest version of ShredOS you will find /dmesg.txt. You will need to manually copy it onto the USB stick USB manual transfer

Binary-Bear commented 5 months ago

Great news about the ISO versions, thank you.

Q1. Do you see the ShredOS progress info reach 100% then the blank screen & flashing cursor?

I do not see any progress info, just the flashing cursor.

Q2. What's the make/model of the system

Home built using a GIGABYTE M720-US3 main-board with a AMD Athlon II X4 620

Q3. What's the display card make/model

AMD Radeon HD 5870

Q4. What's the method of connecting to the monitor, DVI, display port, VGA. How many display connectors are present.

One display connector via VGA to a KVM (this may be the cause of a separate problem)

Q5. Are there multiple display cards, i'e onboard graphics and PCie adapter?

Just one PCI adaptor

I downloaded "shredos-2023.08.2_25.0_x86-64_0.35.img" and made a bootable USB flash drive (SanDisk 32GB) using Rufus.

I did not make any modifications to the grub.cfg I left everything default.

When I booted I could see the progress text.

The average installation speed seemed to be about 650 KiB/s.

The boot process took approximately 6 minutes and 40 seconds in total.

It worked! :0)

I think I might have found another problem.

During one of the times I was testing "shredos-2023.08.2_25.0_x86-64_0.35.img" I switched computers (via the KVM) so I could write my response to you. When I returned to the computer running shredos, after a time longer than 6 minutes and 40 seconds, my monitor went into sleep mode as if it did not receive a VGA signal at all from the computer booting shredos.

I then tried booting the previous version "shredos-2021.08.2_23_x86-64_0.34" from within Ventoy/Yumi to see if I had been stopping shredos from booting too soon in previous tests. I left the black screen with the flashing cursor on for over 9 minutes with no change and no progress text.

Thank you for your assistance.

PartialVolume commented 5 months ago

The boot process took approximately 6 minutes and 40 seconds in total.

That's pretty awful even for USB 2.0. The USB 2.0 controller should be able to achieve higher speeds, however USB 2.0 flash drives can be pretty slow in comparison to the USB 2.0 standard. A quick Google suggests read speeds of 10-25Mbps (1.25MB/s to 3.125MB/s), however I'm actually getting 15MB/s using a cheap chinese USB flash drive, see later posts below. Taking that lower value of 1.25MB/s, as worst case, the theoretical speed in minutes of downloading of ShredOS would be calculated as (SizeOfShredOS/1.25)60 = minutes. (218/1.25)60 = 2.9 minutes. Your speed is half that so about so ~625KB/s as you found. I must admit I don't tent to boot very often using USB 2 but I'm pretty sure I get quicker speeds than that. I'll do a test on an old Dell I've got with USB and let you know what I get in terms of install speed.

PartialVolume commented 5 months ago

I've got a real cheap pack of USB 2 flash drives, cost about £2.50 each I just tested on two systems, both have USB 2 & 3 ports. It doesn't matter which port I use, whether it 2.0 or 3.0 I always get a speed of about 15MiB/s which translates to about 25 seconds to load the kernel i.e progress reaches 100% and nwipe displays at 49 seconds due to the delay caused by shredOS waiting for all USB devices to become available. So you should be able to achieve 49 second boot times using USB 2.0 hardware and a suitable USB flash drive.

PartialVolume commented 5 months ago

For those that are interested in such things, this Chinese USB 2 flash drive has a Chipsbank CBM2199EST USB 2 controller at it's heart. The controller itself is capable of the following speeds.

2 Features USB Interface High-speed USB 2.0 interface Fastest data transfer rate on the market Single-channel mode(16bit): 32MB/s for Read, 20MB/s for Write Single-channel mode(8bit): 26MB/s for Read, 20MB/s for Write Fastest file copy rate on the market. On-the-fly ECC built-in Hardware enhances reliability ECC for NAND flash: 16/25/29/30 bit per page (1 page = 1024 bytes) Special wear leveling algorithm to improve the flash life-time Hardware & Software Data Protection Technology Prevent data corruption even if it is powered off or unplugged during data transfer.

chipsbank_CBM2199EST

PartialVolume commented 5 months ago

Data sheet for this controller CBM2096-Chipsbank.pdf

These are the ones I bought and as per the description I do indeed get a read speed of about 15 MBytes/s https://www.amazon.co.uk/dp/B075GQ58JV?psc=1&ref=ppx_yo2ov_dt_b_product_details

PartialVolume commented 5 months ago

I use a 32GB USB2.0 flash drive (formatted exFAT) to boot from using YUMI (YUMI-exFAT-1.0.2.4)

First time I've heard of Yumi Does it support .img files like Ventoy does?. I had a quick read but could only see .iso files mentioned. If it doesn't support .img then I can see why you need a .iso build.

[Correction] I just read the yumi description fully this time and it does indicate it supports .img.

Drag and Drop ISO Support: You can also create your own storage folders within the YUMI folder on the flash drive and then just drag and drop your ISO, IMG, WIM, VHD(x), VDI.vtoy, and EFI files into those folders on the USB flash drive. During startup, the system will add entries for discovered items. Stored files can be larger than 4GB.

I may have to give that a try sometime.

Binary-Bear commented 5 months ago

The USB flash drive I used was a SanDisk 32GB Ultra Flair suitable for USB3.

This SanDisk works very quickly in windows even on USB2 connections. It just seems to be slow when I try to boot shredos via Rufus.

Yes YUMI is very good I hope you enjoy using it. It would be great if I could use YUMI to boot shredos.

Now I can boot the .img with Rufus I have something to test. I have noticed some issues which might very well be my fault but I wondered if I could ask you on this thread rather than distributing them across your whole forum :)

I presume as v2023.08.2_25.0_x86-64_0.35 worked when I used Rufus that it will work on Ventoy/Yumi when the ISO is released?

I removed the HPA from a Seagate hard drive using hdparm however shredos indicated that the HPA remained by displaying [HS? YES]

When the user toggles "b" to change the background colour the white text is quite faint when the user selects the black background. Please could the white text be enhanced a little bit?

Please could the user have the option to set a time until shredos powers off the display? The display could be powered up by the mouse or keyboard. Much of the time shredos is running the user does not need to watch.

I am really looking forward to being able to delete/overwrite the HPA from within shredos which I understand may be available in the next version.

I noticed that some of the improvements to the latest shredos were related to hard drive temperature monitoring. It would be nice if the user could see the temperature of the drives in real time.

Thank you for shredos it's very reassuring to be able to securely wipe hard drives.

PartialVolume commented 5 months ago

The USB flash drive I used was a SanDisk 32GB Ultra Flair suitable for USB3. This SanDisk works very quickly in windows even on USB2 connections. It just seems to be slow when I try to boot shredos via Rufus.

I have a Sandisk 32GB Ultra Flair arriving tomorrow to test it's performance on USB 2, it will be interesting to see how it compares with yours in terms of speed.

Maybe your flash drive has bad sectors in certain places so some OS's boot fine while others don't due to their location on the flash drive. If I suspected the flash drive has issues I would run ShredOS with a DOD-short wipe on it with verification on all passes. It would show any bad blocks or faults especially if you monitor throughput and find it drastically drops off during the wipe. If ShredOS reports a I/O error or the throughput is unusually low, or any pass/fdatasync or verification errors are reported, I would destroy the drive. It's just not worth the hassle working with hardware that's starting to fail. I would also test it on a second computer, just in case there is something wrong with your computers USB hardware.

Now I can boot the .img with Rufus I have something to test. I have noticed some issues which might very well be my fault but I wondered if I could ask you on this thread rather than distributing them across your whole forum :)

I presume as v2023.08.2_25.0_x86-64_0.35 worked when I used Rufus that it will work on Ventoy/Yumi when the ISO is released?

Yes, .img & .iso both work on Ventoy and according to Yumi's description .img and .iso should also work with ShredOS. Remember that you will still need to disable secure boot even though Ventoy and Yumi may apparently work with secure boot. This might be why you can't get ShredOS .img to work?

I removed the HPA from a Seagate hard drive using hdparm however shredos indicated that the HPA remained by displaying [HS? YES]

After using hdparm to remove HPA always power cycle the drive and reboot the OS/ShredOS, otherwise libata will report the original size. Also power cycling the drive and retesting using hdparm as on some drives this may fail. I would need to see the verbose log from nwipe, if you still have ShredOS report the HPA is still present ALT F2 to switch the the virtual terminal. type nwipe -v -q and once nwipe drive selection appears use Control C to exit nwipe. A verbose (-v) and anonymised (-q) log will be created in the root directory. If you could post that log here.

When the user toggles "b" to change the background colour the white text is quite faint when the user selects the black background. Please could the white text be enhanced a little bit?

Yes, however rather than replace the existing faint text a extra format would be introduced where the text is brighter.

Please could the user have the option to set a time until shredos powers off the display? The display could be powered up by the mouse or keyboard. Much of the time shredos is running the user does not need to watch.

I don't have any problem with that being added, but it would be low priority for me, I'm happy to accept PR requests if somebody else wants to work on that.

I am really looking forward to being able to delete/overwrite the HPA from within shredos which I understand may be available in the next version.

Yes, HPA deletion will require the host computer (bios) accepts ShredOS shutting down the system and powering the system back up after 5 seconds. Most systems will probably work fine some may require a setting in bios is set.

I noticed that some of the improvements to the latest shredos were related to hard drive temperature monitoring. It would be nice if the user could see the temperature of the drives in real time.

The user can see the temperature in real time in both drive selection and during the wipe. That assumes that the drive has temperature support. USB drives, flash drives and very old drives won't show temperature. Some USB drive temperature support may be added in the future but that also depends on the capability of the USB-SATA/IDE adapter being used.

Thank you for shredos it's very reassuring to be able to securely wipe hard drives.

No problem, glad it's useful.

PartialVolume commented 5 months ago

@Binary-Bear Regarding the Sandisk 32GB Flair (USB 3.0) USB drive, which I received today, these are the boot times I get for USB 3 and USB 2. ShredOS is written to the Sandisk with the dd command (Linux)

  1. USB2: 20 seconds to load the shredos.img image and a total of 43 seconds to reach the nwipe display.
  2. USB3: 16 seconds to load the shredos.img image and a total pf 33 seconds to reach the nwipe display.

So your 6 minutes would suggest you have a faulty Sandisk or a faulty USB port/controller on your computer or Rufus is doing something really weird. I'd probably try a different USB stick first, then use the windows version of dd to write to the stick to rule out Rufus and finally if none of that works it would suggest there is something not right with the USB ports on your computer.

PartialVolume commented 5 months ago

@Binary-Bear However, if the shredos.img image loads in 16-20 seconds but then you get a blank screen for 5.5 minutes it would suggest some other issue in which case I would need to see the dmesg.txt file that you will find on the USB stick after you have control C out of nwipe.

PartialVolume commented 5 months ago

@Binary-Bear Sorry, correction to my last post, not the nwipe log file, I meant I would need to see dmesg.txt file that can be found in the root folder of the USB stick.

Binary-Bear commented 5 months ago

Hi PartialVolume

Thank you for all your research!

I am just a simple home PC user and not very technical which I appreciate must be very frustrating for you. However I like the idea of shredos which is why I am eagerly watching its development.

I have been doing a lot of testing today starting very early this morning!

I have performed various speed tests on the SanDisk 32GB Ultra Flair and everything seems ok. I have also run shredos with a DOD-short wipe on with verification on all passes which reviled no errors which took well over 4 hours. So the SanDisk seems ok.

I have also used another USB flash drive to boot from which booted at almost exactly the same speed as the SanDisk approximately 650 KiB/s. Both flash drives were inserted into several different USB sockets on the main-board with no change.

So I would suggest that the flash drives are not the problem.

However I have noticed something. To carry out the test you recommended "DOD-short wipe on with verification on all passes" I loaded the original version of shredos as my custom version (mentioned in my first post) contained --nousb and also critically I had set gfxpayload=800x600x16.

I tried to boot the original version of shredos-2021.08.2_23_x86-64_0.34.iso and I noticed "Welcome to Grub" together with the same flashing cursor at the top left. I think this is the stage I am getting to with any version of shredos later than shredos-2021.08.2_21_x86-64_0.32.023.iso it was just that I wasn't seeing "Welcome to Grub" presumably because of gfxpayload=800x600x16.

I hope this helps you work out why I cannot boot the later versions of shredos.

Thanks again for your patience.

I have attached a copy of dmesg.txt which was created when using the SanDisk. dmesg.txt

Binary-Bear commented 5 months ago

With regard to my issues with HPA.

I did as you suggested. After using hdparm to remove HPA I powered off the PC. I then rebooted and retested using hdparm which reported that HPA was again enabled.

So either hdparm is not working properly for me or my hard drive is somehow resistant to modification.

The hard drive is an old Seagate Barracuda 7200 80GB.

When using shredos on this same hard drive I do not see any temperature reports. I presume this again is due to the hard drive.

Thank you.

Binary-Bear commented 5 months ago

On another test I used the same hardware as above but instead booted gparted which installed at an average rate of about 18.7 MB/s.

I have tested another hard drive (Western Digital Black 250GB) and noticed shredos-2023.08.2_25.0_x86-64_0.35 does not report the temperature on this model.

The (Western Digital Black 250GB) has the ATA secure erase feature. I wondered if it would please be possible for shredos to provide this as an option within the Methods menu?

PartialVolume commented 5 months ago

I have performed various speed tests on the SanDisk 32GB Ultra Flair and everything seems ok. I have also run shredos with a DOD-short wipe on with verification on all passes which reviled no errors which took well over 4 hours. So the SanDisk seems ok.

If i have my calculation correct, that's 3 write + 3 read passes = 192000000000 bytes Divide by number of seconds in 4 hours 192000000000/144000 = 13333333 = ~ 13Mbytes/sec Seems reasonable for USB 2

I tried to boot the original version of shredos-2021.08.2_23_x86-64_0.34.iso and I noticed "Welcome to Grub" together with the same flashing cursor at the top left. I think this is the stage I am getting to with any version of shredos later than shredos-2021.08.2_21_x86-64_0.32.023.iso it was just that I wasn't seeing "Welcome to Grub" presumably because of gfxpayload=800x600x16.

gfxpayload is unnecessary for all recent versions except when using nomodeset which prevents switching to DRM (direct rendering manager). From version v2021.08.2_22_x86-64_0.32.023 ShredOS switches during boot from simple framebuffer for display to using Linux's DRM graphics. gfxpayload has no control over resolution in DRM. So there is not much point adding gfxpayload in grub. I should really mention this in the README.md.

I have attached a copy of dmesg.txt which was created when using the SanDisk.

Having looked at dmesg, I don't see any issues, it seems to switch from framebuffer to DRM without any issues and the kernel and drivers all seem to be loaded within 7 seconds. So I'm unsure whether the long delay you see is happening before the kernel boots, i.e being handled by grub, or after the kernel has booted but there is some graphics issue so nothing is displayed on screen.

So either hdparm is not working properly for me or my hard drive is somehow resistant to modification. The hard drive is an old Seagate Barracuda 7200 80GB. When using shredos on this same hard drive I do not see any temperature reports. I presume this again is due to the hard drive.

Yes, I suspect both the HPA & temperature issues are both drive related due to it's vintage. I'm assuming you are connecting this to an IDE connector on the motherboard and not via a USB bridge?

I have tested another hard drive (Western Digital Black 250GB) and noticed shredos-2023.08.2_25.0_x86-64_0.35 does not report the temperature on this model.

Are you connecting this direct to a SATA port on the motherboard or via a USB adapter? Most USB adaptors don't support ATA pass through so temperature data may not be available via the HWMON library. See #128

I wondered if it would please be possible for shredos to provide this as an option within the Methods menu?

That's the plan :-)

Binary-Bear commented 5 months ago

Having looked at dmesg, I don't see any issues, it seems to switch from framebuffer to DRM without any issues and the kernel and drivers all seem to be loaded within 7 seconds. So I'm unsure whether the long delay you see is happening before the kernel boots, i.e being handled by grub, or after the kernel has booted but there is some graphics issue so nothing is displayed on screen.

Hmm... so how can I help find out what is going wrong?

Yes, I suspect both the HPA & temperature issues are both drive related due to it's vintage. I'm assuming you are connecting this to an IDE connector on the motherboard and not via a USB bridge?

The Seagate drive is probably 10 years old at least and I connect via internal SATA direct to the main-board.

Are you connecting this direct to a SATA port on the motherboard or via a USB adapter? Most USB adaptors don't support USB pass through so temperature data may not be available via the HWMON library. See #128

All hard drives I am testing are connected direct to the main-board via internal SATA.

That's the plan :-)

Looking forward to it!

PartialVolume commented 5 months ago

Hmm... so how can I help find out what is going wrong?

I think you need to rule out Rufus as the cause of the delay. Rufus doesn't do a block by block copy like dd, it changes the boot front end and replaces ShredOS EFI code. So I would download dd for windows from here and use this command to create the ShredOS USB stick.

ddrelease64 --progress if=shredos-2023.08.2_25.1_x86-64_0.35_20240101.img of=\\.\E: bs=1M

You will need to change the ShredOS filename to whatever version you are using and you will also need to change the E: to whatever the windows drive letter is allocated for the USB stick when you plug it in. Makesure you get that drive letter correct as it will wipe out your C: drive if you type C: there !!.

This will do a block by block copy, hopefully just like dd under Linux .. having just been playing with Windows it reminds me how much I love Linux for doing this sort of thing :-) Once it is finished try booting it.

PartialVolume commented 5 months ago

After having written those notes and then having tested the resultant USB stick having used ddrelease64 to create it on Windows, it completely failed and resulted in ending up at the grub prompt...

You haven't thought about loading up a computer with a very nice KDE Neon Linux distribution where this stuff is installed as standard ? How I love Linux.. :-)

Binary-Bear commented 5 months ago

I think you need to rule out Rufus as the cause of the delay.

I think we can already rule out Rufus as I have the same problem when using Venoty/YUMI.

You haven't thought about loading up a computer with a very nice KDE Neon Linux distribution where this stuff is installed as standard ?

I would do to help diagnose the problem but this would not be useful to others. I am worried that other people might be tying shredos for the first time and having the same problems I am having but they might not be bothered to report it and just think shredos doesn't work.

Today I have tried different main-boards, flash drives, versions of shredos and hard drives.

I am going to do some more tests and report back tomorrow.

Thanks for your help.

Firminator commented 5 months ago

For what it's worth mentioning I also noticed that the latest ShredOS boots much slower compared to the last release. It used to be 30 to 45 seconds from POST to usable and now it takes like 4 minutes.

PartialVolume commented 5 months ago

For what it's worth mentioning I also noticed that the latest ShredOS boots much slower compared to the last release. It used to be 30 to 45 seconds from POST to usable and now it takes like 4 minutes.

Do you think it might be related to this #169 fixed by adding noapic to the kernel command line? i.e hangs at clocksource: switched to clocksource TSC, seen by switching to loglevel 7. The timings in dmesg that @Binary-Bear provided would suggest it's not that.

I'm testing on all my systems, but so far am seeing normal boot times of under 50 secs. Is this delay occurring prior to the kernel loading, during or after?

Binary-Bear commented 5 months ago

Hi Firminator

Can I ask if you are using Ventoy/Yumi?

@PartialVolume

My results today using the same hardware as described above. I am using latest versions of Venoty/Yumi exFAT with a fresh install on both flash drives. Testing only original versions of shredos (no modifications) All other iso's I have installed on Ventoy/Yumi work as expected.

shredos-2021.08.2_21_x86-64_0.32.023.iso and shredos-2021.08.2_23_x86-64_0.34.iso both boot at around 650 Kib/s

shredos-2021.08.2_21_x86-64_0.32.023.iso Booting using "Normal Mode" often gets stuck at bzImage. Sometimes gets stuck at "uncompression error system halted" Booting using "Grub2 mode" works and is usable.

shredos-2021.08.2_23_x86-64_0.34.iso Booting using "Normal Mode" Hangs at "Decompressing Linux... Parsing ELF... Performing relocations... done Booting the Kernel" Booting using "Grub2 mode" Hangs at "Wrong EFI loader signature".

shredos-2023.08.2_25.0_x86-64_0.35.img Instantly goes to black screen showing Grub2 command options.

As mentioned above my concern is that many people use Ventoy/Yumi and they may try shredos and just assume it doesn't work. These users may never return.

Firminator commented 5 months ago

Can I ask if you are using Ventoy/Yumi?

No. I'm using MintStick (which is using dd) or dd directly from CLI depending which device I'm working from.

I'm testing on all my systems, but so far am seeing normal boot times of under 50 secs. Is this delay occurring prior to the kernel loading, during or after?

During 'Booting ShredOS' on the left showing the speed on the top right. Something is slowing down USB access (or uncompressing the compressed image?) during that process compared to the last version is what I suspect.

PartialVolume commented 5 months ago

@Firminator & @Binary-Bear can you run lspci on the hardware that slow boots ShredOS. I want to see what USB hardware you have so I can see if I can reproduce it. Thanks.

Binary-Bear commented 5 months ago

I have set up an entirely different computer.

I loaded shredos-2023.08.2_25.0_x86-64_0.35.img using Rufus.

When I boot I still get the same speed of around 650 KiB/s.

Unfortunately shredos does not boot completely on this second computer it freezes on "common interrupt: 1.55 No irq handler for vector"

lspci ???

Firminator commented 5 months ago

Either ALT+F2 or ALT+F3 or ALT+F4 brings up another console. I forgot which one. There you type in lspci (list PCI devices) or lsusb (list USB devices) or lsblk (list block-level devices aka storage devices like HDDs, SSDs).

PartialVolume commented 5 months ago

@Binary-Bear

common interrupt: 1.55 No irq handler for vector

If you Google that you will find it occurs in various other Linux distros, not just ShredOS. It seems to be very system specific as often these things are. As to what causes it, the error itself doesn't help much. Can you take a photo of the screen when you get this error, assuming it is preceded by some informative text. If there is nothing useful displayed, edit the kernel command line in grub.cfg (choose the /EFI/BOOT/grub.cfg or /boot/grub/grub.cfg depending upon whether you are booting UEFI or legacy) if unsure edit both and change the loglevel from 3 to 7. This will give you a verbose log output to the terminal and may help identify the driver or hardware that is causing the problem. Loglevel should never be left at 7 on a normal working system as the verbose output of the kernel will corrupt the nwipe display. Loglevel 7 is only for diagnosing boot issues.

Unfortunately, I suspect this is only something that occurs on a specific hardware configuration, possibly even a specific bios version so it may magically start working when the buildroot/kernel version is bumped. So far I'm not seeing the interrupt error or 620KB boot speeds on any of my ten systems, slowest I've seen so far is ~3Mbyte/s on a old Sony Vaio. So that's the importance of gathering as much information about your systems as possible using as @Firminator suggests lspci, lsusb and lsblk. These linux programs (installed as default in ShredOS) are incredibly useful at identifying specifically what hardware you are running. BTW it's ALT-F2 for the second terminal window, ALT-F1 for the default nwipe window, and ALT-F3 for a second terminal window where you login as root no password.

PartialVolume commented 5 months ago

@Firminator & @Binary-Bear Just to clarify something, the slow 620kb loading of shredos.img cannot be anything to do with the shredos kernel, i.e Linux as that's whats being loaded and hasn't started yet. Grub being responsible for loading shredos, decompressing and starting it.

So the things that could be causing this are, in no particular order:

  1. A bug in grub
  2. A bug in the bios
  3. A faulty USB stick
  4. Faulty USB hardware on the system.
  5. Something else?

As both UEFI and legacy both use Grub to load the shredos image, extract it and run it, I'm assuming on a system that uses both legacy & UEFI you are seeing the slow boot whichever method you use?

PartialVolume commented 5 months ago

shredos-2021.08.2_21_x86-64_0.32.023.iso Booting using "Normal Mode" often gets stuck at bzImage. Sometimes gets stuck at "uncompression error system >halted" Booting using "Grub2 mode" works and is usable.

shredos-2021.08.2_23_x86-64_0.34.iso Booting using "Normal Mode" Hangs at "Decompressing Linux... Parsing ELF... Performing relocations... done Booting >the Kernel" Booting using "Grub2 mode" Hangs at "Wrong EFI loader signature".

shredos-2023.08.2_25.0_x86-64_0.35.img Instantly goes to black screen showing Grub2 command options.

I don't think I've seen so many errors on one system. Firstly, what do you mean by normal mode, and what's Grub2 mode?

Booting using "Normal Mode" Hangs at "Decompressing Linux... Parsing ELF... Performing relocations... done Booting the Kernel"

That sounds like the kernel booted but there was an issue switching from simple framebuffer to DRM. Does adding nomodeset make any difference.

Booting using "Normal Mode" often gets stuck at bzImage.

What's normal mode and getting stuck at bzimage? Does that mean you don't see the percentage progress go from 1-100%? i.e it doesn't even start loading the kernel? Could be a corrupt .img or .iso, bad USB hardware or stick. bios issues.

Sometimes gets stuck at "uncompression error system halted" Now that really does sound like a hardware error or corrupt image file. Maybe even worth doing a memtest overnight to look for bad system RAM. Also bad USB stick, USB controller. Even a bad system PSU with marginal 5v and 3.3v outputs. How old is this system?

How are you copying the .img or .iso onto the Ventoy/Yumi disk? Could the .img being corrupted in that process? Once copied to USB stick verify with sha1sum to match the checksum in release details, to verify the integrity of the copy.

shredos-2023.08.2_25.0_x86-64_0.35.img Instantly goes to black screen showing Grub2 command options.

Again, the ShredOS image isn't even being loaded, ShredOS hasn't even been started yet. This is a frontend loader issue, that could be caused by hardware or yumi/grub/bios.

You seem to have many different issues on this system but with the exception of one item they all have one theme, they all seem to occur before the ShredOS image has even been loaded fully by grub.

As mentioned above my concern is that many people use Ventoy/Yumi and they may try shredos and just assume it doesn't work. These users may never return.

Thank you for the concern. The latest version of ShredOS was released two months ago and has been downloaded 17,000 times. I would have thought that if these errors were typical on most systems they would have been reported long before now, however you never know, maybe the majority don't have the time or inclination to report issues although that's certainly not what I've found in the past. I think these loader errors are very specific to either your system or are hardware specific as in the interrupt issue or somehow the shredos image is getting corrupted before it can even be uncompressed.

Binary-Bear commented 5 months ago

@PartialVolume

I don't think I've seen so many errors on one system.

This is on two separate computers (both Gigabyte main-boards several years apart) using two separate flash drives.

Firstly, what do you mean by normal mode, and what's Grub2 mode?

When using Ventoy/Yumi the user gets the option to boot in normal mode or Grub2 mode. I am using Venoty/Yumi and Rufus on each flash drive. Interestingly all other iso's work ok which is why I assumed this might be a shredos issue.

That sounds like the kernel booted but there was an issue switching from simple framebuffer to DRM. Does adding nomodeset make any difference.

No, it made no difference for me.

What's normal mode and getting stuck at bzimage?

When using Ventoy/Yumi the user gets the option to boot in normal mode or Grub2 mode. bzimage is displayed along with the progress percentage.

How are you copying the .img or .iso onto the Ventoy/Yumi disk? Could the .img being corrupted in that process? Once copied to USB stick verify with sha1sum to match the checksum in release details, to verify the integrity of the copy.

All SHA1 hashes are ok.

I managed to get to the terminal and run the commands you provided using shredos-2021.08.2_21_x86-64_0.32.023.iso.

I tried lspci >>lspcitest.txt assuming I would find lspci.txt on the flash drive but it did not work.

@Binary-Bear Just to clarify something, the slow 620kb loading of shredos.img cannot be anything to do with the shredos kernel, i.e Linux as that's whats being loaded and hasn't started yet. Grub being responsible for loading shredos, decompressing and starting it.

Ah ok I understand. I am sorry it appears that I might have wasted your time.

I will keep experimenting and let you know what I find out. I think it might be best if I wait for the next version of shredos in case anything changes with grub in the meantime.

Thanks for your help and good luck with the new version of shredos which I hope will be in an iso format :)

PartialVolume commented 5 months ago

All SHA1 hashes are ok.

Good to know.

I tried lspci >>lspcitest.txt assuming I would find lspci.txt on the flash drive but it did not work.

No, that wouldn't have worked as the root folder that you created the file lspcitest.txt in, is ShredOS's RamDisk, this is a virtual disk that exists only in RAM and is not connected to the USB flash drive. To copy the file from the RAM disk to USB you have to mount a folder from one to the other as described here however there is no need for all that, a photo on your smart phone posted here will be fine. :-)

Ah ok I understand. I am sorry it appears that I might have wasted your time.

You've not wasted my time, I would far rather issues are reported than not and after all grub is still provided in the ShredOS .img file so could be an issue in the grub software provided by ShredOS. What confuses things is that Ventoy/Yumi take over that front end loading so it's harder to determine what code is actually running. A pure Vanilla ShredOS created with dd would be preferable, rather than copying onto Ventoy/Yumi as at least then it's only ShredOS EFI loader and Grub that's running and not Ventoy/Yumi code beforehand. That would help with at least diagnosing the boot issues.

Don't give up on it. I've fixed thousands of systems in my time and most are far more complicated than a PC and those can be complicated enough. Getting to the bottom of these problems means you have to be doggedly persistent. :-)

PartialVolume commented 5 months ago

An updated version of this table and graph can be found here #209

These are the boot times I'm logging for various systems, they vary by a significant amount, not surprisingly the 2022 Thunderbolt 4 USB-3 is the fastest at 46 MiB/s kernel load. The slowest so far is a 2008 USB-2 with a kernel load of just under 2 minutes at 2.3 MiB/s. More systems to be added in due course. The 620Kb/s you report are 3.7 times slower than the slowest system here, almost I would imagine USB 1.0 speeds.

Typical Boot times on various systems

ShredOS Version shredos-2023.08.2_25.0_x86-64_0.35_20231110.img (written to flash using dd) on a USB2 Flash drive. ShredOS kernel image is 218.1 MiB.

Screenshot_20240122_185328

Make Model Processor Graphics USB Controller Boot Speed secs (POST to Nwipe) cmdline mods legacy/UEFI
Sony (circa 2006) VAIO (laptop) Intel Core2Duo P8400@2.26GHz AMD RV620/M82 (Mobility Radeon HD3450/3470 82801I (ICH9 Family) UHCI & EHCI USB2 92 secs None legacy
Novatech (circa 2022) NP5x_NP6x_NP7xPNP (laptop) Intel i7-12700H 12th Gen Nvidia GA106M GeForce RTX3060/Mobile/Max-Q Intel Alder Lake-P Thunderbolt 4 (USB2 49 secs (~15 MiB/s)) (USB3 36 seconds (~45 MiB/s)) none UEFI
Novatech circa 2008) M761SU (laptop) Intel Core2Duo T5750@2.0Ghz Nvidia G98M GeForce 9300M GS SiS 1.1/2.0 1 min 58 secs (~2.3MiB/s) none legacy
Packard Bell (circa 2013) EasyNote TE69KB (laptop) AMD Quadcore A45000 AMD Radeon HD8330 AMD FCH OHCI & EHCI (USB2 54 secs (~10 MiB/s)) none legacy
PartialVolume commented 5 months ago

Just for reference your GIGABYTE M720-US3 motherboard was released circa early 2009, which makes the design about 15 years old. The USB specification is reported as

12 USB 2.0/1.1 ports (8 on the back panel, 4 via the USB brackets connected to the internal USB headers)

The fact it quotes USB 2.0/1.1 and is the same vintage as the Novatech M761SU (2008) may mean it also has SiS USB controllers, which is the worst performing in the table above. The maximum speed of USB1.1 is sub 1.5 MiB/s so your speeds are falling within that range. The motherboard should be capable of USB2.0 but it's like it's operating at USB1.1.

Binary-Bear commented 5 months ago

Please don't waste any more of your time on this if I am the only one still having trouble. I am going to keep tinkering and try to find out what is happening.

I am having the same issue on two separate computers with two separate flash drives so it must be something I am doing.

If you do have time could you try booting shredos-2023.08.2_25.0_x86-64_0.35.img using Rufus and also again separately with Ventoy/Yumi?

I understand what you say about my old computer but the same computer and flash drives load gparted.iso at over 18MB/s.

Hopefully I will be able to try shredos on a laptop tomorrow so then I will have tried on three separate systems.

Thank you.

PartialVolume commented 5 months ago

If you do have time could you try booting shredos-2023.08.2_25.0_x86-64_0.35.img using Rufus and also again separately with Ventoy/Yumi?

This is Ventoy/Yumi, the linux version which basically created the empty USB flash drive but loaded with the Ventoy EFI code (which you don't see, it just looks like an empty formatted USB stick, then dragged the ShredOS .img into the root folder of the Ventoy/yumi USB stick. Here's the result.

https://github.com/PartialVolume/shredos.x86_64/assets/22084881/457f0866-28f7-4002-bdd7-2d745614f1a3 .

Binary-Bear commented 5 months ago

Thank you for testing that.

I notice you used the Linux version of Ventoy/Yumi which may be important.

I use the windows version on a Win7 OS. (Yes I know I have old computers and an old OS LOL)

I just cannot think what is causing the problems I am having. All of a sudden I am having difficulties with hardware and software that I have booted for years.

Your time is better spent upgrading shredos than helping me, I am starting to feel guilty about the time you are spending on this.

Binary-Bear commented 5 months ago

I think I should mark this as closed because a new and more relevant thread has been started by someone else here

Thank you PartialVolume for your help.