Working with UUIDs instead

[Zaephor]

Not necessarily a bug, I guess this is more of a feature request. Working with Ubuntu 12.10 on a system that I can't actually use the hard drives for. I came to the conclusion of loading the OS and /home to two different thumbdrives.

The process I have done: Installed Ubuntu 12.10 to USB[0](Grub here too) Grabbed the 12.10 script. Followed its directions and connected a second USB drive to the system for use as home. So /home is on USB[1]

My concern: The system could give different labels to the thumbdrives randomly on boot. Like if a hard drive is added or removed, or other thumbdrives. So sdb and sdj could suddenly become sdc and sdd. That would break the fstab wouldn't it?

My solution and it's issue: Assign /home by UUID instead of the drive labels. So I reformatted the partition so it's UUID would end in a number to bypass that partition checking portion of the code. Then to notice the UUID changes again because the script reformats the /home partition. So the script is currently going thinking $DEVICE is one thing, when it just finished changing it to another. Using the UUIDs should get around the labelling issue, but I'm not sure how much work I've got trying to fix the UUID everywhere...

So I'm working on a quick fix around the mkfs.ext4 chunk that should find the /dev/sd_# using the UUID, format the partition, then check what the new UUID is and pass that as $DEVICE the rest of the way. I think that should at least let me get running.

Otherwise thanks so much for your work!

[tal-zvon]

mkfs.ext4 only formats the device. Since the script will only need to identify the device to format it once (and not between reboots), this part should be fine. You'll basically identify USB[1] by its /dev/ device name while the script is running, let it format, and that's it. It won't be any less formatted if it's device name changes after a reboot. Grub also uses UUIDs in my script to find the root partition, so that should not be a problem either. The only problem I can find would be the CopyHome()'s:

    #Modify RAM session's fstab to mount /home partition
    sudo bash -c 'echo "'$DEVICE'   /home   ext4    auto    0 0" >> '$DEST'/etc/fstab'

    #Modify regular system to mount /home partition as well
    sudo bash -c 'echo "'$DEVICE'   /home   ext4    auto    0 0" >> /etc/fstab'

lines, which would cause /home to be identified by the device name instead of the UUID each time the OS boots. To change this, you need to figure out the UUID of /home, and write that to /etc/fstab instead of the device name.

I quickly made the change, and pushed it to a new branch on github. You can get it by running:

git clone -b identify_partitions_by_UUID git://github.com/terminator14/RAM_booster.git

I'm pretty sure this should not effect any other part of the code, while making it work the way you want. I have not had a chance to actually test it though by setting up a VM of Ubuntu 12.10 and running the script. If you have the opportunity, give it a try and let me know if it works. I'll give it a try as soon as I can as well.

[Zaephor]

Wow! I did not expect such a quick response lol I had only briefly skimmed the code before starting to use it and saw that. I actually ended up making the same change but did a far less graceful HOME_UUID=$(sudo blkid -o value -s UUID $DEVICE) line. So far seems to work, I need to make a tweak on my /home thumbdrive, it used to be a Win7 installer thumbdrive so I think it still has some bootloader code in it somewhere. My first reboot stalled out because it tried to start from the wrong thumbdrive. Otherwise everything seems to be working as I'd hoped though.

[tal-zvon]

You can use:

$ sudo dd if=/dev/sda bs=512 count=1 2>/dev/null | hexdump -Cv

to see your thumbdrive's MBR. If the Win7 bootloader is there, you should see it. You can find more details on the above dd command here: http://blog.philippklaus.de/2010/02/have-a-look-at-your-mbr-using-dd-and-hexdump/

And the following will get rid of your Win7 bootloader:

$ sudo dd if=/dev/zero of=/dev/sda bs=446 count=1

See here for details: http://www.axllent.org/docs/data-storage/erase-your-mbr/

NOTE: Make sure to change /dev/sda to your USB drive (note /dev/sda, NOT /dev/sda1 or /dev/sda2 - make sure you don't give it a partition but a device), and make sure to use bs=446 and NOT bs=512 in your case, or you will destroy the partition table.

Hope that helps.

[tal-zvon]

The code appears to work, improves the script, and has no downside as far as I can tell. I think I'm going to merge it into master.

[Zaephor]

Hey thanks for the extra info, I had it bookmarked somewhere on a different computer but had a midterm today so never got around to being able to play with things like that. And I do agree, haven't seen any disadvantages to this method either.

Only additional thoughts I have(since I'm tossing in the UUID and thumbdrive monkey wrenches), is would it be a good idea to clone grub from the USB(0) to USB(1)? Incase the system stalls on a USB device despite not having a bootloader? That's probably a bit of a chunk of extra code to determine if the storage mediums are thumbdrives though huh?

Also would an additional script like rupdate/rchroot be beneficial that could upgrade/migrate/backup the /home partition be a good idea? Or even support for /home to be hosted on an NFS share(Which I'm horribly thinking about)?

Sorry for tossing in other thoughts/feature ideas, when I find something cool like this I always run through a billion other ideas on how I could use it. Like I just started thinking of jumping with my root thumbdrive from system to system in my network with my /home partition stored on my NFS server. I know there are disadvantages to that layout but could be a strong option?

[tal-zvon]

Cloning grub from USB(0) to USB(1) might take some code, since I doubt it would be as easy as one dd line to copy the first 446 bytes from one drive to the other.

What would even be the advantage of it? "Incase the system stalls on a USB device despite not having a bootloader"? How would that even happen? Why would the system stall?

upgrade/migrate/backup the /home partition?

Upgrade: How do you upgrade a /home partition?

Migration:

2 steps:

1) cp -ax /home/* /mnt/home_dest/ 2) modify one line in /etc/fstab

Backup:

1 step:

1) tar -cvpzf backup.tar.gz --exclude=/backup.tar.gz --one-file-system /

These are such short commands that they do not warrant their own script, which would literally be harder to make and more complicated to explain to users how to use than the cp and tar commands they likely already know.

The NFS thing sounds interesting. I don't see a big use for it, but if that's what you wanna do, it shouldn't be too complicated. Let me know if you need advice on how to get the script working with a /home over NFS.

[Zaephor]

lol Yep I just keep running my mind, sorry about that. I have a history working in oem rackmount server equipment, you'd be amazed the number of times and motherboards I would find that would have heart attacks if you left a USB storage device in it when you walked away and it reboots. Alot like the old days where if you left a non-bootable floppy disk in the drive, the good old "Non-system disk or disk error, please remove or replace" and it would be stuck at that BIOS screen till you removed it. We also used to have a specific client that would call in on a regular basis because he'd pull all his drives out of his software RAID(physically) and forget which one was plugged in to which port. Which would end up screwing up the BIOS boot sequences since you could only choose from the first 4 of the 24 drives to be in the default boot list. So before sending him anything we'd run a script that would run the grub install command down every connected /dev/sd_, and keep a /boot partition SW RAID1'd across all 24 drives.

But yea, good point on cp/tar, I just have a bad habit of compensating for user error(70% of any code I write is normally data checking to make sure it's safe to do something regardless of what the user wants.)

The NFS idea was more just a quick thought, I don't know if I'll find the time to migrate to or test that until summer.

[tal-zvon]

While the grub solution you guys came up with may have been the best, and possibly only solution to your problem, it is difficult to call it anything other than a hack, and a dirty one at that. While they are sometimes unavoidable, putting one in a program that addresses a problem that most users will never face seems like a bad course of action. Imagine if most of your admins at the company that dealt with that server were not aware of this hack, and were spending days trying to figure out which was the correct drive to boot from, or why they were all working when logic says they shouldn't be. This could have caused hours, or even days worth of downtime while they were trying to figure it out.

Also, the idea behind the RAM_booster project came from the same place lots of my script ideas come from: a head full of crazy, mostly useless ideas of a sleep-deprived mind at 2 am that thought "Hey - what if I wrote a script that could do this? Wouldn't that be awesome?" without asking "Why?". It's given me some very interesting scripts.

[Zaephor]

Oh I agree, it was the last step we resorted to in resolving the situation. Initially we tried to work with the motherboard manufacturer to get the BIOS upgraded to support the increased number of devices. Contacted the HBA company asking if there were any way to flag a unique serial number to always be the first bootable. Tried to get the customer to newer hardware(darn proprietary PCI cards).

I didn't like the solution all that much more either, there was another alternative suggested of using a small USB thumbdrive on a USB header in the system with the /boot partition on it and the bootloader, but it was dismissed thinking it would be more likely someone would look at the bios and wonder where the usb drive was... and assumed the thought process would go further downhill.

We printed out and included instructions on how to perform the solution, how to repair things in the event of a drive replacement, and so on. Laminated, folded and also taped the instructions on the inside the cover of the system. We kept a copy of it with their order in our systems with flags all over it basically saying "not simple". Internally there was no way for our people to miss it. The company specialized in "custom solutions" so we were also the last resort for the client in finding a system that worked with their proprietary stuff.