How to restore dead SD card used to boot a HDD?

procount commented 6 years ago

Pro count uh I need help. So basically last night I shutdown my raspberry pi 3B+, turned it on this morning and nothing happens. I plugged the SD card into a reader on my Mac and the card could not be initialised. I am thinking the card is dead. Basically I had the boot on the SD card, and installed the OS on a HDD. The HDD still works. How can I get the Raspberry pi working again?

Thanks DarkPlatinim

RaspberryTech01 commented 6 years ago

No error on raspbian 1 mouse and keyboard working fine on it.

RaspberryTech01 commented 6 years ago

Retropie also recieve ing kernel module error, keyboard not working.

procount commented 6 years ago

There's no setup script for the projectspaces, cos they have no OS in them. I wonder, did we get Raspbian2 and gentoo partitions the wrong way round? Maybe worth checking them again. How long ago did you install these OSes to your HDD? Are they still the same version as the versions you installed the boot partitions from?

RaspberryTech01 commented 6 years ago

TBH I don't really need Gentoo or LibreELEC as I don't have any data in them, the only ones I really need are the raspbians and retropie. My retropie was installed previously from an ad card using your dual boot tutorial.

RaspberryTech01 commented 6 years ago

Should I try fixing the retro and raspbian using filesystem check?

RaspberryTech01 commented 6 years ago

And btw I am not sure about their version.

RaspberryTech01 commented 6 years ago

RetroPie is probably not the latest version. Same as raspbian 2

procount commented 6 years ago

Do you still have the original SD card that you installed Retropie from? If so, you could copy that Boot partition over the retropie boot partition you just installed to your new SD card. You perhaps need to do the same thing for Raspbian2.

RaspberryTech01 commented 6 years ago

Raspbian 2 was installed via PINN sometime ago. the dead SD card probably messed that up. The only problem with retropie is that I may have updated once since using PINN, will it still work?

RaspberryTech01 commented 6 years ago

Let me check if I have the original SD for retropie.

RaspberryTech01 commented 6 years ago

Il call it a day and contact you tommorow.

procount commented 6 years ago

Well, you have the general principle now of how to fix it, and the good news is you got your Raspbian1 working. Your issue now is just finding the correct version of boot partitions to match the original root partitions.

If all else fails, you can always copy your datafiles off your HDD now and then reinstall the latest versions of the OSes and copy the data files back again.

RaspberryTech01 commented 6 years ago

I do not have the SD card, I have probably wiped it and installed another OS for another Raspberry Pi.

procount commented 6 years ago

Oh dear. Never mind. You can check the version of the kernel used in your Raspbian OSes (and probably others too) by looking on the root partition at /lib/modules and seeing what folders it has. Something like 4.14.71. check Retropie as well. The you can copy the matching kernel and boot image for it.

RaspberryTech01 commented 6 years ago

Raspbian 2: 4.14.70+ and 4.14.70-v7+, Retropie 4.14.30-v7+, 4.14.70+, 4.14.70-v7+

procount commented 6 years ago

The latest Raspbian in NOOBS is 4.14.71, so I guess you may have started from an earlier version (4.14.50) and upgraded? Likewise, the latest Retropie is 4.14.30, so the presence of 4.14.70 suggests you have upgraded that as well.

The main idea is to restore the boot partitions on your SD card to what existed on your SD card when it went bad. What we did up to now was to restore the boot partitions with the very latest version, which I assumed you had, but it seems to have caused some issues. So here are some proposals for you to get back to your original working OSes on your HDD.

Option1

Remove your HDD, and restore your installed_os.json file to what it was with your new installations, so they run off your SD card. (Keep the existing installed_os.json because you'll need it again later).
Replace the OSes that didn't work with an older version. You might have to construct your own os_list_v3.json file and put it on the PINN partition (/dev/mmcblk0p1) to reference such an old version. You can find the old Raspbian OSes at http://downloads.raspberrypi.org/raspbian/archive/.... Also you should disable all other download sources so you only get the specific versions you want, and they are not overridden by later versions.
(You can use uname -a to see what version your kernel is. I think it would get this from the kernel itself rather than any info on the rootfs, but you'd need to check)
Separately boot into the OSes that didn't work and upgrade them to the same version as is on your HDD. You may have to use rpi-update and pick a github revision to exactly match the kernel version on your HDD. Just doing sudo dist-upgrade may bring you to a version that is too new.
Finally go back to your modified installed_os.json file to see if the upgraded boot partitions now match the rootfs kernel version and work correctly.

Option 2

For Retropie, I can't imagine you have done much customisation, apart from adding some ROMS etc. So if the above doesn't work, or takes too long to do, you might be better to cut your losses by installing a new version of Retropie (maybe into one of your free ProjectSpaces) and just copying your ROMS and any other data files across. Perhaps you can do the same with Raspbian

RaspberryTech01 commented 6 years ago

And how do i get my original installed json file back? It was replace when i did the cp command earlier.

RaspberryTech01 commented 6 years ago

I may just choose option 2, is it just basically installing the new os, then copy pasting the root filesystem?

procount commented 6 years ago

And how do i get my original installed json file back? It was replace when i did the cp command earlier.

You pasted a copy of it above ^ 😄

I may just choose option 2, is it just basically installing the new os, then copy pasting the root filesystem?

Yes, that's one way. I would take a copy of your existing rootfs first in case it goes wrong. Or install a complete new copy to another partition (one of your project spaces maybe), then copy the data files across. many other ways to do it too. But in all cases, try to keep a backup copy of everything as it could be a bit of trial and error to get ti right.

RaspberryTech01 commented 6 years ago

Ok I'll go with option 2. I'll report back later.

RaspberryTech01 commented 6 years ago

Ok so I installed retropie on a project space, logged in as root on raspbian 1, deleted all files in prjroot, then copied files from original retropie to prjroot. Boot up still has the same errors. What files am I supposed to transfer across? As i said i had transfered the whole FS, so i guess it would have the same errors. Which parts of the FS do i replace?

procount commented 6 years ago

No, that's just ended up where you were before (i.e. using the new BOOT files with the old ROOT files) The idea was to keep the contents of the new retropie boot AND root partitions and then copy the ROMS and data files only over from the old retropie rootfs. The ROMS should be easy to find/search for, but I've no idea about the data files that would have been created (e.g. that save game data), as I've never really used Retropie. Maybe ask on the Retropie forums if you can't work it out. You could mount both partitions and do a diff to see what files exist on your HDD that don't exist on the projroot partition and copy those over 🤷‍♂️ .

RaspberryTech01 commented 6 years ago

And do you know what directories I have to transfer on Raspbian?

I really appreciate all the help you have given me 🙂

procount commented 6 years ago

If you have installed programs, best to just reinstall them on the new version rather than trying to copy them, because the files get spread over lots of different folders. Most data files will be stored in /home/pi (don't forget hidden files aswell)

RaspberryTech01 commented 6 years ago

So for best results just transfer /home/Pi? What do you recommend?

procount commented 6 years ago

Open a terminal. Make sure the 2 partitions are mounted. Assuming your old HDD raspbian root is mounted at /media/pi/source and your new root partition is mounted at destination (change as appropriate) try:

cp -r /media/pi/source/home/pi/. /media/pi/destination/home/pi

(https://superuser.com/questions/61611/how-to-copy-with-cp-to-include-hidden-files-and-hidden-directories-and-their-con)

RaspberryTech01 commented 6 years ago

ATM I am transferring the retropie /home/Pi . I'll have a search to see if I can find where the save files are stored.

RaspberryTech01 commented 6 years ago

Ok, I've got retropie working. Now it's time for the last raspbian. I had to copy /home/Pi, and I had to copy /opt (game saves here)

BTW I've been having an issue that used to happen on the dead SD card, everytime I plug in a USB stick into my Pi, I cannot access it unless I am in root. I'm thinking it's something to do with user permissions. Do you know how to enable this? It's kinda a hassle to log in to root everytime just to read or write something off a USB stick.

procount commented 6 years ago

something like:

sudo chown pi:pi /media/pi/<USBSTICK>

But for a more permanent solution see man udev or google udev

RaspberryTech01 commented 6 years ago

Thanks, I'll look in to this later. If I was to choose option 1 for raspbian, is it basically finding the kernel and download boot.tar.xz? Then uncompressed it and transfer it to the boot partition?

procount commented 6 years ago

Basically, yes. But you need to re-run the partition_setup script on it before booting it in order to fix-up all the partition links to their new locations.

RaspberryTech01 commented 6 years ago

hmm... so i llooked at the link that you gave to the boot.tar.xz, how do i find the kernel version? I uncompressed one of the 2018 kernels but not sure what version it is.

RaspberryTech01 commented 6 years ago

something like:
sudo chown pi:pi /media/pi/<USBSTICK>
But for a more permanent solution see man udev or google udev

Found an easier solution: sudo chown pi:pi /media/pi/*

procount commented 6 years ago

They normally mention it in the release notes. Or decompress the rootfs and look in lib/modules as before.

procount commented 6 years ago

Another option..... Copy your Raspbian2 root partition to another empty projectspace (so preserving your original one). Copy the /lib/modules from Raspbian1 root to your newly copied raspbian2 root. Copy raspbian1 boot to raspbian2 boot partition. Adjust the partitions in installed_os.json to match. Rerun partition setup from pinn on raspbian2. See if it boots.

So instead if trying to match the boot partition to the root kernel, you are matching the root kernel modules to the boot partition.

But as it modifies your raspbian2 data, that is why I suggest doing it on a copy.

RaspberryTech01 commented 6 years ago

Thanks for your help! It will be quite sometime till I get time to finish off the raspbian 2 fix. I appreciate all the help you have given me. I'll tell you what I do in the end thanks!!

procount commented 6 years ago

You're welcome. Hope you get it all working. No reason why you shouldn't now you've learnt some more techniques.

RaspberryTech01 commented 6 years ago

Whoops... I got raspbian 2 working. I wanted to replace the spare raspbians that were used to recreate the SD card and in doing so i replaced raspbian 1 :(. luckily i had a spare version from a couple minths ago. I got that version working, update everything. But on my user, pi, my desktop prefrences don't work. whem i click ok, nothing updates. However, if i try with root user, it works fine; do you know how to fix this?

procount commented 6 years ago

Great! which route did you take in the end - finding the correct boot partition version, or updating the kernel and modules in root and boot?

Sorry, I have no idea about the desktop preferences.

RaspberryTech01 commented 6 years ago

I updated the modules in root, I took a backup, but I used the original root. Ill ask about desktop prefs in rpi forums. Is there a way i can rename each indivual raspbian so i know which is which? possibly changing the png image?

procount commented 6 years ago

I'm looking at implementing something similar in the near future, but more of an "alias" rather than a full rename, in order to keep the links with the original source and be able to identify when a new version is available. So if you want to remain compatible, I suggest to keep the same names for now. Maybe change the description field instead (in /settings/installed_os.json) to indicate your use of this "Raspbian". You can change (or add) an "icon" field to each OS to use a different one, or just replace the corresponding "raspbian.png" or "icon.png" that is in /settings/os/raspbian...

RaspberryTech01 commented 6 years ago

So i can add multiple pngs to the os/raspbian, then choose it in the installed_os.json, correct?

procount commented 6 years ago

Yes. It's a bit non-standard, but it should work.

RaspberryTech01 commented 6 years ago

Ok.... I'm having a problem. Raspbian 1 is in the process of being restored, after restoring raspbian 2, all is well except whenever I try to open chromium, nothing happens. I think something has broke somehow.

RaspberryTech01 commented 6 years ago

Raspbian 1 just finished restore.... Booted it up getting a fail, unable to set console text.

RaspberryTech01 commented 6 years ago

Looks like something went wrong with the copy... Going to try again.... HDD has written 90 gigs alerady today lel

RaspberryTech01 commented 6 years ago

Ok.... So reinstalled backup yet again, the issue is the same, I'm getting a underscore that's flashing on boot up after the boot process has finished. It's a totally black screen with a underscore top right corner.

RaspberryTech01 commented 6 years ago

I'm honestly going mad here..... I had another raspbian downloaded to a project space, lets call this raspbian 3. So raspbian 3, I removed /home, replaced it with backup of /home, then I log into Pi, then guess what happens.... Desktop preferences don't work... WHY??? Works completely with the USB backup...

procount commented 6 years ago

That's a shame, after you originally got it working. You might be better just to cut your losses, salvage any data you have and just start again with a new version, install any apps again then copy the datafiles back.

procount commented 6 years ago

Do the desktop prefs work ok before you replace /home? Maybe the desktop prefs are stored in a hidden file/folder in /home/pi ?

procount / pinn

How to restore dead SD card used to boot a HDD? #195

Option1

Option 2