search

balena-io / etcher

Flash OS images to SD cards & USB drives, safely and easily.
https://etcher.io/
Apache License 2.0
29.65k stars 2.1k forks source link

Etcher is corrupting SD and USB drives. #1426

Closed ghost closed 7 years ago

ghost commented 7 years ago

Etcher has corrupted two brand new SD cards and one USB drive. I formatted a new 64 gig SD drive to fat 32 using gparted then flashed Raspbian to the drive. I decided to use Raspbian Lite and wrote it without a reformat. The flash failed and now the drive cannot be modified, though the files can still be read. For giggles, I tried a second drive that was a new 8 gig drive. Again I formatted, flashed, it worked, reformatted, flashed again, it worked, did not format, flashed and it's corrupted like the first. I decided to try a cheap Lexar USB drive and the exact same thing happened if I try to flash twice without reformatting first.

I've tried the Linux and Windows fix suggested in the docs and it doesn't work. I tried flashing a different image, it doesn't work. I tried formatting on another PC and in windows, it doesn't work. None of my drives were bad before this, two were fresh out of the box, and one was a 5 month old USB drive. For me the issue happens like clockwork with SanDisk and Lexmark drives. I would love to provide logs, but after being disgusted with myself for choosing this program over Imagewriter that I shutdown and about threw my laptop into the road.

kureci commented 6 years ago

@jviotti tried unmounting first, erasing using diskutil eraseDisk. I've also tried using a USB adapter instead of a SD Card adapter and even though dd if=/dev/zero of=/dev/diskN zero-ed out the whole SD card by writing 30GB worth of zeros in there, I open the SD card and the previous files are still there! Nothing has changed. Next step is to order an SD card and hammer these to dust!

I just remembered, I've used these SD cards on Asus tinkerboard, I wonder if it's that that's corrupted the SD cards! Will look there too

lurch commented 6 years ago

even though dd if=/dev/zero of=/dev/diskN zero-ed out the whole SD card by writing 30GB worth of zeros in there, I open the SD card and the previous files are still there!

That sounds very very strange. Is it possible the SD cards themselves are faulty? Maybe it's worth buying some SD cards from a different manufacturer, to rule out the problem being with Etcher (or your computer) rather than the SD cards themselves?

ShiftLeftLogical commented 6 years ago

Ok. I have just caused two different cards to go from fresh out of the package working to seeing on linux/windows 'no medium' / 'no media' for both mount / disk management tools, respectively. This is Etcher on windows.

I figured the first card was just bad so I moved to a different card / manufacturer, same results. Flashing appeared to succeed and so did verification of data but then the card reader / usb media drive is no longer recognized as even having media installed.

Is there any logging mechanism I can submit to help understand what happened and why I cannot see these media devices?

Note that I have used Etcher just fine with this personal card reader/writer on my personal windows machine w/o a single issue, this is not my windows machine so I am not sure if there might be configuration details of the OS which are impacting this (patches/reg/etc.) Are there any registry requirements which are known problems? Are there any BitLocker setting which are known problems?

Scripting-the-Void commented 5 years ago

Just to add in my own recent story to this thread: I've also run into issues recently using Etcher for loading Proxmox onto a USB. It corrupted my drive and Windows formatting tools weren't able to do anything with it after a flash had failed. Since Windows wouldn't even poke at it from a distance, I opened Rufus and tried running a flash with some other ISO file I had laying around to see if Rufus could work its magic. It worked and the drive is now usable again. Looking forward to using Etcher because Rufus won't flash the ISO file I want.

99sono commented 5 years ago

Following the tutorial:

https://tutorials.ubuntu.com/tutorial/tutorial-create-a-usb-stick-on-macos?_ga=2.110392581.1832303617.1543043795-1884538113.1543043795#2

It looks like i just managed to get a Fried up USB 8 GB pen.

The make disk utility to rescuse the device seems not to work at all.

I would recommend everyone to not use this tool. One thing is the flashing process to fail and need to be repeated. Another one is for a storage device to become irrecoverable.

NOTE: The good thing is that it was possible to recover the drive. I needed to login to a linux terminal.

Then:

  1. lsblk to figure out the device number of the pen

  2. Then sudo parted /dev/sdb mklabel msdos

this created a new master boot record/partition table on the device.

  1. I then used linux fdisk to create a new FAT partition. So the pen is servicable again.

I am re-running the process of flashing the pen, but I am expecting the partition table to be ruined again at the end of the process. The process had problems the second time as well. My pen had two paritions in it, the first one of the size of the ISO with uubuntu the second one tiny. Ultimately, I just deleted the second partition via linux fdisk. I then made sure I made the first partition bootable. Mac OS does not reconize the pen, but in linux I can mount mount /dev/sdb just fine. So I am now running the istallation from the PEN, using a bootable pen that MAC cannot recognize or mount. In the end of the process I will use a mkfs -t vfat on /dev/sdb1 parition to make the parition readable again for mac. Actually, I will recreate the partition first because I do not need 1.9 GB partition i want an 8 GB parition.

We will see.

amquei commented 5 years ago

I think thal I solved the problem using GPARTED and creating a new partition table (msdos) on the my pendrive several times and then I formatted the pendrive using the software called DISKS - gnome-disk-utility 3.18.3.1 on Ubuntu.

lurch commented 5 years ago

Interesting to see that authors of other disk-writing software also get just as many unsubstantiated complaints about the software "breaking" their drives https://github.com/pbatard/rufus/issues/313 :roll_eyes:

kureci commented 5 years ago

@lurch interesting choice of words there.. "unsubstantiated" complaints? I bet you think complainers here are conspiring to complain about all such software. Dude, if you're a contributor and not willing to see what could be causing this, at least don't just talk nonsense. What "substantial" evidence do you need to believe that this software has corrupted so many sd cards? Send me your address and I'll post my two 32gb U-class high quality microSD cards that have been corrupted by Etcher. If you manage to fix them then they're yours to keep.

thundron commented 5 years ago

@kureci What do you mean by "corrupted"? What's the issue you're having with them? Do you have any logs? Did you try solutions provided for different OSes here as well as reading through the whole document to verify you're not in one of those cases?

I hope you can now easily see why @lurch referred to these as "unsubstantiated complaints" and I'd advise you to read this issue as well (the last 3 comments in particular) that might enlighten many aspects of users that got corrupted sd cards "by Etcher".

kureci commented 5 years ago

@thundron if you go back in comments you can see that I've tried a few things, tried to put relevant bits of information regarding what the output is.. etc.

I won't be using Etcher again, and not because it has issues (even if minor), but because its developers seem to be denouncing any kind of claim that Etcher is causing SD card corruption. Everywhere I read comments from its promoters, the first thing they do is deny Etcher having anything to do with it, rather than listen and perhaps think of any possibility. Because clearly Etcher isn't just a UI on top of simple commands like dd on Unix, because I have used the dd command on the same machine numerous times in the past, with no significant issues.

Almost everyone who replies to these complains says "Oh Etcher is being used by thousands of people, if a few people are having issues with it then it's not Etcher" - seriously? OK, you can say it works 99% of the times, but at least be accepting enough to just consider the 1%

Anyway I'm not expecting to really have any useful output from this issue or the conversation. If I do get any spare time I suppose I could check the source and see if I can spot anything, otherwise I'll just not touch it again.

kureci commented 5 years ago

Oh and, I've read a few comments from promoters/developers claiming that software cannot possibly "break" hardware - they probably think of the word break as some sort of smashing the cards into pieces or something.. goes to show how open-minded they are to accepting that software could be "damaging" the hardware not in a physical way, but in a way that makes other software impossible to write to it. Hardware isn't damaged as per se, but these SD cards aren't to be opened up and their circuits manipulated by hand to check that all parts are in place and that Etcher hasn't moved any parts!

thundron commented 5 years ago

It's also not too hard to understand that byte-to-byte flashing, which relies on the OS, cannot in any way "break", "damage", "corrupt" or do any harm in general to the flashed target, unless:

Which isn't even that rare, if you read the last 3 comments of the issue I linked just to give you an example:

For example, a bunch of the negative reviews from around 2014 for this 8GB Kingston MicroSD card received the following response:

[...] We apologize for the issue experienced with the microSDHC cards. We stand by all our products and can replace those cards under their lifetime warranty. If they were used in Android 4.0 or later devices, the issue may be related to a problem that was discovered last year that caused the cards to become corrupt and then undetectable in any device. This issue was resolved August 2013 but it is possible you may have received cards that were manufacturer prior to that time. [...]

And all of the reasoning above is exactly why, at some point, we're almost forced to just say "Etcher cannot corrupt sdcards": because the external factors are so many and because flashing byte-to-byte does absolutely no harm and we cannot fix external issues. If the user doesn't have a clue about them, we can only suggest to look into this or that (and as you see in this issue for example, we always do), but other than that, it's phisically out of our range.

I saw you were looking into your Asus Tinkerboard but didn't mention anything else, so I hope you'll get back to it and tell us if you had any luck with it or even recovering the sdcard with another OS (I'd usually suggest Linux, but again there are other ways to do it which you can find in the User documentation)

ShiftLeftLogical commented 5 years ago

I keep getting comments in my inbox and largely ignored them regarding this issue as I stopped using this tool as all I can say for certain:

  1. I have never had an issue with Win32DiskImager which I believe was the recommended way to flash from windows for raspbian, in times past.
  2. Etcher has worked fine for a 4 of 6 cards.
  3. Etcher failed two cards in a row (5/6 and 6/6) using the same card reader, different host machine (USB HW), and the cards were completely different brands. The odds are small but I the more probabilistic answer is that I got two bad SD cards, if I had more to waste I would have formatted via dd or Win32DiskImager, verified, then reformatted with Etcher. Because I do not want to risk a third card I am not going to sling mud in one direction but what I do wish is that there was some logging past what I could find in the windows logs that could help in diagnosing the issue, I don't use windows often so maybe I missed where the logs pertaining to disk imaging is located but from what I found, the OS logs contained minimal information at best. I understand the frustration on both sides but as a developer for both private and open source software I guess I would provide diagnostics, even if just gathering existing logs such that I could better understand the true occurrence of the problem (opt. in reporting) and compare that to the statistical likelihood of a bad SD card per vendor, some card writers tending to have HW/driver issues, etc. ... it would quash any doubts I had about my software and at least allow for me to point to the gathered statistics as reasonable evidence that it truly is just the odds of getting a bad card or problematic writers.

I guess why I care is because I want to side with the contributors but having this happen to me and the odds being so small that it was two new cards in a row along with the fact that the tool is promoted over the older methods right on the raspbian website the general response seems dismissive, maybe understandably so but can a dev at least answer this:

  1. Is there any more diagnostic information that could be coalesced?
  2. Has an opt in feature to check the relevant HW and OS version for reporting purposes been proposed or implemented and I just did not read closely enough? Would this be considered?

Edit: I did not consider the case of an extension cable and USB 3.0, I am not certain I used 3.0 in the case that Etcher did work but I am certain I used 3.0 ports in the case where it did not. This says nothing definitive but should be noted.

lurch commented 5 years ago

Is there any more diagnostic information that could be coalesced?

Yes, see https://raw.githubusercontent.com/balena-io/etcher/master/.github/ISSUE_TEMPLATE.md for how to access Etcher's logs.

some card writers vid:pid tending to have HW/driver issues

With the cards that have hacked firmware to report fake capacity, I guess they'd have no problem reporting a fake vid:pid too?

EDIT: Could you clarify what you mean by "Etcher failed two cards" ?

techsolveprac commented 5 years ago

Your flash drive can be recovered, read here --> https://www.techsolveprac.com/repair-flash-drive/

kipoph commented 5 years ago

I had this problem with a very stubborn corrupted USB flash drive I tried every solution in windows and nothing worked (literally spent a few hours trying online solutions). Finally found a fix! Put it in a Linux virtual machine reformated and stuck some stuff on it and even windows can read it now!

techsolveprac commented 5 years ago

@kipoph Check this: - > https://github.com/balena-io/etcher/issues/1426#issuecomment-468651933

hjaspe commented 5 years ago

This recipe save my pendriver: 1.- Use Linux, fdisk /dev/xxx and drop all partitions 2.- Execute dd if=/dev/zero of=/dev/XXX count=1 bs=4096, where XXX is your especific device 3.- Format with Windows or Linux

stephenpfaff commented 4 years ago

Ran into this issue with using etcher to burn a linux image to SD card on OSX using Etcher. The issue wasn't that the SD card became corrupt rather OSX claimed it was corrupt because it knows the physical disk size exceeds the visible total combined partitions. Opening terminal and running diskutil list will output the SD card partition map where you will see the missing disk space under a linux partition type which does not appear visible to OSX or Windows disk utilities.

To fix the issue you want to clear off that linux partition which can also be done within OSX terminal using diskutil...

gabrielsaviank commented 4 years ago

I created a bootable Ubuntu USB on Etcher and corrupted my USB stick, I can't format it, I tried on my mac and on linux and couldn't format it. The error I get on mac is " The operation couldn't be completed. Permission denied"

thundron commented 4 years ago

@gabrielsaviank have you tried re-running the tools you used with sudo? That's more likely the reason you saw the message

mikemonagle commented 3 years ago

After successfully flashing a micro SD card with Etcher, my entire card reader is no longer recognized by the OS (Windows 7). How is this even possible? This is a card reader/write that's part of the computer. Used to show up as four empty drives when there were no cards in it. Now it doesn't show up at all.

kureci commented 3 years ago

Ran into this issue with using etcher to burn a linux image to SD card on OSX using Etcher. The issue wasn't that the SD card became corrupt rather OSX claimed it was corrupt because it knows the physical disk size exceeds the visible total combined partitions. Opening terminal and running diskutil list will output the SD card partition map where you will see the missing disk space under a linux partition type which does not appear visible to OSX or Windows disk utilities.

To fix the issue you want to clear off that linux partition which can also be done within OSX terminal using diskutil...

  • diskutil list (note/copy the linux partition name on your drive)
  • sudo diskutil eraseVolume free none /dev/name (replace name with one copied on prior step i.e. /dev/disk3s1)
  • repeat step 2 for any other partitions
  • ????
  • Full SD disk should be recovered and available for use now.

Can you clarify what "???" is in step 4 please?

ovictorjo commented 3 years ago

Etcher is corrupting SD and USB drives. Why?

kureci commented 3 years ago

Etcher is corrupting SD and USB drives. Why?

Contributors will tell you it's all your fault, Etcher has nothing to do with it. BS! I havent used Etcher since my last corruption. Just use your terminal commands and you should be good. I'm surprised Etcher is even suggested as a tool by the Linux websites.

mikemonagle commented 3 years ago

Etcher is corrupting SD and USB drives. Why?

Contributors will tell you it's all your fault, Etcher has nothing to do with it. BS! I havent used Etcher since my last corruption. Just use your terminal commands and you should be good. I'm surprised Etcher is even suggested as a tool by the Linux websites.

In my case it was recommended by this book, "Make: Jumpstarting the Raspberry Pi Zero W". My first and last time using Etcher.

gabrielsaviank commented 3 years ago

To fix mine I did the following: I tried to format on Linux and Mac using loads of ways and none worked. Then I tried to format on windows (I can't remember the format that I used) and later I formatted on my mac again using exFAT (if I remember correctly) and worked like a charm. Plus: I formatted using the tools, both on mac and windows. Not the definitive guide, but could work for people who are having issues on UNIX systems.

k2xl commented 3 years ago

So I believe this may of just happened to me too.

Tried to flash Hassio using Balena Etcher... And at the end it said checksums didn't match. Then I noticed I couldn't get any file changes to the sdcard to stay permanent. Went and bought a new sd card and the exact same thing happened. Seems like it gets into write protected mode somehow

kureci commented 3 years ago

So I believe this may of just happened to me too.

Tried to flash Hassio using Balena Etcher... And at the end it said checksums didn't match. Then I noticed I couldn't get any file changes to the sdcard to stay permanent. Went and bought a new sd card and the exact same thing happened. Seems like it gets into write protected mode somehow

@k2xl no, it had nothing to do with Etcher. Maybe the temperature at your location, or your cat meowing at the wrong moment, whatever it may have been, it was NOT Etcher.. the contributors will tell you.

Sorry mate, your SD cards are most likely gone forever.