andlabs / reallymine

WD MyBook encrypted hard drive decryption (still WIP).
https://github.com/andlabs/reallymine/issues/38
GNU General Public License v3.0
216 stars 48 forks source link

[BUG] Error reading sectors in DecrpytNextSector(): read /dev/sdb: input/output error #2

Open koalaeagle opened 8 years ago

koalaeagle commented 8 years ago

I am receiving this error at 3050 MB / 1430799 MB every time I run the tool. Any idea what could be causing this? The device is 1.5TB and JMicron if that is relevant! Thanks.

MrDecay commented 8 years ago

Sounds like bad sectors on the source drive On Jun 16, 2016 12:43 PM, "binxycraft" notifications@github.com wrote:

I am receiving this error at 3050 MB / 1430799 MB every time I run the tool. Any idea what could be causing this? The device is 1.5TB and JMicron if that is relevant! Thanks.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/andlabs/reallymine/issues/2, or mute the thread https://github.com/notifications/unsubscribe/AQE6xao4eZwhobb1BtkHmp-WIlNLSyppks5qMYtQgaJpZM4I3ofb .

andlabs commented 8 years ago

Yes; see if you can dd or ddrescue the drive to see if the sectors in question are bad first. If not, it could also be your USB bridge failing?

athomic1 commented 8 years ago

Which USB bridge would that be? I would assume the original one had already failed, if binxy is trying to use this tool for recovery. That's why I was using it.

Does the tool currently have any way to flag and/or skip over bad sectors? Assuming this is the only problem area, or at least the drive isn't rife with bad sectors, they should still be able to recover most of their data.

andlabs commented 8 years ago

The one you're using to connect the drive to your computer right now.

No, this program doesn't do that yet. That would basically be reimplementing GNU ddrescue. I wonder if there's a way we can combine the two...

MrDecay commented 8 years ago

Well lets not wander all this tool is for is the wdc decryption not for dealing with bad sectors . I think if any recovery implementing should be at the next level maybe a way of decryption on the fly but as it is now it works for what it is intended

But now in hind sight. Maybe a 600ms time out might be beneficial and skip sector

On Jun 20, 2016 10:55 AM, "athomic1" notifications@github.com wrote:

Which USB bridge would that be? I would assume the original one had already failed, if binxy is trying to use this tool for recovery. That's why I was using it.

Does the tool currently have any way to flag and/or skip over bad sectors? Assuming this is the only problem area, or at least the drive isn't rife with bad sectors, they should still be able to recover most of their data.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub, or mute the thread.

MrDecay commented 8 years ago

Or even calling it for that sector On Jun 20, 2016 11:35 AM, "Pietro Gagliardi" notifications@github.com wrote:

The one you're using to connect the drive to your computer right now.

No, this program doesn't do that yet. That would basically be reimplementing GNU ddrescue. I wonder if there's a way we can combined the two...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/andlabs/reallymine/issues/2#issuecomment-227196423, or mute the thread https://github.com/notifications/unsubscribe/AQE6xVrZCXQaTN8KdJZ_58uA2U-5eM2Iks5qNsFKgaJpZM4I3ofb .

koalaeagle commented 8 years ago

It is definitely bad sectors and not the new USB bridge (I tried with 2 different ones that are both very new and working well with other drives). This drive has traveled with me across the pacific ocean twice and has been stored in a few different locations due to permanent relocation's back and forth. I have been running ddrescue since it was suggested 4 days ago. I believe it is finally almost done with its process. It is on the "Splitting failed blocks" stage if that means anything. I am far from an expert at any of this but I used it to create a .img file. Will I then be able to use reallymine and point to the image file directly? Here are some of the stats if they are somehow helpful: rescued: 1500GB errsize: 782 kB errors: 183 ipos/opos: 3188 MB

MrDecay commented 8 years ago

How big was the total drive size? If you have recovered the magic sector it should decrypt the image to another Image. Then either mount or run recover software on the decrypted image On Jun 20, 2016 1:10 PM, "binxycraft" notifications@github.com wrote:

It is definitely bad sectors and not the new USB bridge (I tried with 2 different ones that are both very new and working well with other drives). This drive has traveled with me across the pacific ocean twice and has been stored in a few different locations due to permanent relocation's back and forth. I have been running ddrescue since it was suggested 4 days ago. I believe it is finally almost done with its process. It is on the "Splitting failed blocks" stage if that means anything. I am far from an expert at any of this but I used it to create a .img file. Will I then be able to use reallymine and point to the image file directly? Here are some of the stats if they are somehow helpful: rescued: 1500GB errsize: 782 kB errors: 183 ipos/opos: 3188 MB

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/andlabs/reallymine/issues/2#issuecomment-227222619, or mute the thread https://github.com/notifications/unsubscribe/AQE6xaDvDWKNgct0GfoeTAnNpVPnMr0bks5qNteSgaJpZM4I3ofb .

koalaeagle commented 8 years ago

This drive is a 1.5TB drive. It actually just now finished with 197 errors/ 767kB errsize. I now have a "ddrescue.img" file on another drive. Would I just mount this? Or do I still run reallymine on the .img first?

MrDecay commented 8 years ago

Your. Almost there. Run reallymine on the ddrescue image and decrypt. To another image Then extract the Decrypted image And files On Jun 20, 2016 1:18 PM, "binxycraft" notifications@github.com wrote:

This drive is a 1.5TB drive. It actually just now finished with 197 errors/ 767kB errsize. I now have a "ddrescue.img" file on another drive. Would I just mount this? Or do I still run reallymine on the .img first?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/andlabs/reallymine/issues/2#issuecomment-227225025, or mute the thread https://github.com/notifications/unsubscribe/AQE6xajP0U_4sMYpNVjPP49Bj0Ju3CSuks5qNtmPgaJpZM4I3ofb .

andlabs commented 8 years ago

Did you also make a .log or .map (they recently changed the name) file? That will tell you which sectors were bad.

koalaeagle commented 8 years ago

Thanks for the help so far I will begin that last step soon. For now here is the log file:

http://pastebin.com/ppUX5J3M

koalaeagle commented 8 years ago

At 6600 MB / 1430799 MB so far so that is a good sign!

andlabs commented 8 years ago

Do you happen to remember the partitioning scheme of your drive?

koalaeagle commented 8 years ago

It was exclusively used on Windows so just one huge ntfs partition afaik. However the drive came by default. I don't think I had even touched linux back then (I am still new!)

andlabs commented 8 years ago

Hm... I'm not aware of any tools that will tell you what each block marked - on that log file maps to then; sorry. I'm sure one exists, though...

MrDecay commented 8 years ago

Look int ntfsbad part of the ddrutility suite On Jun 20, 2016 5:41 PM, "Pietro Gagliardi" notifications@github.com wrote:

Hm... I'm not aware of any tools that will tell you what each block marked - on that log file maps to then; sorry. I'm sure one exists, though...

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/andlabs/reallymine/issues/2#issuecomment-227291854, or mute the thread https://github.com/notifications/unsubscribe/AQE6xRd3aK-84SkL5U-dDOWQXDsRRqSrks5qNxcqgaJpZM4I3ofb .

binkz1337 commented 8 years ago

Okay I have finished running reallymine on the "ddrescue.img" and have created "yesmine.img". The operation completed successfully. I am unable to mount the .img file. I am using Linux Mint and it has a built-in mount tool which doesn't appear to work. I also tried in Windows 10 since its NTFS and Windows 10 can mount .img files naively. I decided to use fdisk -l on the .img file and got the following result:

Disk yesmine.img: 1500.3 GB, 1500301910016 bytes 255 heads, 63 sectors/track, 182401 cylinders, total 2930277168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x8f1c6711

Disk yesmine.img doesn't contain a valid partition table

So it seems my partition table is damaged?

Figured I would ask you guys what the best course of action from here would be.

Thanks in advance for the help!

MrDecay commented 8 years ago

Testdisk yesmine.img and scan to see if it finds partition. If not I've used osfmount in windows . use a partition offset. Of 63 or 2048 On Jun 22, 2016 12:55 PM, "binkz1337" notifications@github.com wrote:

Okay I have finished running reallymine on the "ddrescue.img" and have created "yesmine.img". The operation completed successfully. I am unable to mount the .img file. I am using Linux Mint and it has a built-in mount tool which doesn't appear to work. I also tried in Windows 10 since its NTFS and Windows 10 can mount .img files naively. I decided to use fdisk -l on the .img file and got the following result:

Disk yesmine.img: 1500.3 GB, 1500301910016 bytes 255 heads, 63 sectors/track, 182401 cylinders, total 2930277168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x8f1c6711

Disk yesmine.img doesn't contain a valid partition table

So it seems my partition table is damaged?

Figured I would ask you guys what the best course of action from here would be.

Thanks in advance for the help!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/andlabs/reallymine/issues/2#issuecomment-227824999, or mute the thread https://github.com/notifications/unsubscribe/AQE6xU0SzDEZPSOELDqfqem9_2PEP6LWks5qOXcFgaJpZM4I3ofb .

MrDecay commented 8 years ago

I used the offset because in my case somebody initialized. Drive before the decryption. Destroying the mbr of the drive On Jun 22, 2016 1:02 PM, "Tony Salinas" adroflaredo@gmail.com wrote:

Testdisk yesmine.img and scan to see if it finds partition. If not I've used osfmount in windows . use a partition offset. Of 63 or 2048 On Jun 22, 2016 12:55 PM, "binkz1337" notifications@github.com wrote:

Okay I have finished running reallymine on the "ddrescue.img" and have created "yesmine.img". The operation completed successfully. I am unable to mount the .img file. I am using Linux Mint and it has a built-in mount tool which doesn't appear to work. I also tried in Windows 10 since its NTFS and Windows 10 can mount .img files naively. I decided to use fdisk -l on the .img file and got the following result:

Disk yesmine.img: 1500.3 GB, 1500301910016 bytes 255 heads, 63 sectors/track, 182401 cylinders, total 2930277168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x8f1c6711

Disk yesmine.img doesn't contain a valid partition table

So it seems my partition table is damaged?

Figured I would ask you guys what the best course of action from here would be.

Thanks in advance for the help!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/andlabs/reallymine/issues/2#issuecomment-227824999, or mute the thread https://github.com/notifications/unsubscribe/AQE6xU0SzDEZPSOELDqfqem9_2PEP6LWks5qOXcFgaJpZM4I3ofb .

andlabs commented 8 years ago

The ddrescue log says your MBR wasn't damaged, but if someone overwrote the MBR, then yeah... :/ Hopefully testdisk can find the partition boundaries.

MrDecay commented 8 years ago

Any luck on your recovery? On Jun 22, 2016 1:12 PM, "Pietro Gagliardi" notifications@github.com wrote:

The ddrescue log says your MBR wasn't damaged, but if someone overwrote the MBR, then yeah... :/ Hopefully testdisk can find the partition boundaries.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/andlabs/reallymine/issues/2#issuecomment-227830198, or mute the thread https://github.com/notifications/unsubscribe/AQE6xZEFY9d0jaOVXTVt8IRzWozi3vcHks5qOXr8gaJpZM4I3ofb .

binkz1337 commented 8 years ago

No luck. I tried the entire recovery process again but seemed to get the same result! Testdisk can't seem to find anything either. Yet to try osfmount Just realized I was logged in with an old account on one computer I was replying from hopefully it was apparent that I am binxycraft :P

athomic1 commented 8 years ago

Okay, there's something I'm not clear on, here. Just to see where you are:

  1. It sounds like you've run ddrescue, and created a separate image of the failing drive. Unless you say otherwise, I'm assuming that's correct.
  2. You're now trying to mount the image, but having no luck. That SOUNDS like the stage you're at right now.
  3. What I have NOT seen is any mention of having run the image through reallymine, and generating a decrypted image. That would be a new, SEPARATE image from the one you created with ddrescue, and THAT's the one you would have to mount. The ddrescued image is still encrypted, and highly unlikely to contain anything directly mountable.

Having just said that, I will note that these drives generally came with an unencrypted CD image stuffed in near the end of the drive, so some scanning software MIGHT find a mountable image up there. That's obviously not what you're looking for, of course. Again, running ddrescue is only the first step to recovering your data. You still have to run the resulting image through reallymine, which will generate a NEW image containing the (hopefully) unencrypted data. You'll know it should be working if it gets past 'Finding Key Sector'. At least, I'm pretty sure. If you haven't already done that, it should be your next step.

athomic1 commented 8 years ago

Okay, looks like I missed the part where you did run reallymine to create the yesmine image. Do you remember it specifically saying it found the key sector? I would assume so, but I 'd like to know for sure, if you recall.

Assuming the image was successfully decrypted, it should contain readable plain text that you could search the raw image for. That would include file names, as well as text file content. If you can find anything you specifically recall, like a unique filename, or some text you wrote at some point, you could grep the image for it to see if you have at least something recognizable in there. If we can establish that much, we can probably recover at least some of your content.

I think grep is probably the tool of choice, here. I'd probably start with a line like this:

cat yesmine.img | grep 'myUniqueText'

using your own text in place of myUniqueText, of course. It's likely to take awhile, since it could be anywhere, but it gives you at least an idea whether you have something recoverable. Then we can get into filesystem recovery tools.

You could also try piping the image through 'strings' and see if anything recognizable pops up. Though it probably won't pull up anything specific to your filesystem immediately, it might be quicker picking up any plain text, so there's that.

Best of luck!

binkz1337 commented 8 years ago

It does say that it found the key sector at the start of running reallymine.

Weirdly enough the original hard drive occasionally pops up in both Windows and Linux and shows its name "Full of win" any attempts to explore the drive end up in it disappearing again.

I will try out grep and see if I can find anything. Trying to remember exactly what was on there. It has been several years! haha.

Thanks for the help thus far.

binkz1337 commented 8 years ago

Forgot to mention. I did try osfmount with an offset of both 63 and 2048 but both times could not read the mounted disk and it was unable to detect the file system.

athomic1 commented 8 years ago

Maybe try 4096?

binkz1337 commented 8 years ago

Tried 4096 and also the grep command. Grep gave an out of memory error and 4096 didn't work :/

andlabs commented 8 years ago

Can you provide a hexdump of the first 4096 bytes?

binkz1337 commented 8 years ago

Sorry for the long delay - life got in the way of this project!

I have never done a hexdump before and am not entirely sure what I am doing but hopefully this is it:

00000000  53 49 c2 5a 9c 07 38 f6  71 84 48 b9 53 ad 2c 75  |SI.Z..8.q.H.S.,u|
00000010  85 b2 74 11 40 39 af af  ac b4 3e ee b1 0c 33 87  |..t.@9....>...3.|
00000020  3c c4 77 98 6f 15 4e 8f  eb 20 1e 3f 9f 10 19 9a  |<.w.o.N.. .?....|
00000030  20 d9 aa b4 be 32 fe 05  11 a1 75 57 bd 64 f8 2c  | ....2....uW.d.,|
00000040  98 5c cb 83 b8 45 2e a6  e2 df cc ce 70 d6 1a 13  |.\...E......p...|
00000050  3d f3 b5 32 0a d3 a4 d7  c0 b8 2e 03 68 d3 20 33  |=..2........h. 3|
00000060  58 94 c3 69 0e dc 68 73  43 5a 3c d5 07 0f ba 90  |X..i..hsCZ<.....|
00000070  ae 4b 0b a6 39 21 51 cf  be c1 14 6e d7 88 8b 5f  |.K..9!Q....n..._|
00000080  a9 ce e6 1e 3a 84 72 94  7e 2f a0 16 bf aa 6d c4  |....:.r.~/....m.|
00000090  54 a4 80 57 00 4f 56 a4  f6 d2 32 55 82 c5 2a b0  |T..W.OV...2U..*.|
000000a0  20 3c 7a 7c 75 d7 3a 71  e8 ec de d9 a1 b6 fe fb  | <z|u.:q........|
000000b0  e1 af 5d d3 a4 a0 cd 42  7c a3 bc f7 14 b5 da 18  |..]....B|.......|
000000c0  08 8d de 49 03 bf 1b 64  15 c8 4c 2b 46 de f1 e8  |...I...d..L+F...|
000000d0  15 1a 54 1a d2 37 43 0b  5c 58 f0 35 ec 26 ea e7  |..T..7C.\X.5.&..|
000000e0  37 dc 40 1f c1 13 30 d6  2a 6f 85 4d 9a ce 5b ff  |7.@...0.*o.M..[.|
000000f0  ae 61 77 95 4b 03 9f 1b  a4 35 c1 6c cf ed 9e c7  |.aw.K....5.l....|
00000100  65 0e 47 0b 89 94 e1 98  5d 0d fe f3 0d 99 a5 75  |e.G.....]......u|
00000110  29 bf e6 80 c3 df 66 e1  00 27 b4 43 2a 27 d9 27  |).....f..'.C*'.'|
00000120  3e d3 78 56 c3 a9 e0 4f  c0 90 ee 4b b8 e7 70 a9  |>.xV...O...K..p.|
00000130  69 30 79 86 a9 ad 0a 77  9e ce 46 79 c1 c6 25 65  |i0y....w..Fy..%e|
00000140  31 6a ec a6 77 36 21 27  8d 8c 69 e7 b0 5d 88 8b  |1j..w6!'..i..]..|
00000150  19 88 61 aa 55 90 f6 5e  9b 98 7e a8 f4 c7 1a ca  |..a.U..^..~.....|
00000160  d3 22 ef 0e f0 03 cf 9c  60 e0 66 4d 93 70 c1 50  |."......`.fM.p.P|
00000170  d9 1c a0 a6 c2 6c 4c 5f  c7 9d e2 0d 19 4d 78 95  |.....lL_.....Mx.|
00000180  58 24 03 60 34 06 67 67  56 c0 83 68 45 7c ab 9a  |X$.`4.ggV..hE|..|
*
000001b0  a3 dc ea 67 a8 eb c2 45  11 67 1c 8f 14 e9 9c f1  |...g...E.g......|
000001c0  9c 05 60 6e 38 d3 c2 cb  9a dd f6 06 ad b6 3c 54  |..`n8.........<T|
000001d0  58 24 03 60 34 06 67 67  56 c0 83 68 45 7c ab 9a  |X$.`4.ggV..hE|..|
*
000001f0  75 11 d6 72 5b 26 70 2d  26 75 23 c2 7b 96 5f 93  |u..r[&p-&u#.{._.|
00000200  58 24 03 60 34 06 67 67  56 c0 83 68 45 7c ab 9a  |X$.`4.ggV..hE|..|
*
00001000

If this is wrong then let me know the best way of achieving this either on Windows 10 or Linux Mint and I will try it again!

athomic1 commented 8 years ago

That doesn't look right. Assuming your drive was originally prepped to an MBR layout, which these should be, there should be a '55 aa' at the very end of line 0..1f0 near the end of your dump, there, if the image were properly decrypted. I would have expected just a bit of readable text, too, for one or more error messages the bootloader might need.

Something else interesting are the lines just before each asterisk: lines 180, 1b0, and 200 are identical. The asterisk indicates a repeating pattern of the preceding line. If you were to force a verbose dump, say, using 'hexdump -v', you'd see that same line repeated several times. Usually, you only get that from a long block of zero bytes, but when you 'decrypt' such a block the way ReallyMine and these interfaces do, you get a repeating pattern like this. I saw it myself when I was working on my drive. Especially telling is that line 200 probably repeats all the way to line 1000. That's basically everything after the first 512 bytes; every sector past the MBR is "empty."

By any chance, did you happen to repartition, or attempt to repartition the drive WITHOUT going through the original interface? For example, after removing it from the original enclosure, did you perhaps put it into a generic enclosure and attempt to access the drive through that? If the system prompted you to 'prepare the drive for use' or something like that, I think there's a good chance you unintentionally overwrote the MBR. This is a BAD thing, but not necessarily a total loss. What you might have to do is run a scanner against the image to locate possible/probable partitions. I'll leave it to others to recommend options there, for now.

What I would suggest, meanwhile, is to extract just the first sector directly from the original drive (not the image), and see if that looks more like an MBR. Just post the hex dump here like you did before, if you want help identifying it. If the raw image IS a valid MBR, the original correct one was almost certainly overwritten, and you might need to do a bit of forensics to recover the partition. Assuming you didn't do something "weird," like repartition the unit with multiple partitions, this might actually be fairly simple.