Open stdedos opened 4 months ago
Hi,
Thank you for this report.
I tried to reproduce this issue but not successfully. I created a USB disk using Mac OS (disk utility) and ran clean, remove, and format a new empty USB key. Partclone does not have the error message from my Linux machine.
$ docker run -it --privileged=true --device=/dev/sdc1 tlinux/partclone partclone.hfsp -d -c -s /dev/sdc1 -o dhfsp.img
Unable to find image 'tlinux/partclone:latest' locally
latest: Pulling from tlinux/partclone
9c704ecd0c69: Pull complete
...
0a56c68b2479: Pull complete
Digest: sha256:29e78676175a0753a8848e028529a037aec2846e1ed2f9cc2fd4327fbc8408e2
Status: Downloaded newer image for tlinux/partclone:latest
Partclone v0.3.31 http://partclone.org
Starting to clone device (/dev/sdc1) to image (dhfsp.img)
Reading Super Block
Calculating bitmap... Please wait...
Elapsed: 00:00:01, Remaining: 00:00:00, Completed: 100.00%
Total Time: 00:00:01, 100.00% completed!
done!
File system: HFS Plus
Device size: 123.1 GB = 30043904 Blocks
Space in use: 13.9 GB = 3391837 Blocks
Free Space: 109.2 GB = 26652067 Blocks
Block size: 4096 Byte
Elapsed: 00:01:30, Remaining: 00:00:00, Completed: 100.00%, Rate: 9.26GB/min,
Current block: 30043904, Total block: 30043904, Complete: 100.00%
Total Time: 00:01:30, Ave. Rate: 9.26GB/min, 100.00% completed!
Syncing... OK!
Partclone successfully cloned the device (/dev/sdc1) to the image (dhfsp.img)
Cloned successfully.
and I can't run partclone.hfsp outside the docker image due to different GLIBC.
My partition_signature is similar:
sudo dd if=/dev/sda1 of=partition_signature.bin bs=4096 count=1
00000400: 482b 0004 8000 2100 4846 534a 0000 0396 H+....!.HFSJ....
00000410: defb 99f5 e2b4 fd6f 0000 0000 defb 2975 .......o......)u
00000420: 0000 0052 0000 0011 0000 1000 01ca 6f00 ...R..........o.
00000430: 0196 ada3 0041 0002 0001 0000 0001 0000 .....A..........
00000440: 0000 009c 0000 027e 0000 0000 0000 0001 .......~........
00000450: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000460: 0000 0000 0000 0000 1349 d693 9282 f76f .........I.....o
00000470: 0000 0000 0039 5000 0039 5000 0000 0395 .....9P..9P.....
As you mentioned we have the same HFS_Signature(0x482b), so I guess we can do some tests:
I made a static partclone.hfsp, and please try again and check /var/log/partclone.log.
download link: https://file.io/1zqKPM48Mfds
wget https://file.io/1zqKPM48Mfds -o partclone.hfsp
sha256sum ./src/partclone.hfsp 4597a5a38f8810e1e0595fcba5bb66f5d16208bf409ecba3856fb73dd37b1858 ./src/partclone.hfsp
check the log file and grep 'HFS_Plus signature'
cat /var/log/partclone.log |grep signature
hfsplusclone.c: HFS_Plus signature original endian 0
hfsplusclone.c: HFS_Plus signature 482b
Please kindly share your test, thank you.
I tried your command, but it does not work:
$ docker run -it -v "`pwd`:`pwd`" --privileged=true --device=/dev/sda1 tlinux/partclone partclone.hfsp -d -c -s /dev/sda1 --overwrite "`pwd`/dhfsp.img"
Partclone v0.3.31 http://partclone.org
Starting to clone device (/dev/sda1) to image (/home/stdedos/Documents/WorkBulk/dhfsp.img)
Reading Super Block
hfsplusclone.c: HFS_Plus Volume '/dev/sda1' is scheduled for a check or it was shutdown
uncleanly. Please fix it by fsck.
Partclone fail, please check /var/log/partclone.log !
$ docker run -it -v "`pwd`:`pwd`" --privileged=true --device=/dev/sda1 tlinux/partclone bash
root@b42d9df5a934:/partclone/tests# fsck /dev/sda1
fsck from util-linux 2.39.3
** /dev/sda1
Executing fsck_hfs (version 540.1-Linux).
** Checking Journaled HFS Plus volume.
The volume name is Install macOS Sierra
** Checking extents overflow file.
** Checking catalog file.
** Checking multi-linked files.
** Checking catalog hierarchy.
** Checking extended attributes file.
** Checking volume bitmap.
** Checking volume information.
** The volume Install macOS Sierra appears to be OK.
root@b42d9df5a934:/partclone/tests# fsck /dev/sda
fsck from util-linux 2.39.3
e2fsck 1.47.0 (5-Feb-2023)
ext2fs_open2: Bad magic number in super-block
fsck.ext2: Superblock invalid, trying backup blocks...
fsck.ext2: Bad magic number in super-block while trying to open /dev/sda
The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem. If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>
or
e2fsck -b 32768 <device>
Found a dos partition table in /dev/sda
(-v "`pwd`:`pwd`"
- I'd like the output file in the end, --overwrite
- too many tests, I guess 😅)
Here's the signature: partition_signature.tar.gz
Downloading the file failed. Would you mind pushing a branch? I'll rebuild locally instead
... I know a filesystem is a filesystem, but could you please try to create a recovery USB instead? https://support.apple.com/en-us/101578#create "Maybe the layout is a bit more special" in that case.
Hi,
Please follow this to run fsck to check and repair macOS disks.
or you want to work with linux docker, please replace as fsck.hfsplus, because the default fsck is for Linux extended file system eg: ext4, so
my example:
docker run -it --privileged=true --device=/dev/sdc1 tlinux/partclone /bin/bash
root@fb09e7fb3fa2:/partclone/tests# fsck.hfsplus /dev/sdc1
** /dev/sdc1
Executing fsck_hfs (version 540.1-Linux).
** Checking Journaled HFS Plus volume.
The volume name is 未命名
** Checking extents overflow file.
** Checking catalog file.
** Checking multi-linked files.
** Checking catalog hierarchy.
** Checking extended attributes file.
** Checking volume bitmap.
** Checking volume information.
** The volume 未命名 appears to be OK.
I am pretty sure that fsck
can auto-detect/invoke for the correct fs.
root@b42d9df5a934:/partclone/tests# fsck.hfsplus /dev/sda1
** /dev/sda1
Executing fsck_hfs (version 540.1-Linux).
** Checking Journaled HFS Plus volume.
The volume name is Install macOS Sierra
** Checking extents overflow file.
** Checking catalog file.
** Checking multi-linked files.
** Checking catalog hierarchy.
** Checking extended attributes file.
** Checking volume bitmap.
** Checking volume information.
** The volume Install macOS Sierra appears to be OK.
root@b42d9df5a934:/partclone/tests# fsck.hfsplus -f /dev/sda1
** /dev/sda1
Executing fsck_hfs (version 540.1-Linux).
** Checking Journaled HFS Plus volume.
The volume name is Install macOS Sierra
** Checking extents overflow file.
** Checking catalog file.
** Checking multi-linked files.
** Checking catalog hierarchy.
** Checking extended attributes file.
** Checking volume bitmap.
** Checking volume information.
** The volume Install macOS Sierra appears to be OK.
The volume Install macOS Sierra appears to be OK
- similar to my previous comment.
I don't have "regular" macOS access - so I cannot check it via the OS.
... but I'd like to keep the disk image handy "just in case", so if need be, I don't have to spend a week+ to re-create it.
Great! The fsck.hfsplus works well and should be ready to backup.
I downloaded your partition_signature.bin file and tried to initial the clone. I have a similar error message:
Partclone v0.3.23 http://partclone.org
Starting to clone device (/dev/sdc1) to image (hfsp.img)
Reading Super Block
hfsplusclone.c: HFS_Plus Volume '/dev/sdc1' is scheduled for a check or it was shutdown
uncleanly. Please fix it by fsck.
Now, your partition is clean so we can ignore that and go back to do Signature test.
I updated the code to the master branch, you can clone/rebuild.
Could you please dump super block and then run partclone.hfsplus again?
... I know a filesystem is a filesystem, but could you please try to create a recovery USB instead? https://support.apple.com/en-us/101578#create
"Maybe the layout is a bit more special" in that case.
I will give it a try later.
I'm surprised how just https://github.com/Thomas-Tsai/partclone/commit/d6f993b4d6dc7c0d02bc2f7203d9f26ddc3fc678 could "fix everything" 😕
It seems that it will work in the end:
Starting to clone device (/dev/sda1) to image (dhfsp.img)
UID is root.
source=/dev/sda1, target=dhfsp.img
open source file/device /dev/sda1
open target file/device dhfsp.img
Initiate image options - version 0002
Initial image hdr - get Super Block from partition
Reading Super Block
hfsplusclone.c: HFS_Plus signature original endian 0
hfsplusclone.c: HFS_Plus signature 482b
hfsplusclone.c: blockSize:4096
hfsplusclone.c: totalBlocks:30504704
hfsplusclone.c: freeBlocks:29217719
hfsplusclone.c: superBlockUsedBlocks:1286985
hfsplusclone.c: logicalSize: 0x303a0000000000
hfsplusclone.c: clumpSize: 3813376
hfsplusclone.c: totalBlocks: 931
hfsplusclone.c: exten 0 startBlock: 1
hfsplusclone.c: exten 0 blockCount: 931
hfsplusclone.c: exten 1 startBlock: 0
hfsplusclone.c: exten 1 blockCount: 0
hfsplusclone.c: exten 2 startBlock: 0
hfsplusclone.c: exten 2 blockCount: 0
hfsplusclone.c: exten 3 startBlock: 0
hfsplusclone.c: exten 3 blockCount: 0
hfsplusclone.c: exten 4 startBlock: 0
hfsplusclone.c: exten 4 blockCount: 0
hfsplusclone.c: exten 5 startBlock: 0
hfsplusclone.c: exten 5 blockCount: 0
hfsplusclone.c: exten 6 startBlock: 0
hfsplusclone.c: exten 6 blockCount: 0
hfsplusclone.c: exten 7 startBlock: 0
hfsplusclone.c: exten 7 blockCount: 0
hfsplusclone.c: logicalSize: 0x700000000000
hfsplusclone.c: clumpSize: 7340032
hfsplusclone.c: totalBlocks: 1792
hfsplusclone.c: exten 0 startBlock: 5029
hfsplusclone.c: exten 0 blockCount: 1792
hfsplusclone.c: exten 1 startBlock: 0
hfsplusclone.c: exten 1 blockCount: 0
hfsplusclone.c: exten 2 startBlock: 0
hfsplusclone.c: exten 2 blockCount: 0
hfsplusclone.c: exten 3 startBlock: 0
hfsplusclone.c: exten 3 blockCount: 0
hfsplusclone.c: exten 4 startBlock: 0
hfsplusclone.c: exten 4 blockCount: 0
hfsplusclone.c: exten 5 startBlock: 0
hfsplusclone.c: exten 5 blockCount: 0
hfsplusclone.c: exten 6 startBlock: 0
hfsplusclone.c: exten 6 blockCount: 0
hfsplusclone.c: exten 7 startBlock: 0
hfsplusclone.c: exten 7 blockCount: 0
hfsplusclone.c: logicalSize: 0xa00400000000
hfsplusclone.c: clumpSize: 77594624
hfsplusclone.c: totalBlocks: 18944
hfsplusclone.c: exten 0 startBlock: 319397
hfsplusclone.c: exten 0 blockCount: 18944
hfsplusclone.c: exten 1 startBlock: 0
hfsplusclone.c: exten 1 blockCount: 0
hfsplusclone.c: exten 2 startBlock: 0
hfsplusclone.c: exten 2 blockCount: 0
hfsplusclone.c: exten 3 startBlock: 0
hfsplusclone.c: exten 3 blockCount: 0
hfsplusclone.c: exten 4 startBlock: 0
hfsplusclone.c: exten 4 blockCount: 0
hfsplusclone.c: exten 5 startBlock: 0
hfsplusclone.c: exten 5 blockCount: 0
hfsplusclone.c: exten 6 startBlock: 0
hfsplusclone.c: exten 6 blockCount: 0
hfsplusclone.c: exten 7 startBlock: 0
hfsplusclone.c: exten 7 blockCount: 0
hfsplusclone.c: logicalSize: 0xf00600000000
hfsplusclone.c: clumpSize: 116391936
hfsplusclone.c: totalBlocks: 28416
hfsplusclone.c: exten 0 startBlock: 6821
hfsplusclone.c: exten 0 blockCount: 28416
hfsplusclone.c: exten 1 startBlock: 0
hfsplusclone.c: exten 1 blockCount: 0
hfsplusclone.c: exten 2 startBlock: 0
hfsplusclone.c: exten 2 blockCount: 0
hfsplusclone.c: exten 3 startBlock: 0
hfsplusclone.c: exten 3 blockCount: 0
hfsplusclone.c: exten 4 startBlock: 0
hfsplusclone.c: exten 4 blockCount: 0
hfsplusclone.c: exten 5 startBlock: 0
hfsplusclone.c: exten 5 blockCount: 0
hfsplusclone.c: exten 6 startBlock: 0
hfsplusclone.c: exten 6 blockCount: 0
hfsplusclone.c: exten 7 startBlock: 0
hfsplusclone.c: exten 7 blockCount: 0
hfsplusclone.c: logicalSize: 0
hfsplusclone.c: clumpSize: 0
hfsplusclone.c: totalBlocks: 0
hfsplusclone.c: exten 0 startBlock: 0
hfsplusclone.c: exten 0 blockCount: 0
hfsplusclone.c: exten 1 startBlock: 0
hfsplusclone.c: exten 1 blockCount: 0
hfsplusclone.c: exten 2 startBlock: 0
hfsplusclone.c: exten 2 blockCount: 0
hfsplusclone.c: exten 3 startBlock: 0
hfsplusclone.c: exten 3 blockCount: 0
hfsplusclone.c: exten 4 startBlock: 0
hfsplusclone.c: exten 4 blockCount: 0
hfsplusclone.c: exten 5 startBlock: 0
hfsplusclone.c: exten 5 blockCount: 0
hfsplusclone.c: exten 6 startBlock: 0
hfsplusclone.c: exten 6 blockCount: 0
hfsplusclone.c: exten 7 startBlock: 0
hfsplusclone.c: exten 7 blockCount: 0
256 blocks per checksum
memory needed: 5910244 bytes
bitmap 3813088 bytes, blocks 2*1048576 bytes, checksum 4 bytes
initial main bitmap pointer 0x27d2e00
Initial image hdr - read bitmap table
Calculating bitmap... Please wait...
hfsplusclone.c: HFS_Plus signature original endian 0
hfsplusclone.c: HFS_Plus signature 482b
hfsplusclone.c: tb = 30504704
hfsplusclone.c: extent_block = 0
hfsplusclone.c: allocation_exten = 0
hfsplusclone.c: allocation_start_block = 4096
hfsplusclone.c: allocation_block_size = 3813376
hfsplusclone.c: next exten
hfsplusclone.c: extent_bitmap:-239235056
hfsplusclone.c: bfree:29217719
hfsplusclone.c: bused:1286985
hfsplusclone.c: tb = 30504704
hfsplusclone.c: extent_block = 30504704
hfsplusclone.c: allocation_exten = 1
hfsplusclone.c: allocation_start_block = 0
hfsplusclone.c: allocation_block_size = 0
hfsplusclone.c: tb = 30504704
hfsplusclone.c: extent_block = 30504704
hfsplusclone.c: allocation_exten = 2
hfsplusclone.c: allocation_start_block = 0
hfsplusclone.c: allocation_block_size = 0
hfsplusclone.c: tb = 30504704
hfsplusclone.c: extent_block = 30504704
hfsplusclone.c: allocation_exten = 3
hfsplusclone.c: allocation_start_block = 0
hfsplusclone.c: allocation_block_size = 0
hfsplusclone.c: tb = 30504704
hfsplusclone.c: extent_block = 30504704
hfsplusclone.c: allocation_exten = 4
hfsplusclone.c: allocation_start_block = 0
hfsplusclone.c: allocation_block_size = 0
hfsplusclone.c: tb = 30504704
hfsplusclone.c: extent_block = 30504704
hfsplusclone.c: allocation_exten = 5
hfsplusclone.c: allocation_start_block = 0
hfsplusclone.c: allocation_block_size = 0
hfsplusclone.c: tb = 30504704
hfsplusclone.c: extent_block = 30504704
hfsplusclone.c: allocation_exten = 6
hfsplusclone.c: allocation_start_block = 0
hfsplusclone.c: allocation_block_size = 0
hfsplusclone.c: tb = 30504704
hfsplusclone.c: extent_block = 30504704
hfsplusclone.c: allocation_exten = 7
hfsplusclone.c: allocation_start_block = 0
hfsplusclone.c: allocation_block_size = 0
Total Time: 00:00:01, Ave. Rate: 60.00byte/min, 100.00% completed!
check main bitmap pointer 0x27d2e00
Writing super block and bitmap...
io_all: write 110, 0 left.
io_all: write 3813088, 0 left.
io_all: write 4, 0 left.
done!
print image information
MODE: clone
DEBUG: 2
...
Theoretically, the docker image is "latest":
root@b42d9df5a934:/partclone-clone# ../partclone/src/partclone.hfsp -v
Partclone : v0.3.31 (0.3.31)
From your experienced position, should I be able to re-format the flash drive with the dhfsp.img
, and end up "exactly with a bootable recovery drive"?
I have no idea about 'bootable recovery drive' but you can try using this image to create another one.
Partclone is tool for imaging individual partitions (like sda1), NOT disk(sda), so normally, the disk related partition table and MBR is missing. Of course you can re-run createinstallmedia then restore data from your partclone image.
If you're concerned about handling special cases, using dd or Clonezilla(default with partclone) might be a better option.
The recovery drive from mac not works for me. I am not familar with mac os. let me give up.
... I know a filesystem is a filesystem, but could you please try to create a recovery USB instead? https://support.apple.com/en-us/101578#create "Maybe the layout is a bit more special" in that case.
I will give it a try later.
any partclone issue is welcome.
Of course you can re-run createinstallmedia then restore data from your partclone image.
I cannot 😅 Apple, in its infinite wisdom, has "things" not be "backwards compatible" with themselves. (Or rather, it will be a complex and annoying ordeal).
So I cannot use a "nowadays" Mac to burn a Sierra - regardless if my laptop supports only that. ofc, the laptop supporting "only that" is also annoying (but at least that one I get)
If you're concerned about handling special cases, using dd or Clonezilla(default with partclone) might be a better option.
I thought about it. But my flash drive is 128GB 😱 I don't want to store a 128GB img "just in case" I need that again. ... Storing 5G is acceptable, though
I have some "obvious" expected issues at recovery:
# partclone.hfsp -d -r -s macOSSierraInstallUSB.partclone.img -o /dev/sdb1
Partclone v0.3.31 http://partclone.org
Starting to restore image (macOSSierraInstallUSB.partclone.img) to device (/dev/sdb1)
Reading Super Block
Calculating bitmap... Please wait...
Target partition size(31041 MB) is smaller than source(124948 MB). Use option -C to disable size checking(Dangerous).
Partclone fail, please check /var/log/partclone.log !
root@806aa000b95e:/home/stdedos/Videos/macOSSierra# fsck.hfsplus -df /dev/sdb1
** /dev/sdb1
Using cacheBlockSize=32K cacheTotalBlock=1024 cacheSize=32768K.
Executing fsck_hfs (version 540.1-Linux).
Block 60626942 is not an MDB or Volume Header
** Checking Journaled HFS Plus volume.
Invalid number of allocation blocks
(4294967295, 0)
IVChk - volume header total allocation blocks is greater than device size
volume allocation block count 30504704 device allocation block count 7578368
** The volume could not be verified completely.
volume check failed with error 7
volume type is pure HFS+
primary MDB is at block 0 0x00
alternate MDB is at block 0 0x00
primary VHB is at block 2 0x02
alternate VHB is at block 60626942 0x39d17fe
sector size = 512 0x200
VolumeObject flags = 0x03
total sectors for volume = 60626944 0x39d1800
total sectors for embedded volume = 0 0x00
I still have my source intact, so I can retry stuff. However, unless necessary, I'd like to treat it as RO (until I can have a working backup).
Do you think there is something I could reasonably do here?
On a tangent, do you expect that ^C
should work? Because I get
# partclone.hfsp -d -r -s macOSSierraInstallUSB.partclone.img -o /dev/sdb1 -C
Partclone v0.3.31 http://partclone.org
Starting to restore image (macOSSierraInstallUSB.partclone.img) to device (/dev/sdb1)
Reading Super Block
Calculating bitmap... Please wait...
done!
File system: HFS Plus
Device size: 124.9 GB = 30504704 Blocks
Space in use: 5.3 GB = 1286985 Blocks
Free Space: 119.7 GB = 29217719 Blocks
Block size: 4096 Byte
^C^Csed: 00:00:12, Remaining: 00:00:16, Completed: 41.77%, 11.01GB/min,
Current block: 1653748, Total block: 30504704, Complete: 5.42%
^\^C^C^C^C^C^\^\^\^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^Z^C^C^C
The first restore to sdb1 is failed.
Target partition size(31041 MB) is smaller than source(124948 MB). Use option -C to disable size checking(Dangerous).
to prevent this, preparing a large disk or partition is necessary. The alternative is to resize your partition before backup.
The option '-C' is not for this case, it's only for some dangerous test.
The alternative is to resize your partition before backup.
"I didn't create the partition", and I guess the OS decided to take the full size of that USB 😕
I thought that, since the file was 5,0G
, it could be restored in anything > 5,0G
sized.
I tried to check if I could do some losetup
magics to "resize" the partition - but it was not possible.
... good news for me is, -C
actually works!
Yes, it causes errors at the end of the restoration, and while 5% finishes in seconds, ~93.86% takes much more time, and it fails:
...
read more: blocks_read = 256 and copied = 1286656
read more: blocks_read = 73 and copied = 1286912
# PARTIAL CHUNK
read more: check latest chunk's checksum covering 73 blocks
target seek ERROR:Invalid argument
#
But the USB works (after some manual, intelligent dd
-ings at the first MB)
I guess I can now try to resize the source, and re-image my source
It seems to be impossible to do: https://superuser.com/a/412685/533196
Well, you can only resize volumes on disks that use the GUID partition table, not MBR disks.
See Apple Support:
Mac OS X v10.5 Leopard can resize any disk partition except a Master Boot Record Partition. You can use Disk Utility to see if a partition is a Master Boot Record Partition, and also to resize eligible partitions.
Important: Resizing a Master Boot Record partition will erase all data on it, so be sure to back up important files on affected partitions first.
So I guess I will live with that limitation.
I have already spent ... about the same time to create that flash drive, as I have spent "backing it up" 😓 (which is already 100x times more than someone should have to spend in a "formatting")
Thank you for your time and effort Thomas 🙃
I have used
docker run -it tlinux/partclone
to subsequently buildpartclone
, like so:Then, I extracted the resulting
docker cp a2f52e88be97:/partclone/src/partclone.hfsp .
, and used it like so:Which I don't believe to be accurate.
I have verified the partition signature to be correct:
i.e.: https://github.com/Thomas-Tsai/partclone/blob/f5dbe57ecfffdb8b6de115960183d69eb68e70e6/src/hfsplusclone.h#L69-L73
Given how the error message appears, and the code
https://github.com/Thomas-Tsai/partclone/blob/f5dbe57ecfffdb8b6de115960183d69eb68e70e6/src/hfsplusclone.c#L141-L144
I'd say that it falls into "those places" that are full of zeroes?
In either way, I don't think that the
%c
is a "safe" way to dump what the software is seeing.... also, I have successfully used the drive to re-install a Sierra. So it must be a valid device/partition.
My end purpose is to store that Bootable USB somewhere, so "if it ever becomes a problem", I won't need to spend yet-another-week of hacking around, trying to re-create it. I hope the command above is correct.
Otherwise, please suggest what I should do 🙏