jpz4085 / BCD-SYS

Setup the Windows BCD and boot files from Linux or macOS.
GNU General Public License v3.0
9 stars 1 forks source link

Windows Install.app #5

Closed Sergey-Galan closed 3 months ago

Sergey-Galan commented 5 months ago

Hello. I wrote to you there https://sourceforge.net/p/windows-install/discussion/general/thread/2fd478e439/?limit=25#ed25/cbdd/c114/1a05/5e38/eb08/67b0 But I think it’s convenient to communicate here. I adapted the creation of BCD UEFI for the program and it works well. But there are difficulties with BCD BIOS. This is an MBR disk

 iMac:~ sergey$ diskutil info /Volumes/Windows
 Device Identifier:         disk6s1
 Device Node:               /dev/disk6s1
 Whole:                     No
 Part of Whole:             disk6

 Volume Name:               Windows
 Mounted:                   Yes
 Mount Point:               /Volumes/Windows

 Partition Type:            Windows_NTFS
 File System Personality:   ExFAT
 Type (Bundle):             exfat
 Name (User Visible):       ExFAT

 OS Can Be Installed:       No
 Media Type:                Общий
 Protocol:                  USB
 SMART Status:              Not Supported
 Volume UUID:               7F7C6600-E4EC-32D9-9064-A2DBA7EFCD2A
 Partition Offset:          1048576 Bytes (2048 512-Byte-Device-Blocks)

 Disk Size:                 320.1 GB (320071532544 Bytes) (exactly 625139712 512-Byte-Units)
 Device Block Size:         512 Bytes

 Volume Total Space:        320.1 GB (320060129280 Bytes) (exactly 625117440 512-Byte-Units)
 Volume Used Space:         11.3 MB (11272192 Bytes) (exactly 22016 512-Byte-Units) (0.0%)
 Volume Free Space:         320.0 GB (320048857088 Bytes) (exactly 625095424 512-Byte-Units) (100.0%)
 Allocation Block Size:     512 Bytes

 Media OS Use Only:         No
 Media Read-Only:           No
 Volume Read-Only:          No

 Device Location:           External
 Removable Media:           Fixed

 Solid State:               Info not available

I get:

 iMac:~ sergey$ /Users/sergey/Desktop/test15 ; exit;
 part - /dev/disk6s1
 disk - /dev/disk6
 FDisk_partition_scheme
 Password:
 Disk: /dev/disk6   geometry: 38913/255/63 [625142448 sectors]
 Signature: 0xAA55
          Starting       Ending
  #: id  cyl  hd sec -  cyl  hd sec [     start -       size]
 ------------------------------------------------------------------------
  1: 07 1023 254  63 - 1023 254  63 [      2048 -  625139712] HPFS/QNX/AUX
  2: 00    0   0   0 -    0   0   0 [         0 -          0] unused      
  3: 00    0   0   0 -    0   0   0 [         0 -          0] unused      
  4: 00    0   0   0 -    0   0   0 [         0 -          0] unused      
 FDisk_partition_scheme
 disksig - 00000000
 partoffset - 1048576
 diskbytes - 00,00,00,00,00,00,00,00,00,00,00,00,0
 partbytes - 00,00,10,00,00,00,00,00,00,00,00,00,00,00,00,00
 logout

 Saving session...
 ...copying shared history...
 ...saving history...truncating history files...
 ...completed.

GPT disk with hybrid MBR

 iMac:~ sergey$ diskutil info /Volumes/Windows
 Device Identifier:         disk6s3
 Device Node:               /dev/disk6s3
 Whole:                     No
 Part of Whole:             disk6

 Volume Name:               Windows
 Mounted:                   Yes
 Mount Point:               /Volumes/Windows

 Partition Type:            Microsoft Basic Data
 File System Personality:   NTFS
 Type (Bundle):             ntfs
 Name (User Visible):       Windows NT File System (NTFS)

 OS Can Be Installed:       No
 Media Type:                Общий
 Protocol:                  USB
 SMART Status:              Not Supported
 Disk / Partition UUID:     0E7E4ACF-CE00-4595-B642-D972CDFEAB57
 Partition Offset:          139829706752 Bytes (273104896 512-Byte-Device-Blocks)

 Disk Size:                 100.1 GB (100092870656 Bytes) (exactly 195493888 512-Byte-Units)
 Device Block Size:         512 Bytes

 Volume Total Space:        100.1 GB (100092866560 Bytes) (exactly 195493880 512-Byte-Units)
 Volume Used Space:         11.1 GB (11061256192 Bytes) (exactly 21604016 512-Byte-Units) (11.1%)
 Volume Free Space:         89.0 GB (89031610368 Bytes) (exactly 173889864 512-Byte-Units) (88.9%)
 Allocation Block Size:     512 Bytes

 Media OS Use Only:         No
 Media Read-Only:           No
 Volume Read-Only:          Yes (read-only mount flag set)

 Device Location:           External
 Removable Media:           Fixed

 Solid State:               Info not available

I get:

 iMac:~ sergey$ /Users/sergey/Desktop/test15 ; exit;
 part - /dev/disk6s3
 disk - /dev/disk6
 GUID_partition_scheme
 Password:
 Disk: /dev/disk6   geometry: 29185/255/63 [468862128 sectors]
  Signature: 0xAA55
          Starting       Ending
  #: id  cyl  hd sec -  cyl  hd sec [     start -       size]
 ------------------------------------------------------------------------
  1: EE    0   0   2 -   25 127  14 [         1 -     409639] <Unknown ID>
  2: AF   25 127  15 - 1023 254  63 [    409640 -  272431832] HFS+        
 *3: 07 1023 254  63 - 1023 254  63 [ 273104896 -  195493888] HPFS/QNX/AUX
  4: 08 1023 254  63 - 1023 254  63 [ 272841472 -     263424] AIX fs      
 FDisk_partition_scheme
 disksig - 00000000
 partoffset - 139829706752
 diskbytes - 00,00,00,00,00,00,00,00,00,00,00,00,0
 partbytes - 00,00,80,8e,20,00,00,00,00,00,00,00,00,00,00,00
 logout

 Saving session...
 ...copying shared history...
 ...saving history...truncating history files...
 ...completed.

Script-Test for testing

#!/bin/bash

endian () {
 v=$1
 i=${#v}

 while [ $i -gt 0 ]
 do
     i=$[$i-2]
    echo -n ${v:$i:2}
 done
 echo
}

guid_bytes () {
guidstr="$1"
IFS='-'

read -a strarr <<< "$guidstr"

guidbytes=$(endian ${strarr[0]})
guidbytes+=$(endian ${strarr[1]})
guidbytes+=$(endian ${strarr[2]})
guidbytes+=${strarr[3]}
guidbytes+=${strarr[4]}

echo $guidbytes
}

blksize () {
sectsz=$(diskutil info $1 | grep "Device Block Size:" | awk '{print $4}')
echo $sectsz
}

mntpoint=/Volumes/Windows
part=$(diskutil info "$(basename "$mntpoint")" | grep "Device Node:" | awk '{print $3}')
echo part - $part
disk=$(printf $part | sed 's/s[0-9]*$//')
echo "disk - $disk"
scheme=$(diskutil info $disk | grep "Content (IOContent):" | awk '{print $3}')
echo $scheme

if [[ "$scheme" == "GUID_partition_scheme" ]]; then
    if [[ ! -z $(sudo fdisk "$disk" | grep -E '2:|3:|4:' | grep -v unused) ]]; then
       scheme="FDisk_partition_scheme" #Use fdisk option for GPT disk with hybrid MBR.
    fi
fi
sudo fdisk -y "$disk" <<EOF
   p
   quit
EOF
echo $scheme  

if   [[ "$scheme" == "GUID_partition_scheme" ]]; then
     sectsize=$(blksize $disk)
     echo "sectsize- $sectsize"
     diskbytes=$(sudo xxd -u -p -s $(($sectsize + 56)) -l 16 $disk | sed 's/.\{2\}/&,/g;s/,$//')
     partguid=$(diskutil info $part | grep "Partition UUID:" | awk '{print $5}')
     echo "partguid - $partguid"
     partbytes=$(printf "%s" $(guid_bytes $partguid) | sed 's/.\{2\}/&,/g;s/,$//')
     schemebyte="00"
elif [[ "$scheme" == "FDisk_partition_scheme" ]]; then
     disksig=$(sudo xxd -u -p -s 440 -l 4 $disk)
     partoffset=$(diskutil info $part | grep "Partition Offset:" | awk '{print $3}')
     echo "disksig - $disksig"
     echo "partoffset - $partoffset"
     if [[ -z $partoffset ]]; then
        hidsect=$(sudo fdisk $disk | tail -n +6 | awk -v partnum=${part##*s} 'FNR == partnum {print $11}')
        sectsize=$(blksize $disk)
        partoffset=$(($sectsize * $hidsect))
     fi
     diskbytes=$(printf "%s" $(printf "%x%024x" "0x$disksig") | sed 's/.\{2\}/&,/g;s/,$//')
     partbytes=$(printf "%s" $(endian $(printf "%032x" "$partoffset")) | sed 's/.\{2\}/&,/g;s/,$//')
     schemebyte="01"
fi
echo "diskbytes - $diskbytes"
echo "partbytes - $partbytes"

exit
jpz4085 commented 5 months ago

It appears you need to write a Windows Disk Signature to the MBR. You can do this with the dd command or signmbr script as show below. Assuming disk6 from your examples.

sudo dd if=/dev/random of=/dev/disk6 bs=1 count=4 seek=440 sudo signmbr /dev/disk6

The output above shows "disksig" is all zeros and "diskbytes" is too short because of the missing four byte value. It should appear as shown below.

MBR disk:

part - /dev/disk0s1
disk - /dev/disk0
FDisk_partition_scheme
Disk: /dev/disk0    geometry: 6527/255/63 [104857600 sectors]
Signature: 0xAA55
         Starting       Ending
 #: id  cyl  hd sec -  cyl  hd sec [     start -       size]
------------------------------------------------------------------------
 1: 07 1023 254  63 - 1023 254  63 [      2048 -  104855552] HPFS/QNX/AUX
 2: 00    0   0   0 -    0   0   0 [         0 -          0] unused      
 3: 00    0   0   0 -    0   0   0 [         0 -          0] unused      
 4: 00    0   0   0 -    0   0   0 [         0 -          0] unused      
FDisk_partition_scheme
disksig - D0C90E40
partoffset - 1048576
diskbytes - d0,c9,0e,40,00,00,00,00,00,00,00,00,00,00,00,00
partbytes - 00,00,10,00,00,00,00,00,00,00,00,00,00,00,00,00

Hybrid MBR:

part - /dev/disk0s3
disk - /dev/disk0
GUID_partition_scheme
Disk: /dev/disk0    geometry: 6527/255/63 [104857600 sectors]
Signature: 0xAA55
         Starting       Ending
 #: id  cyl  hd sec -  cyl  hd sec [     start -       size]
------------------------------------------------------------------------
 1: EE    0   0   2 -   29  49  20 [         1 -     468991] <Unknown ID>
*2: 07   29  49  21 - 1023 254  63 [    468992 -  102543360] HPFS/QNX/AUX
 3: 27 1023 254  63 - 1023 254  63 [ 103012352 -    1843200] <Unknown ID>
 4: 00    0   0   0 -    0   0   0 [         0 -          0] unused      
FDisk_partition_scheme
disksig - 77AC48CC
partoffset - 240123904
diskbytes - 77,ac,48,cc,00,00,00,00,00,00,00,00,00,00,00,00
partbytes - 00,00,50,0e,00,00,00,00,00,00,00,00,00,00,00,00

Everything else looks good.

Sergey-Galan commented 5 months ago

Thank you. The command worked:

 disk=/dev/disk6
 sudo dd if=/dev/random of="$disk" bs=1 count=4 seek=440

But to do this you need to disconnect all disk partitions:

 diskutil unmountDisk "$disk"

I assume that if one of the disk partitions is system, then the resource will be busy and no recording will occur.

 iMac:~ sergey$ /Users/sergey/Desktop/test15 ; exit;
part - /dev/disk6s3
disk - /dev/disk6
GUID_partition_scheme
Unmount of all volumes on disk6 was successful
Password:
4+0 records in
4+0 records out
4 bytes transferred in 0.008483 secs (472 bytes/sec)
Disk: /dev/disk6    geometry: 29185/255/63 [468862128 sectors]
Signature: 0xAA55
         Starting       Ending
 #: id  cyl  hd sec -  cyl  hd sec [     start -       size]
------------------------------------------------------------------------
 1: EE    0   0   2 -   25 127  14 [         1 -     409639] <Unknown ID>
 2: AF   25 127  15 - 1023 254  63 [    409640 -  272431832] HFS+        
*3: 07 1023 254  63 - 1023 254  63 [ 273104896 -  195493888] HPFS/QNX/AUX
 4: 08 1023 254  63 - 1023 254  63 [ 272841472 -     263424] AIX fs      
FDisk_partition_scheme
disksig - EEAF7745
partoffset - 139829706752
diskbytes - ee,af,77,45,00,00,00,00,00,00,00,00,00,00,00,00
partbytes - 00,00,80,8e,20,00,00,00,00,00,00,00,00,00,00,00
logout

Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
Sergey-Galan commented 5 months ago

I found how to determine install language, it seems to work. I don't know, maybe there is a better way.

 langCod="$(printf "cd ControlSet001\Control\Nls\Language\nlsval InstallLanguage\nunload\n" | ./tools/bin/hivexsh "$MountPoint/Windows/System32/config/SYSTEM")"
 locale="$(printf "cd Microsoft\Input\Locales\loc_"$langCod"\nlsval Fluency Model Directory\nunload\n" | ./tools/bin/hivexsh "$MountPoint/Windows/System32/config/SOFTWARE")"
jpz4085 commented 5 months ago

I assume that if one of the disk partitions is system, then the resource will be busy and no recording will occur.

Probably would have to use a live Linux USB in that situation.

I found how to determine install language, it seems to work. I don't know, maybe there is a better way.

Looks good. That's essentially the same way I read in the Windows product name.

Sergey-Galan commented 5 months ago

Looks good. That's essentially the same way I read in the Windows product name.

Yes, that was an example.

Sergey-Galan commented 5 months ago

In the script, I can make the disk hybrid in this way. x=disk y=partition

sudo ./Tools/bin/gdisk /dev/disk"$x"  <<EOF
r
p
h
$y
y
07
y
n
o
w
y
quit
EOF

But I'm not sure it's reliable. Is there another way to make the disk hybrid?

jpz4085 commented 5 months ago

But I'm not sure it's reliable. Is there another way to make the disk hybrid?

May be better to use sgdisk (if possible) to configure the hybrid MBR as shown below.

y=GPT partition

sudo ./Tools/bin/sgdisk -h $y /dev/disk"$x"

Then set the partition active using fdisk since sgdisk doesn't provide that ability.

y=MBR partition

printf "f $y\nq\n" | sudo fdisk -e /dev/disk"$x"

Sergey-Galan commented 5 months ago

Yes, sgdisk was created for the script))) This is how it works:

printf "f $y\nq\ny\n" | sudo fdisk -e /dev/disk"$x"

Agree at the end.

Now Install Windows (legacy boot) works, but I'm testing on a PC. I'll look for someone to help me check it on an old Mac. Conditions - the system is not higher than MacOS 12 Monterey and the disk should not be a system one (unless it is a hybrid one and Windows has not been installed before) Thank you very much for your help!

Sergey-Galan commented 5 months ago

Oh no, I went back to the gdisk utility, it itself determines the MBR partition number relative to the GPT partition number.

jpz4085 commented 5 months ago

Oh no, I went back to the gdisk utility, it itself determines the MBR partition number relative to the GPT partition number.

Yeah, that's true. You could format the commands using printf and pipe them in to gdisk if you want to have everything on one line.

Alternatively you could get the MBR partition which corresponds to the GPT partition using the start sector as shown below.

pstart=$(diskutil info /dev/disk"$x"s"$y" | grep "Partition Offset:" | awk '{print $5}' | sed 's/(//') pmbr=$(sudo fdisk /dev/disk"$x" | grep -w $pstart | awk '{print $1}' | sed 's/://')

Then use the $pmbr variable in fdisk rather than $y.

printf "f $pmbr\nq\n" | sudo fdisk -e /dev/disk"$x"

Either way will probably work just as well.

Sergey-Galan commented 5 months ago

BCD for Windows 7 UEFI is not suitable, there is an error on BCD at startup, but after pressing Enter twice it boots the system. With Windows 7 legacy boot it’s even worse; after trying to boot, it reboots the system.

jpz4085 commented 5 months ago

Attach the problematic BCD files as well as a screenshot of the Windows error if possible. If this issue occurred with the BCDs created by my scripts provide the verbose output of bcd-sys too. If the issue occurred with BCDs created by the script(s) you are working on provide a copy of the script(s) and any screen output that would be relevant.

Sergey-Galan commented 5 months ago

This part is from the main script.

build_BCD()
{
echo "Build the main BCD store..."
echo "$tf3" | sudo -S ./scripts/winload "$1" "$2" "$3" > "$HOME"/winload.txt
cp -v ./Templates/BCD-NEW "$HOME"/BCD-Windows
sudo -S ./tools/bin/reged -I -C "$HOME"/BCD-Windows BCD00000001 ./Templates/winload.reg
./tools/bin/hivexsh -d -w -f "$HOME"/winload.txt "$HOME"/BCD-Windows
if [[ "$2" == "uefi" ]]; then
    echo "Build the recovery BCD store..."
    sudo -S ./scripts/recovery "$3" > "$HOME"/recovery.txt
    cp ./Templates/BCD-NEW "$HOME"/BCD-Recovery
    sudo -S ./tools/bin/reged -I -C "$HOME"/BCD-Recovery BCD00000001 ./Templates/recovery.reg
    ./tools/bin/hivexsh -d -w -f "$HOME"/recovery.txt "$HOME"/BCD-Recovery
    echo "Copy the BCD hives to the ESP folders..."
    head -c 27000 "$HOME"/BCD-Windows > /Volumes/EFI/EFI/Microsoft/Boot/BCD
    head -c 27000 "$HOME"/BCD-Recovery > /Volumes/EFI/EFI/Microsoft/Recovery/BCD
elif [[ "$2" == "bios" ]]; then
    echo "Copy the main BCD hive to the boot folder..."
    head -c 27000 "$HOME"/BCD-Windows > "$HOME"/Boot/BCD
fi
rm "$HOME"/BCD-Windows
rm "$HOME"/winload.txt
if [[ "$2" == "uefi" ]]; then
    rm "$HOME"/recovery.txt
    rm "$HOME"/BCD-Recovery
fi
}

Log, recovery, winload and update_device attached. Resources.zip

It seems that the system does not like the object [BCD00000001\Objects{7ff607e0-4395-11db-b0de-0800200c9a66}] in winload.reg

IMG_0397

jpz4085 commented 5 months ago

It seems that the system does not like the object [BCD00000001\Objects{7ff607e0-4395-11db-b0de-0800200c9a66}] in winload.reg

Apparently the Hypervisor Settings Group is corrupted for some reason.

This part is from the main script.

It appears you are using the head command to write the first 27,000 bytes of the new BCD stores to boot folders.

 echo "Copy the BCD hives to the ESP folders..."
    head -c 27000 "$HOME"/BCD-Windows > /Volumes/EFI/EFI/Microsoft/Boot/BCD
    head -c 27000 "$HOME"/BCD-Recovery > /Volumes/EFI/EFI/Microsoft/Recovery/BCD
elif [[ "$2" == "bios" ]]; then
    echo "Copy the main BCD hive to the boot folder..."
    head -c 27000 "$HOME"/BCD-Windows > "$HOME"/Boot/BCD

A typical main BCD file for a single Windows installation is closer to 28KBs so copying it this way may lose the last part of the file. Try using cp to copy the BCD stores to the destination and see if that resolves the issue.

Sergey-Galan commented 5 months ago

I'll try with your recommendations, but there seems to be enough space. BCD.zip

Sergey-Galan commented 5 months ago

The error remains BCD.zip

Sergey-Galan commented 5 months ago

I specifically installed Windows 7 using a flash drive and compared the BCD. Found an error:

bcd

But in winload.reg Element is present. It is at the very bottom, I can assume that it is being bitten off. (((

Project BCD-No problem.zip BCD-Problem.zip

Sergey-Galan commented 5 months ago

Ok. Several times during installation the element was registered, and there were no problems with the BCD. Looks like this is a random problem. Maybe add an empty line at the end of winload.reg?

jpz4085 commented 5 months ago

Ok. Several times during installation the element was registered, and there were no problems with the BCD. Looks like this is a random problem. Maybe add an empty line at the end of winload.reg?

An extra line doesn't seem to make a difference when applying the winload.reg and recovery.reg files. The 250000f5 key is present either way at least when doing this manually. Maybe it's worth making the script sleep for a few seconds before running reged in both situations in case there's any previous steps that haven't completed yet. Can't say more unfortunately about a random problem.

Sergey-Galan commented 5 months ago

Ok, I will prescribe a sleep for 2 seconds and make 4-5 system installations.

Sergey-Galan commented 5 months ago

Windows installs on a GPT disk with a hybrid MBR. I'm trying to install Windows on MBR and getting an error.

IMG_0424

To install on MBR I disconnected - sgdisk -h $y /dev/disk"$x" Further, there were no problems with the installation. For boot records I use ms-sys utility

diskutil unmountDisk /dev/disk"$x"
sudo ./tools/bin/ms-sys -7 /dev/disk"$x"
diskutil unmount disk"$x"s"$y"
sudo ./tools/bin/ms-sys -n /dev/disk"$x"s"$y"

With hybrid and MBR I get the following entry:

Unmount of all volumes on disk10 was successful
Windows 7 master boot record successfully written to /dev/disk10
disk10s1 was already unmounted
NTFS Windows 7 boot record successfully written to /dev/disk10s1

The utility is working out, but why such an error with MBR?

jpz4085 commented 5 months ago

The utility is working out, but why such an error with MBR?

It appears the NTFS boot record doesn't like something about the partition. Make sure the start sector, heads, sectors per track and drive ID have been written to the partition. You might want to use the partition information feature of ms-sys:

sudo ./tools/bin/ms-sys -p -n /dev/disk"$x""s"$y"

That's as much as I can suggest.

Sergey-Galan commented 4 months ago

It's my fault. I forgot that I wrote this only for GPT

partstart=$(sudo gpt show disk$x 2>&1 | grep "$y  GPT part" | awk '{ print $1 }')
sudo ./Tools/bin/mkntfs -p "$partstart" -S 63 -H 255 -f -L "$PartName" /dev/disk"$x"s"$y"

You gave me another option and I used it elsewhere, and it is universal.

pstart=$(diskutil info /dev/disk"$x"s"$y" | grep "Partition Offset:" | awk '{print $5}' | sed 's/(//')
jpz4085 commented 3 months ago

Issues resolved. Closed ticket.