[Bug] Version 2302 system image format changed to vhdx

Doddddd commented 1 year ago

Steps to reproduce/复现步骤

build.sh --arch arm64 --release-type WIF --magisk-ver stable --gapps-brand none --gapps-variant pico --remove-amazon --root-sol magisk --compress --compress-format 7z

Expected behaviour/预期行为

成功构建

Actual behaviour/实际行为

Expand images e2fsck: No such file or directory while trying to open /tmp/wsa-build-R1rBgkPfon_/wsa/arm64/system_ext.img Possibly non-existent device? Build: an error has occurred, exit

Cleanup Work Directory Cleanup Download Directory

MagiskOnWSALocal commit full SHA/MagiskOnWSALocal 提交的完整哈希

a4957ff

Linux distribution info/Linux 发行版信息

Ubuntu 20.04

Windows version/Windows 版本

10.0.22621.1413

Build Parameters/构建参数

build.sh --arch arm64 --release-type WIF --magisk-ver stable --gapps-brand none --gapps-variant pico --remove-amazon --root-sol magisk --compress --compress-format 7z INFO: Architecture: arm64 INFO: Release Type: WIF INFO: Magisk Version: stable INFO: GApps Brand: none INFO: GApps Variant: pico INFO: Root Solution: magisk INFO: Compress Format: 7z Build: RELEASE_TYPE=Insider Fast

Version requirement/版本要求

[x] I am using latest commit of MagiskOnWSALocal/我正在使用最新 commit 的版本

Logs/日志

build.sh --arch arm64 --release-type WIF --magisk-ver stable --gapps-brand none --gapps-variant pico --remove-amazon --root-sol magisk --compress --compress-format 7z INFO: Architecture: arm64 INFO: Release Type: WIF INFO: Magisk Version: stable INFO: GApps Brand: none INFO: GApps Variant: pico INFO: Root Solution: magisk INFO: Compress Format: 7z Build: RELEASE_TYPE=Insider Fast Generate Download Links Generating WSA download link: arch=arm64 release_type=Insider Fast WSA Version=2302.40000.6.0

download link: http://tlu.dl.delivery.mp.microsoft.com/filestreamingservice/files/94feb808-1ba9-4821-a225-e90ca3b85a2c?P1=1678929536&P2=404&P3=2&P4=Yns4zMKGLks4oj%2bvCPSkKrpuIm3G68hR18%2fRaSVkI1UKOFkrcT5PboJB7e6gZkqi3z6j2ATg9avS0PuMZDp6Pg%3d%3d to /home/runner/MagiskOnWSALocal/download/wsa-WIF.zip download link: http://tlu.dl.delivery.mp.microsoft.com/filestreamingservice/files/924352ce-336b-4716-9379-1bbd98efd7b0?P1=1678902236&P2=404&P3=2&P4=nHVdbWlx%2bBvrMQCdJyU0UDrcRDAMA5444uYvtQJWTxYxOWIArgMDNld9jrEHxY9PDTWuOBIdmF7pGOZLbgOBEg%3d%3d to /home/runner/MagiskOnWSALocal/download/Microsoft.UI.Xaml_arm64.appx Generating Magisk download link: release type=stable download link: https://cdn.jsdelivr.net/gh/topjohnwu/magisk-files@25.2/app-release.apk Download Artifacts

03/16 01:32:16 [NOTICE] Downloading 4 item(s)

03/16 01:32:16 [NOTICE] GID#07dafebe2e2e3302 - Download has already completed: /home/runner/MagiskOnWSALocal/download/Microsoft.UI.Xaml_arm64.appx

03/16 01:32:16 [NOTICE] Download complete: /home/runner/MagiskOnWSALocal/download/Microsoft.UI.Xaml_arm64.appx

03/16 01:32:16 [NOTICE] GID#08ce961e6209bbfd - Download has already completed: /home/runner/MagiskOnWSALocal/download/magisk-stable.zip

03/16 01:32:16 [NOTICE] Download complete: /home/runner/MagiskOnWSALocal/download/magisk-stable.zip

03/16 01:32:17 [NOTICE] CUID#7 - Redirecting to https://download.microsoft.com/download/4/7/c/47c6134b-d61f-4024-83bd-b9c9ea951c25/appx/Microsoft.VCLibs.arm64.14.00.Desktop.appx [DL:337MiB][#036fd7 0B/0B][#d6a10d 138MiB/1.3GiB(10%)]

03/16 01:32:17 [NOTICE] GID#036fd70957335211 - Download has already completed: /home/runner/MagiskOnWSALocal/download/Microsoft.VCLibs.arm64.14.00.Desktop.appx

03/16 01:32:17 [NOTICE] Download complete: /home/runner/MagiskOnWSALocal/download/Microsoft.VCLibs.arm64.14.00.Desktop.appx [#d6a10d 527MiB/1.3GiB(38%) CN:16 DL:376MiB ETA:2s] [#d6a10d 880MiB/1.3GiB(64%) CN:16 DL:368MiB ETA:1s] [#d6a10d 1.0GiB/1.3GiB(78%) CN:14 DL:318MiB] [#d6a10d 1.1GiB/1.3GiB(88%) CN:12 DL:276MiB] [#d6a10d 1.2GiB/1.3GiB(94%) CN:8 DL:230MiB] [#d6a10d 1.3GiB/1.3GiB(98%) CN:2 DL:204MiB] [#d6a10d 1.3GiB/1.3GiB(99%) CN:1 DL:176MiB]

03/16 01:32:27 [NOTICE] Download complete: /home/runner/MagiskOnWSALocal/download/wsa-WIF.zip

Download Results: gid |stat|avg speed |path/URI ======+====+===========+======================================================= 07dafe|OK | 0B/s|/home/runner/MagiskOnWSALocal/download/Microsoft.UI.Xaml_arm64.appx 08ce96|OK | 0B/s|/home/runner/MagiskOnWSALocal/download/magisk-stable.zip 036fd7|OK | 0B/s|/home/runner/MagiskOnWSALocal/download/Microsoft.VCLibs.arm64.14.00.Desktop.appx d6a10d|OK | 159MiB/s|/home/runner/MagiskOnWSALocal/download/wsa-WIF.zip

Status Legend: (OK):download completed. Extract WSA unzipping to /tmp/wsa-build-R1rBgkPfon/wsa unzipping from /tmp/wsa-build-R1rBgkPfon/wsa/WsaPackage_2302.40000.6.0_ARM64_Release-Nightly.msix Extract done

Extract Magisk Magisk version: 25.2 (25200) done

Expand images e2fsck: No such file or directory while trying to open /tmp/wsa-build-R1rBgkPfon_/wsa/arm64/system_ext.img Possibly non-existent device? Build: an error has occurred, exit

Cleanup Work Directory Cleanup Download Directory

Howard20181 commented 1 year ago

All image formats from version 2302 have changed from img to vhdx Need to rewrite the processing

发件人: Doddddd @.> 发送时间: 2023年3月16日 3:04 收件人: LSPosed/MagiskOnWSALocal @.> 抄送: Subscribed @.***> 主题: [LSPosed/MagiskOnWSALocal] [Bug] 2302.40000.6.0 Expand images 报错 (Issue #414)

Steps to reproduce/复现步骤

build.sh --arch arm64 --release-type WIF --magisk-ver stable --gapps-brand none --gapps-variant pico --remove-amazon --root-sol magisk --compress --compress-format 7z

Expected behaviour/预期行为

成功构建

Actual behaviour/实际行为

Expand images e2fsck: No such file or directory while trying to open /tmp/wsa-build-R1rBgkPfon_/wsa/arm64/system_ext.img Possibly non-existent device? Build: an error has occurred, exit

Cleanup Work Directory Cleanup Download Directory

MagiskOnWSALocal commit full SHA/MagiskOnWSALocal 提交的完整哈希

a4957ffhttps://github.com/LSPosed/MagiskOnWSALocal/commit/a4957ff798156004e71335339485911156bb69ac

Linux distribution info/Linux 发行版信息

Ubuntu 20.04

Windows version/Windows 版本

10.0.22621.1413

Build Parameters/构建参数

build.sh --arch arm64 --release-type WIF --magisk-ver stable --gapps-brand none --gapps-variant pico --remove-amazon --root-sol magisk --compress --compress-format 7z INFO: Architecture: arm64 INFO: Release Type: WIF INFO: Magisk Version: stable INFO: GApps Brand: none INFO: GApps Variant: pico INFO: Root Solution: magisk INFO: Compress Format: 7z Build: RELEASE_TYPE=Insider Fast

Version requirement/版本要求

I am using latest commit of MagiskOnWSALocal/我正在使用最新 commit 的版本

Logs/日志

build.sh --arch arm64 --release-type WIF --magisk-ver stable --gapps-brand none --gapps-variant pico --remove-amazon --root-sol magisk --compress --compress-format 7z INFO: Architecture: arm64 INFO: Release Type: WIF INFO: Magisk Version: stable INFO: GApps Brand: none INFO: GApps Variant: pico INFO: Root Solution: magisk INFO: Compress Format: 7z Build: RELEASE_TYPE=Insider Fast Generate Download Links Generating WSA download link: arch=arm64 release_type=Insider Fast WSA Version=2302.40000.6.0

download link: http://tlu.dl.delivery.mp.microsoft.com/filestreamingservice/files/94feb808-1ba9-4821-a225-e90ca3b85a2c?P1=1678929536&P2=404&P3=2&P4=Yns4zMKGLks4oj%2bvCPSkKrpuIm3G68hR18%2fRaSVkI1UKOFkrcT5PboJB7e6gZkqi3z6j2ATg9avS0PuMZDp6Pg%3d%3d to /home/runner/MagiskOnWSALocal/download/wsa-WIF.zip download link: http://tlu.dl.delivery.mp.microsoft.com/filestreamingservice/files/924352ce-336b-4716-9379-1bbd98efd7b0?P1=1678902236&P2=404&P3=2&P4=nHVdbWlx%2bBvrMQCdJyU0UDrcRDAMA5444uYvtQJWTxYxOWIArgMDNld9jrEHxY9PDTWuOBIdmF7pGOZLbgOBEg%3d%3d to /home/runner/MagiskOnWSALocal/download/Microsoft.UI.Xaml_arm64.appx Generating Magisk download link: release type=stable download link: @.***/app-release.apk Download Artifacts

03/16 01:32:16 [NOTICE] Downloading 4 item(s)

03/16 01:32:16 [NOTICE] GID#07dafebe2e2e3302 - Download has already completed: /home/runner/MagiskOnWSALocal/download/Microsoft.UI.Xaml_arm64.appx

03/16 01:32:16 [NOTICE] Download complete: /home/runner/MagiskOnWSALocal/download/Microsoft.UI.Xaml_arm64.appx

03/16 01:32:16 [NOTICE] GID#08ce961e6209bbfd - Download has already completed: /home/runner/MagiskOnWSALocal/download/magisk-stable.zip

03/16 01:32:16 [NOTICE] Download complete: /home/runner/MagiskOnWSALocal/download/magisk-stable.zip

03/16 01:32:17 [NOTICE] CUID#7 - Redirecting to https://download.microsoft.com/download/4/7/c/47c6134b-d61f-4024-83bd-b9c9ea951c25/appx/Microsoft.VCLibs.arm64.14.00.Desktop.appx [DL:337MiB][#036fd7 0B/0B][#d6a10d 138MiB/1.3GiB(10%)]

03/16 01:32:17 [NOTICE] GID#036fd70957335211 - Download has already completed: /home/runner/MagiskOnWSALocal/download/Microsoft.VCLibs.arm64.14.00.Desktop.appx

03/16 01:32:17 [NOTICE] Download complete: /home/runner/MagiskOnWSALocal/download/Microsoft.VCLibs.arm64.14.00.Desktop.appx [#d6a10d 527MiB/1.3GiB(38%) CN:16 DL:376MiB ETA:2s] [#d6a10d 880MiB/1.3GiB(64%) CN:16 DL:368MiB ETA:1s] [#d6a10d 1.0GiB/1.3GiB(78%) CN:14 DL:318MiB] [#d6a10d 1.1GiB/1.3GiB(88%) CN:12 DL:276MiB] [#d6a10d 1.2GiB/1.3GiB(94%) CN:8 DL:230MiB] [#d6a10d 1.3GiB/1.3GiB(98%) CN:2 DL:204MiB] [#d6a10d 1.3GiB/1.3GiB(99%) CN:1 DL:176MiB]

03/16 01:32:27 [NOTICE] Download complete: /home/runner/MagiskOnWSALocal/download/wsa-WIF.zip

Download Results: gid |stat|avg speed |path/URI ======+====+===========+======================================================= 07dafe|OK | 0B/s|/home/runner/MagiskOnWSALocal/download/Microsoft.UI.Xaml_arm64.appx 08ce96|OK | 0B/s|/home/runner/MagiskOnWSALocal/download/magisk-stable.zip 036fd7|OK | 0B/s|/home/runner/MagiskOnWSALocal/download/Microsoft.VCLibs.arm64.14.00.Desktop.appx d6a10d|OK | 159MiB/s|/home/runner/MagiskOnWSALocal/download/wsa-WIF.zip

Status Legend: (OK):download completed. Extract WSA unzipping to /tmp/wsa-build-R1rBgkPfon/wsa unzipping from /tmp/wsa-build-R1rBgkPfon/wsa/WsaPackage_2302.40000.6.0_ARM64_Release-Nightly.msix Extract done

Extract Magisk Magisk version: 25.2 (25200) done

Expand images e2fsck: No such file or directory while trying to open /tmp/wsa-build-R1rBgkPfon_/wsa/arm64/system_ext.img Possibly non-existent device? Build: an error has occurred, exit

Cleanup Work Directory Cleanup Download Directory

― Reply to this email directly, view it on GitHubhttps://github.com/LSPosed/MagiskOnWSALocal/issues/414, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJRNWK4SA2TLH6EACIBIATTW4IHD3ANCNFSM6AAAAAAV4IEMZI. You are receiving this because you are subscribed to this thread.Message ID: @.***>

YT-Advanced commented 1 year ago

Maybe use qemu-nbd can help you to mount and unmount VHDX (as read-write), @Howard20181 Example code: https://gist.github.com/allenyllee/0a4c02952bf695470860b27369bbb60d

Howard20181 commented 1 year ago

Expand images

We need to expand images before we can copy files to it. No such tool is available for Linux.

YT-Advanced commented 1 year ago

We need to expand images before we can copy files to it. No such tool is available for Linux.

So you will change to windows instead ?

Howard20181 commented 1 year ago

Yes

YT-Advanced commented 1 year ago

Yes

Did you try this method: First convert .vhdx to .img, then resize and edit, copy file in image. Finnally convert img again to vhdx

Howard20181 commented 1 year ago

Did you try this method: First convert .vhdx to .img, then resize and edit, copy file in image. Finnally convert img again to vhdx

This requires taking out the partition table inside the VHDX and extracting the EXT4 partition separately, which requires additional operations. I have given up on the vhdx-img branch, you can try to finish it.

YT-Advanced commented 1 year ago

Oh, I will try and will give feedback soon

OlyAndy commented 1 year ago

Would something like:

qemu-img convert -O raw /"$WORK_DIR"/wsa/"$ARCH"/system_ext.vhdx "$WORK_DIR"/wsa/"$ARCH"/system_ext.img qemu-img convert -O raw /"$WORK_DIR"/wsa/"$ARCH"/product.vhdx "$WORK_DIR"/wsa/"$ARCH"/product.img qemu-img convert -O raw /"$WORK_DIR"/wsa/"$ARCH"/system.vhdx "$WORK_DIR"/wsa/"$ARCH"/system.img qemu-img convert -O raw /"$WORK_DIR"/wsa/"$ARCH"/vendor.vhdx "$WORK_DIR"/wsa/"$ARCH"/vendor.img

And the opposite way after the img manipulation work? (qemu-utils is required)

YT-Advanced commented 1 year ago

Would something like:

qemu-img convert -O raw /"$WORK_DIR"/wsa/"$ARCH"/system_ext.vhdx "$WORK_DIR"/wsa/"$ARCH"/system_ext.img qemu-img convert -O raw /"$WORK_DIR"/wsa/"$ARCH"/product.vhdx "$WORK_DIR"/wsa/"$ARCH"/product.img qemu-img convert -O raw /"$WORK_DIR"/wsa/"$ARCH"/system.vhdx "$WORK_DIR"/wsa/"$ARCH"/system.img qemu-img convert -O raw /"$WORK_DIR"/wsa/"$ARCH"/vendor.vhdx "$WORK_DIR"/wsa/"$ARCH"/vendor.img

And the opposite way after the img manipulation work? (qemu-utils is required)

Convert from img to vhdx work (because some site used it like DietPy) but from vhdx to img maybe not work (I am not tested it)

OlyAndy commented 1 year ago

Actually, it doesn't work... But going via NBD and qemu-nbd might be the right answer, but not sure how to do the image resize.

https://gist.github.com/allenyllee/0a4c02952bf695470860b27369bbb60d

YT-Advanced commented 1 year ago

Actually, it doesn't work... But going via NBD and qemu-nbd might be the right answer, but not sure how to do the image resize.

https://gist.github.com/allenyllee/0a4c02952bf695470860b27369bbb60d

I found one that maybe support write, read, copy on VHDX file from a repos of Microsoft (it look more like just check instead of write). Maybe you can check it. Link: https://github.com/microsoft/lis-test/blob/master/WS2012R2/lisa/remote-scripts/ica/STOR_VHDXResize_ReadWrite.sh

yujincheng08 commented 1 year ago

https://learn.microsoft.com/en-us/azure/virtual-machines/linux/create-upload-generic#resizing-vhds

Well, Microsoft officially recommends qemu-img to size vhd(x) by converting between vhd(x) and raw image. The concern is that it may take a longer time and consume more disk space.

YT-Advanced commented 1 year ago

https://learn.microsoft.com/en-us/azure/virtual-machines/linux/create-upload-generic#resizing-vhds

Well, Microsoft officially recommends to size vhd(x) by converting between vhd(x) and raw image. The concern is that it may take a longer time and consume more disk space.qemu-img

So did you have any idea for the vhdx. I think we should build new script in Powershell instead (it's possible to do)

yujincheng08 commented 1 year ago

@YT-Advanced No. We will keep building in Linux. And we have found a way to do so.

MustardChef commented 1 year ago

@YT-Advanced No. We will keep building in Linux. And we have found a way to do so.

Great to hear!

Thank you all for the hard work that has been put in to the project!

YT-Advanced commented 1 year ago

@YT-Advanced No. We will keep building in Linux. And we have found a way to do so.

Wow, that so very greate. And if you need, I can help you :)))

LSPosed / MagiskOnWSALocal