search

okibcn / wslcompact

Compacts the size of the ever-growing WSL vhdx images.
https://github.com/okibcn/wslcompact
GNU General Public License v3.0
800 stars 19 forks source link

Unspecified error #2

Closed roslov closed 1 year ago

roslov commented 1 year ago

It has been working for a loooooot of time. And finished this way:

PS C:\Users\tr> wslcompact
 WSL compact, v2.2023.01.29
 (C) 2023 Oscar Lopez.
 wslcompact -h for help. For more information visit: https://github.com/okibcn/wslcompact

 Processing: Ubuntu-20.04 distro
 Image file: C:\Users\tr\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc\LocalState\ext4.vhdx
 Image size: 61078 MB
 Estimating target size...54015 В± 1317 MB
 Import in progress, this may take a few minutes.
Unspecified error
Error code: Wsl/Service/E_FAIL
Move-Item : Cannot find path 'C:\Users\tr\AppData\Local\Temp\wslcompact\ext4.vhdx' because it does not exist.
At C:\Users\tr\scoop\apps\wslcompact\current\wslcompact.ps1:70 char:17
+                 Move-Item "$tmp_folder/ext4.vhdx" "$wsl_path" -Force
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (C:\Users\tr\App...mpact\ext4.vhdx:String) [Move-Item], ItemNotFoundExce
   ption
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.MoveItemCommand

 Compacted from 61078 MB to 61078 MB

Processing fstab with mount -a failed.

<3>WSL (8) ERROR: CreateProcessEntryCommon:368: getpwuid(0) failed 2
<3>WSL (8) ERROR: CreateProcessEntryCommon:372: getpwuid(0) failed 2
<3>WSL (8) ERROR: CreateProcessEntryCommon:574: execvpe df failed 2
<3>WSL (8) ERROR: CreateProcessEntryCommon:583: Create process not expected to return
Cannot index into a null array.
At C:\Users\tr\scoop\apps\wslcompact\current\wslcompact.ps1:55 char:9
+         $estimated = ((wsl -d "$wsl_distro" -e df /) | select-string  ...
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : NullArray

 Processing: docker-desktop-data distro
 Image file: C:\Users\tr\AppData\Local\Docker\wsl\data\ext4.vhdx
 Image size: 20451 MB
 Estimating target size...55310906 В± 1349047 MB
 WARNING: there isn't enough free space in temp drive C to process docker-desktop-data.
          There are only 430288 MB available.

 Please, change the TEMP folder to a drive with at least 56659953 MB of free space.
 You cand do it by typing $env:TEMP="Z:/your/new/temp/folder" before using wslcompact.

 Processing: docker-desktop distro
 Image file: C:\Users\tr\AppData\Local\Docker\wsl\distro\ext4.vhdx
 Image size: 125 MB
 Estimating target size...61 В± 1 MB
 Import in progress, this may take a few minutes.
The operation completed successfully.
 Compacted from 125 MB to 118 MB

image

WSL version: 1.1.0.0 Kernel version: 5.15.83.1 WSLg version: 1.0.48 MSRDC version: 1.2.3770 Direct3D version: 1.608.2-61064218 DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp Windows version: 10.0.22623.1245

okibcn commented 1 year ago

I can see two issues there. One is the error when there is a faulty filesystem in the Linux image, the other is just part of the script for tracking errors, this one is my fault. I can see something else. Could you please type this in your windows powershell?

wsl -d docker-desktop-data -e df /

It would help a lot to have that output. It seems that the free space report is not returning coherent results.

In my system, with an SSD it generates about 4GB per minute, so if your target size is 54GB for your Ubuntu, then expect 12-15 mins, or a lot more if you are using a hard drive. I am planning on adding time estimation, at least as a guideline.

For your Ubuntu image, There is a test you can do to see if the problem is in the image or in the hard drive. It requires 7zip installed in your system. The test is just a dump in tar format of all the files in the image. Type:

wsl --shutdown ; cmd /c "wsl  --export Ubuntu-20.04 - | 7z a -si  ""Z:\External\path\dump.tar.7z"" "

It will provide a nicer progress indicator. If that fails, then we will know that the problem is in your current image. If it goes fine, then the problem is somewhere in the drive when creating the optimized VHDX image.

roslov commented 1 year ago 
PS C:\Users\tr> wsl -d docker-desktop-data -e df /
Processing fstab with mount -a failed.

<3>WSL (19) ERROR: CreateProcessEntryCommon:368: getpwuid(0) failed 2
<3>WSL (19) ERROR: CreateProcessEntryCommon:372: getpwuid(0) failed 2
<3>WSL (19) ERROR: CreateProcessEntryCommon:574: execvpe df failed 2
<3>WSL (19) ERROR: CreateProcessEntryCommon:583: Create process not expected to return
okibcn commented 1 year ago

So, the error was the data partition of docker. It is not an OS partition and doesn't have the direct tools to estimate the space. The version v2 released yesterday solves that compatibility. Thanks for your test, that confirmed my suspicions.

I am more interested in your Ubuntu partition. Something must be wrong with that partition and I would like to help you with that. It seems that you can save several gigabytes in that one. If you can dump that partition with the provided instruction, then we could know more about the problem.

BryanDollery commented 1 year ago

Using commit e48534d I also get this error:

PS C:\Users\bryan\wslcompact> .\wslcompact.ps1 -c
 WSL compact, v4.2023.01.31
 (C) 2023 Oscar Lopez
 wslcompact -h for help. For more information visit: https://github.com/okibcn/wslcompact

 Distro's name:  Ubuntu-22.04
 Image file:     C:\Users\bryan\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu22.04LTS_79rhkp1fndgsc\LocalState\ext4.vhdx
 Current size:   42077 MB
 Estimated size: 27421 +/- 669 MB
 The estimated process time using an SSD is about 7 minutes.
 NOTE: You can safely cancel at any time by pressing Ctrl-C
 Import in progress, this may take a few minutes.
Unspecified error
Error code: Wsl/Service/E_FAIL
 WARNING: wslcompact found errors in the current image. It could be a storage problem,
          a corrupted ext4 filesystem, or any other issue. Image not processed.

Reading the thread above, I do use docker, but I do not use docker for windows (I use wsl in systemd mode and have docker locally installed because docker desktop for windows sucks huge donkey balls). It may be worth noting that I also have plenty of free space on my drive.

I'm just trying to execute your zip command above. I'll report back.

okibcn commented 1 year ago

@BryanDollery thanks for your feedback. The error is generated by the WSL.EXE application. I assume you are using the latest WSL, version 1.1.0.0 at this time. In such case, it is a problem with WSL or with your image.

The test command just exports the image in tar format and it is a way to check whether your image is exportable or if it requires a fsck to fix problems somewhere within the ext4 filesystem before it can be processed.

Thanks

BryanDollery commented 1 year ago

@BryanDollery thanks for your feedback. The error is generated by the WSL.EXE application. I assume you are using the latest WSL, version 1.1.0.0 at this time. In such case, it is a problem with WSL or with your image.

The test command just exports the image in tar format and it is a way to check whether your image is exportable or if it requires a fsck to fix problems somewhere within the ext4 filesystem before it can be processed.

Thanks

I just ran wsl --version and got this result:

WSL version: 1.0.3.0 Kernel version: 5.15.79.1 WSLg version: 1.0.47 MSRDC version: 1.2.3575 Direct3D version: 1.606.4 DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp Windows version: 10.0.22621.963

I'm running all updates, but if the latest version is 1.1 I have no idea why I'm on an earlier version.

As for corruption in my image... I created this one about a fortnight ago from stock ubuntu on the play store and all I do with it is basic software development in nodejs with vscode. I have no idea how I could corrupt the file system yet still have it start up and work ok.

okibcn commented 1 year ago

@BryanDollery no worries about the version. Any WSL above 1.0.0 is fine for generating a compacted vhdx image. In any case, you can ensure you are in the latest WSL by typing wsl --update from any PowerShell. You are in the latest production release and it should be fine. We are now in WSL 1.1.2.0 pre-release. If you want to update to pre-release in a non-preview windows, you have to type wsl --update --pre-release to force to update to the latest build.

Regarding the image, You need to do the tar export test to ensure that your image is good for this process. In case of image corruption. The corruption could be somewhere else in the filesystem, in a file that you normally don't use. I have also have one of my images based on Ubuntu, used for development with VScode too, and only once it got corrupted, long ago, by a bug in an old beta version of WSL. In any case, something is wrong, the image or WSL, as wslcompact uses WSL.exe for automating the clean image generation.

okibcn commented 1 year ago

This issue has been solved with error tracking since v5.0. Closing as fixed. Should you have this issue again, we can reopen it again.