how to enter the ext4 partition

[Fbougron]

Hello I am a new user of BASH with Windows preview insider

When i do the command "ls ", i can see all the NTFS partitions like one lettre each Microsoft@DESKTOP-G51CU1Q:/mnt$ ls c e f g h j k q r But no ext4 partition!!!

however i can continue to see the contents of the EXT4 partitions on Windows10-14371 with EXT2FSD ( http://www.ext2fsd.com/ ) how to be able to also view them with BASH (mount ?)

http://www.hostingpics.net/viewer.php?id=479092ext.png

[benhillis]

This is similar to #214 and #530. Currently only fixed NTFS volumes will appear as DriveFs mounts. Definitely something that we will be revisiting in the future.

[fpqc]

Fbougron, reading different FSes in Linux requires kernel-mode modules loaded, I believe, at boot on linux. Since the NT kernel reads NTFS partitions, and LXSS stores Linux-related file properties with NT extended attributes, NTFS is the only FS available (actually also ReFS but you get the picture).

To get real Ext-FS support from windows is something I don't think we can expect that from the MS team. Native Ext-FS support on Windows would be a cool feature in general, and I don't think it is really something directly linked to the Windows Subsystem for Linux (since it amounts basically to a kernel-mode filesystem driver).

WSL does do the VFS (virtual filesystem) emulation, which is the interesting part of the problem anyway. The rest of it is just a matter of coding something like ext2fsd, or a driver for it.

It would be cool if MS opened up the filesystem driver API for the NT Kernel so users could write drivers for the different linux filesystems though.

[therealkenc]

This is why I think FUSE (#414) is so important before someone above the WSL team's pay grade (#423) declares bash.exe 'done'. MS can't reasonably do a "clean room" re-implementation of Ext4, but we can do this. With CUSE someone could do OSS or ALSA over a socket even if it's not a priority for the WSL team. User Voice is here.

[fpqc]

I mean, they could do a clean-room reimplementation of Ext4, but it would be a lot of work and relatively little market for it, since the Linix side already has interop through the ntfs-3g project. But my preference would be for MS to implement something like VFS on the Windows side with a well-documented API so OSS coders could do the hard work of a fully functional Ext4 driver for Windows. The point is though that this would have to be something implemented in directly in the NT Kernel by the Kernel team, not the WSL team (they seem to only work directly with the lxss/lxcore/lxsession/bash.exe stuff, which is my guess why they have to make inter-departmental requests to the Kernel team and the Console team for changes to be made).

[therealkenc]

OSS coders can already do Ext4 on Windows without Microsoft's help, if someone felt so inclined. MS even tells you how to get started here. A VFS type layer is entirely possible too. Windows filesystems do not need to be developed by the MS NT Kernel team. It's already just a driver.

[fpqc]

@therealkenc Oh? Do the Linux FSes support all of the extended attributes and (maybe overcomplicated) security security model used by the NT kernel and implemented in NTFS?

[therealkenc]

Nope. But you could implement all of the NTFS security constructs in a Linux kernel module if one was so inclined, just as you could implement all of the Linux security constructs in a Windows driver if one was so inclined. You were asking for a "a well-documented API so OSS coders could do the hard work of a fully functional Ext4 driver for Windows". If by "fully functional" you mean including Linux security constructs, then yep you'd need to implement those in a Windows driver. But you wouldn't need the MS NT Kernel team to do that. This setting aside the fact that an Ext4 filesystem on Windows would probably care about such security constructs about as much as the ntfs-3g filesystem on Linux does (that is, not that much). You'd probably mount the thing with some blanket permissions for the user that mounted it.

[segin]

WSL queries system-wide drives for DriveFS; letter mounts that are per-user (like some network maps, or Paragon's ext4fs driver for Windows) aren't picked up.

@benhillis can you confirm that I'm accurately describing this?

[benhillis]

@segin You are mostly correct. Currently we only support fixed NTFS volumes.

[michael-wise]

Another voice chiming in. Would be great to access files from my ubuntu partition via WSL.

I'm still excited about WSL--please keep up the morale and hard work!

[GaryHendrick]

I'd like to be able to do this in WSL. Thanks for all the hard work folks

[aurimus]

Would be great to work this out to make Rails 10x faster. #893

[fpqc]

@aurimus This talk by Theodore Ts'o should probably explain why adding ext4 support is not necessarily just a matter of adding a driver:

https://www.youtube.com/watch?v=2mYDFr5T4tY

It turns out that getting good performance and supporting lots of features of the Ext filesystems would require MS to invest quite a lot of resources. Ts'o says in that talk that it takes 7-10 years for new filesystem drivers in the Linux kernel to become mature.

MS could probably relatively quickly write a small compatibility driver (something like ntfs-3g on Linux), but to really get first-party-level-performance out of it would likely take a while (or maybe coding resources that they wouldn't want to spend on such a niche feature).

Edit: But if the Microsoft team wants to prove me wrong, I'm rooting for ya.

[therealkenc]

They don't have to. Ext2/3/4 File System Driver for Windows:

https://sourceforge.net/p/ext2fsd/code/ci/master/tree/

[aurimus]

@therealkenc so you are saying that I can see linux partitions from Bash by mounting them with your mentioned driver?

[aseering]

@therealkenc -- I don't know the provenance of that driver; are you saying that it's as robust and well-optimized as the ext4 driver baked into the Linux kernel?

[aurimus]

There is also Paragon Extfs driver, would that do?

[mstrelan]

Those file system drivers for Windows (both ext2fsd and paragon extfs) work great for mounting an EXT2/3/4 volume in Windows. But as far as I know you can't access them from Bash.

[aurimus]

@mstrelan sounds like it's true..

[therealkenc]

Just saying it's doable, same as ISO9660 or FAT32. You can't mount Ext2 partitions or CDs or DOS floppy disks in WSL Creators, of course, because the infrastructure doesn't exist. Whether you can add a "yet" to that last sentence depends on where this project goes after all the low hanging fruit bits are finished.

[aurimus]

@mstrelan @Fbougron is this true answer that could be used on WSL? https://superuser.com/questions/1190580/how-to-mount-ubuntu-home-directory-for-windows-10-wsl-bash/1204888#1204888

[mstrelan]

I'm certainly not the one to ask. I think a Samba share is a less than ideal workaround. I'm sticking with NTFS partitions for now.

[mredbishop]

Native Ext4 would mean that build times could be 2-3 times quicker for us and npm installs could be up to 10 times as fast, much as they are in native Linux.

Up until yesterday I exclusively used Linux as my dev OS using VS Code but as of yesterday I am now using VS Code in WSL and the only issue is the disk performance. Other than that, I find WSL is very capable, saving me from having to switch back and forth from Windows to Linux when I need one or the other.

[fpqc]

@mredbishop Ext4 doesn't have anything to do with it, really. The trouble starts from the fact that there is no *nix-compatible VFS layer in Windows, so inodes and things like Linux permissions, owner, and attributes all need to be emulated in software, which is slow.

My understanding is that the WSL team is working with the NTFS team to have these features (or at least features that simplify and speed up the most expensive parts of them) implemented directly in NTFS at the driver level.

A side effect of this work is that it will also probably improve interop, since if Linux-style attributes are stored directly as ordinary rather than extended attributes, they should be preserved better by Windows tools, and they should also be faster to compute, since they could be queried directly from the driver instead of read off and parsed in the LX driver from the NTFS extended attributes.

Similarly, computing and caching inodes could be in the NTFS driver, which would be much faster, and perhaps even *nix-style unlink semantics could be implemented without using hacks (the current method moves the open files to a hidden directory, then the unlink happens when the open handle to the file is closed.

But none of this has to do with ext4.

[therealkenc]

Ext4 doesn't have anything to do with it, really.

The ask in the OP is for mount -t drvfs support for extfsd2 (ie the ext2fsd.sys filesystem driver). Much the same Windows 10 native NFS support #1962 (ie msnfsflt.sys/nfsrdr.sys despite the self imploding issue title chosen over there). Much the same Windows 10 native WebDAV (ie mrxdav.sys) support #2163. The OP predates the filesystem improvements, but the equivalent was being done for ntfs.sys (but not cdfs.sys, udfs.sys, exfat.sys which now work) automatically at WSL instantiation since the beginning. At the time of the post, Fbougron wanted to know why his ext2fsd drive was not appearing along with his NTFS drive, despite being available in Windows.

But none of this has to do with ext4.

Rather, almost none of the comments in this thread has to do with the ask.

[Menci]

Now with ext2fsd 0.68 (NOT the latest 0.69) I can cd to the ext4 part. But I can't chroot to it. It shows Input / Output Error.

Here is the strace result:

PS C:\> bash
Menci@Menci-Surface:/mnt/c$ sudo mount --bind /dev /mnt/e/dev
[sudo] password for Menci:
Menci@Menci-Surface:/mnt/c$ sudo mount --bind /proc /mnt/e/proc
Menci@Menci-Surface:/mnt/c$ sudo mount --bind /sys /mnt/e/sys
Menci@Menci-Surface:/mnt/c$ sudo strace chroot /mnt/e/
execve("/usr/sbin/chroot", ["chroot", "/mnt/e/"], 0x7fffea8f0df8 /* 14 vars */) = 0
brk(NULL)                               = 0x7fffc3f40000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
[...snip]
lstat("/mnt/e", {st_mode=S_IFDIR|0777, st_size=512, ...}) = 0
chroot("/mnt/e/")                       = 0
chdir("/")                              = 0
execve("/bin/bash", ["/bin/bash", "-i"], 0x7fffcbd002f0 /* 14 vars */) = -1 ENOEXEC (Exec format error)
execve("/bin/sh", ["/bin/sh", "/bin/bash", "-i"], 0x7fffcbd002f0 /* 14 vars */) = -1 EIO (Input/output error)
openat(AT_FDCWD, "/usr/share/locale/C.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 
[...snip]
write(2, ": Input/output error", 20: Input/output error)    = 20
[...snip]
+++ exited with 126 +++
Menci@Menci-Surface:/mnt/c$ ls /mnt/e
bin   cdrom  etc   initrd.img      lib    lost+found  mnt  proc  run   snap  sys  usr  vmlinuz
boot  dev    home  initrd.img.old  lib64  media       opt  root  sbin  srv   tmp  var  vmlinuz.old
Menci@Menci-Surface:/mnt/c$

[therealkenc]

Your chroot(1) scenario is unfortunately not well formed. The filesystem is ext2, but the filesystem API is still "Windows" (aka DrvFS on WSL). The ext2fsd driver won't support Unix UID/GID, Unix modes, Unix symlinks, device nodes (critically, /dev/tty*), named pipes, Unix sockets, yadda yadda. It does not matter that ext2 (the filesystem) supports these things. The problem is the Windows filesystem API supports these things only very recently, and the ext2fsd driver most certainly does not.

In other words (simplifying), the ext2fsd driver (sic, not WSL) is trying to make a your ext2 filesystem look like NTFS -- by design. What you are trying to do is dupe #2575 (message) on WSL prior to the April 2018 update. Which is the same as trying to chroot() into a Samba share or ntfs-3g on Real Linux™. Which doesn't work.

In theory ext2fsd could implement the new Windows "LX" filesystem API extensions. In theory that could be made compatible with WSL. In theory. But not in the space-time continuum we are in today.

[Rondom]

Is there documentation on what is required of an IFS to support WSL in the best way possible?

[therealkenc]

No not specifically. As you know, documentation on writing a IFS that works on Windows in the "best way possible" is scant enough. I venture anything that could (in principle) be written specifically for WSL (say a blog post) would have a tiny target audience, and anything such a write-up contained today (say: "we know this doesn't work" or "we expect the driver to behave like such-and-such") would be at best a moving target. In any case, "such-and-such" amounts to: if your filesystem works exactly like ntfs.sys, it will work with WSL. Because WSL wouldn't know the difference.

That said, if you had for example dokan1.sys (saw your pinned repos) under live kernel debugging with a second machine, you could watch the callouts pretty clearly and possibly guess what was going wrong. If the problem was nonobvious (to an IFS developer), and you had a clear and concise technical question, you could try opening an issue in this github and maybe get some feedback from the devs. Doesn't hurt to ask, anyway.

FWIW accessing Dokany 1.1.0.2000 mirrorfs currently fails with unknown type:54 from DispatchQueryInformation when a mount -t drvfs is attempted on the WSL side. This appears to be because FileStandardLinkInformation isn't handled. So, you fix that, and then move on to the next thing. Rinse repeat.

C:\Users\there\source\dokany\x64\Debug>mirror /s /r C:\Users\there\source /l L:
AllocationUnitSize: 512 SectorSize: 512
device opened
###QueryVolumeInfo -001
mounted: L: -> \Volume{d6cc17c5-1735-4085-bce7-964f1e9f5de9}
###Create 0000
CreateFile status = 0
###GetFileInfo 0000
        result =  0
  unknown type:54
        DispatchQueryInformation result =  c000000d

[Rondom]

Thanks for your comment. I am aware that one can solve this with trial and error, but I was just being optimistic thinking that maybe some documentation exists or is in the plans, which would probably make this an easier endeavour...

[mcmordie]

This is similar to #214 and #530. Currently only fixed NTFS volumes will appear as DriveFs mounts. Definitely something that we will be revisiting in the future.

Okay, it's the future now. Any plans to implement support for mounting ext4 partitions? This is blocking for a number of our use cases.

[Biswa96]

There may be support for ext4 file system in future. If you've Windows 10 insider build 18262 or greater then run this command using strings from sysinternals.

set X=C:\Windows\System32\lxss\LxssManager.dll
strings.exe %X% | find /i "ext4"

LxssManager.dll has a string ext4.vhdx.

Another evidence supports my investigation. This error showed in Process Monitor while uninstalling a distro. LxssManager can't find the ext4.vhdx file.

[therealkenc]

This is similar to #214 and #530. Currently only fixed NTFS volumes will appear as DriveFs mounts. Definitely something that we will be revisiting in the future.

Okay, it's the future now.

FWIW the future came in 16176, in April 14, 2017. The volume being mounted in the OP was not a fixed NTFS volume; instead, it was a a fixed ext2fsd volume, which according to Menci mounts now. Similar to, like Ben said in the post you quote, #214 or #530. I just didn't have the heart to close the issue because it acts as a useful dupe target.

This is blocking for a number of our use cases.

Why is it always "a number of use cases" and never one use case. [Don't answer that, I'm mumbling to myself again not you.]

There may be support for ext4 file system in future. LxssManager.dll has a string ext4.vhdx.

More likely they need to present a static ext4 image to userspace as a ro block device and/or to satisfy some ABI I haven't thought of. That's my guess. My guess could be wrong. I think it is a better guess than your guess, tho. But to be sure, I wish you were right.

[segin]

Reading this thread, I have come to find it's an utter cluster.

WSL has no bearing upon ext4 support. There's nothing magical about it that would require it to support ext4 - there's no syscall in Linux that is strictly for ext4, all POSIX file system semantics are fully supported already with NTFS, etc.

There's no reason for this feature request to happen. It's outside of the scope of the tool, and all the tool does is translate Linux syscalls to NT kernel syscalls.

Microsoft would be better off writing an ext4 IFS and shipping it as part of Windows, which again, would fall wholly outside of the scope of WSL.

A more short-term solution could be had in implementing the FUSE API, but that would almost certainly restrict the ext4 mount to being available only to the current WSL instance (won't make a Windows drive letter or be available to Windows programs in any way, or to other Linux distros running at the same time. Don't try to mount the volume twice, that'll wreck it.)

In short, this is a feature request that may be difficult, if not impossible, to properly deliver.

On Sun, Nov 18, 2018, 12:57 therealkenc <notifications@github.com wrote:

This is similar to #214 https://github.com/Microsoft/WSL/issues/214 and

530 https://github.com/Microsoft/WSL/issues/530. Currently only fixed

NTFS volumes will appear as DriveFs mounts. Definitely something that we will be revisiting in the future. Okay, it's the future now.

FWIW the future came in 16176 https://docs.microsoft.com/en-us/windows/wsl/release-notes#build-16176, in April 14, 2017. The volume being mounted in the OP https://github.com/Microsoft/WSL/issues/579#issue-162127743 was not a fixed NTFS volume; instead, it was a a fixed ext2fsd volume, which according to Menci mounts now https://github.com/Microsoft/WSL/issues/579#issuecomment-399637504. Similar to, like Ben said in the post you quote, #214 https://github.com/Microsoft/WSL/issues/214 or #530 https://github.com/Microsoft/WSL/issues/530. I just didn't have the heart to close the issue because it acts as a useful dupe target.

This is blocking for a number of our use cases.

Why is it always "a number of use cases" and never one use case. [Don't answer that, I'm mumbling to myself again not you.]

There may be support for ext4 file system in future. LxssManager.dll has a string ext4.vhdx.

More likely they need to present a static ext4 image to userspace as a ro block device and/or to satisfy some ABI I haven't thought of. That's my guess. My guess could be wrong. I think it is a better guess than your guess, tho. But to be sure, I wish you were right.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Microsoft/WSL/issues/579#issuecomment-439716381, or mute the thread https://github.com/notifications/unsubscribe-auth/AAdVxVwDOl2WNQX7BB8FeCm8UEGQa_jZks5uwa2SgaJpZM4I9r0a .

[mcmordie]

@segin not a cluster. We are looking for vital functionality here and it depends on WSL understanding ext4 filesystems properly. WSL is emulating a Linux environment and it cannot do that properly without speaking ext4.

Use cases:

1. Using qemu-arm-static to chroot into embedded filesystems located within WSL's home directory. Status: this almost works, but issues with permissions and ownership still not working properly prevent the use of sudo commands. We use Qt within these systems and qmake returns an error about cannot stat ".".

2. Mounting remote ext4 filesystems in order to have multi-platform projects building eg. Win32/Linux-ARM/Linux-x86.

[therealkenc]

it depends on WSL understanding ext4 filesystems properly. WSL is emulating a Linux environment and it cannot do that properly without speaking ext4.

Replace "ext4" above with btrfs or reiserfs. Now replace that with lxfs. They all have *nix semantics, including stat(2). You can chroot(1) into a arm root on lxfs right now. The statement "speaking ext4" isn't particularly well formed (except in really exceptional edge cases).

Now, I can maybe assume what want. Maybe you want to mount a ext4 image on a loop device. [Not that I would use WSL for the purpose if it were me. WSL is demonstrably the wrong tool for the scenario right now.] But even if this is the case, the scenario that has nothing to do with the OP, which is about mounting an Windows ext2fsd drive in WSL.

In short, "building multi-platform projects" doesn't have anything to do with "speaking ext4" or "speaking reiserfs" or "speaking zfs" or "speaking NFSv4". If you've got problems with qmake or whatever, okay. If you can narrow the problem to some syscall surface that is behaving differently on WSL on lxfs versus Real Linux on ext4, or some other syscall diverge, that's legit enough. Maybe it can be looked at. It won't be a trivial exercise to demonstrate such a diverge, mind. But in any case, such a diverge won't have anything to do with whether your chroot lives on an ext4 filesystem mounted on a loop device (or any other filesystem that respects *nix semantics). qmake doesn't have a clue whether it is living in a ext4 filesystem image (or resierfs image) on a loop device, versus living in lxfs.

[segin]

@mcmordie Let me try to tell you this again because it seems you either didn't understand, or didn't like what you were told and are being arrogant about the reality of the situation.

Where you wrote: "WSL understanding ext4 filesystems properly"... WSL does not understand ANY filesystem. It does not understand NTFS. That's the NTFS driver's job, and not part of WSL. Similarly, ext4 support would be the job of a separate, external, and disconnected ext4 driver. That would be the job of the main Windows kernel team to create, not the WSL team.

Where you wrote: "WSL is emulating a Linux environment and it cannot do that properly without speaking ext4." I can compile a Linux kernel, yes, a real Linux kernel, latest version straight from kernel.org, and configure it's build options so that this very real Linux kernel also does not speak ext4. Because what I have said is fully possible, it completely disproves that ext4 support is needed to "fully" emulate a Linux ABI - hell, the BSD systems do it without needing ext4 as well.

Where you wrote: "Mounting remote ext4 filesystems in order to have multi-platform projects building eg. Win32/Linux-ARM/Linux-x86." Remote filesystems have no bearing over what WSL can and cannot do, unless you're using iSCSI or Fibre Channel connected storage where raw block-level access of drives is performed over the network.

So yes, this thread is a cluster because it's full of people asking the wrong team for something that team isn't responsible for. I wouldn't ask the President of Mexico for a change in US law and push the matter anyway, trying to argue "but it's all on Earth so they should be able to figure it out". That's basically what you're doing here.

[therealkenc]

WSL does not understand ANY filesystem

Nit - WSL understands lxfs and drvfs filesystems equivalently to any other diskless filesystem such as nfs or cifs or even procfs or tmpfs which lives in memory. There is no requirement for a filesystem to have an on-disk layout of any kind. WSL understands filesystems fine. [I know what you meant; I just wanted to clarify the quoted text above, which has correctness problems in boldface.]

So yes, this thread is a cluster because it's full of people asking the wrong team for something that team isn't responsible for.

More than one way to skin a cat, but that's a fair enough (if not exclusive) way to look at it. Mostly this issue is a cluster because no one has read the OP. Also because no one has come to the acceptance part of the grief stages - namely that MSFT isn't doing a clean-room ext4 re-implementation full stop (whatever team one thinks they should assign the task to). Because that would be silly.

[gusbemacbe]

Hello @therealkenc

I have still this issue. I have Linux partitions in the internal disk. I read this whole topic. I also read #689, #2896 and #4531.

I use Paragon ExtFS for Windows and not Ext4FS.

See the screenshots. The drivers E: (/opt), :F (/opt) and :G (/) are Linux ext4 partitions.

I followed the small tutorials above.

Similar to #4531, I use the Link Shell Extension to create a symlink from E:\myuser\ISOs to C:\Users\myuser\:

I know the articles:

1. https://blogs.msdn.microsoft.com/wsl/2017/04/18/file-system-improvements-to-the-windows-subsystem-for-linux/.
2. https://superuser.com/questions/1114341/windows-10-ubuntu-bash-shell-how-do-i-mount-other-windows-drives
3. https://superuser.com/questions/1245210/access-ubuntu-partition-from-windows-sub-system
4. https://superuser.com/questions/1109993/accessing-removable-media-in-bash-on-windows
5. https://superuser.com/questions/1190580/how-to-mount-ubuntu-home-directory-for-windows-10-wsl-bash/1204888

[therealkenc]

I have still this issue.

Given the green open button at the top that is not too surprising. Paragon could put their FS under a debugger and look, if they were motivated to do so. A FOSS developer could do the same with Ext3fsd, which best I can tell hasn't had a commit since early 2018.

There is some recent chatter on the broader topic in #2999. An experiment to try might be to export/share that Paragon filesystem in WIndows, and them mount -t cifs it under WSL2. You'll need to turn off the (default) localhost bridge thing. Haven't tried with Paragon (don't have it) but it might work. Might not too. Bonne chance.

[mcmordie]

Folks, just an update to say that this is completely resolved at least for me, on WSL2. qmake, which was unable to stat filesystem entries due to a missing system call, is now working correctly. Thanks to everyone who helped make this a reality.

[muHashh]

Folks, just an update to say that this is completely resolved at least for me, on WSL2. qmake, which was unable to stat filesystem entries due to a missing system call, is now working correctly. Thanks to everyone who helped make this a reality.

How were you able to mount an ext4 partition? I'm using WSL2 and I can't access my 2nd drive's ext4 partiton (its NTFS partition is the only one I can see).

[mcmordie]

Mine was a side-effect of the OP's issue-- being able to stat filesystem objects inside chroots which wasn't working because the implementation didn't fully support ext4 internally. I have also been able to mount ext4 partitions on a looper in order to prepare boot media. But mounting real hardware with ext4 partitions is another matter entirely because the host OS (Windows) would have to provide access to WSL2 to the underlying block device.

[pcause]

If we could have access to a raw disk partition / USB device from WSL2 Linux could handle the filesystem access. Perhaps admin access of something would be required, but why not support this. Virtualbox enables this although you have to dig in the docs to figure out how. MS should be able to implement this pretty easily.

[craigloewen-msft]

We've now added support to access ext4 and other Linux file formatted physical hard disks in a WSL 2 distro.

See the details here in this blog post.

[therealkenc]

The OP ask is was drvfs interop with ext2fsd. Analogous open #530 #2163 #2999 (probably others). They could all be tagged by-design or external, but #2999 got deemed fair game feature request to look at, ergo....