Azure CLI on WSL2 doesn't handle full file paths which include symlinks into /mnt/c/Users

Related command

ls -lAF ~/src
lrwxrwxrwx 1 myuser myuser 39 Sep 27 01:49 /home/myuser/src -> /mnt/c/Users/MyUserName/Workspaces/

az deployment group create --name MyClientSandboxDeployment-20221017-1311 \
                           --resource-group MyClient-Sandbox-Test \
                           --template-file /home/myuser/src/MyClient/myclient-azure-prototype/deployments/resource-group/tests/resource-group-override-test.bicep

An error occurred reading file. Could not find a part of the path 'C:\home\myusername\src\MyClient\myclient-azure-prototype\deployments\resource-group\tests\resource-group-override-test.bicep'.

Describe the bug The Azure CLI translates a full-path filename from the provided linux path to a non-existent, but seemingly equivalent Windows path using Windows path conventions when the linux path contains a symlink that points back into the Windows filesystem.

Best explained by example:

I am primarily developing some code which includes (initially) a bash-based script to setup parameters and then run Azure deployment commands on a MacBook Pro, with an Azure DevOps repo. To test in bash, because I refuse to install brew (don't like all the changes it makes) on my mac (please create a DMG installer like ALL other similar software! i.e. AWS CLI, PowerShell), and therefore can't test there, I have a separate Windows PC, so I clone/pull the repo into VS Code on Windows 11, then want to run this code within Windows Terminal in a WSL2 Ubuntu 20.04 shell. Once I get any bash variant of these helper scripts working, I usually then create an equivalent version in PowerShell. So, I want to edit in VS Code in Windows, not in WSL2.

In order for me to edit in Windows, but run the result in the WSL2 Ubuntu shell, I've created this symlink, which normally works everywhere I've tried it in the past, until now.

ls -lAF ~/src
lrwxrwxrwx 1 myuser myuser 39 Sep 27 01:49 /home/myuser/src -> /mnt/c/Users/MyUserName/Workspaces/

The repo I'm using is cloned in Windows to ~\Workspaces\MyClient\myclient-azure-prototype, so to run the utility script which prepares and runs the az statement shown above, I would (in an Ubuntu shell window):

cd ~/src/MyClient/myclient-azure-prototype/bin
./create-deployment -R MyClient-Sandbox-Test -d tests/resource-group-override-test

This creates a templatefile variable which points to:

templatefile=/home/myuser/src/MyClient/myclient-azure-prototype/deployments/resource-group/tests/resource-group-override-test.bicep

Which due to the ~/src symlink, is really pointing to the equivalent file within the repo on Windows:

C:\Users\MyUserName\Workspaces\MyClient\myclient-azure-prototype\deployments\resource-group\tests\resource-group-override-test.bicep

Running the code results in this error message: "An error occurred reading file. Could not find a part of the path 'C:\home\myusername\src\MyClient\myclient-azure-prototype\deployments\resource-group\tests\resource-group-override-test.bicep'."

You can see how somewhere inside the Azure code which accepts file-based parameters, it is translating the linux full file name, into something which is not either one of the two original full path names of this file. It's using C:\home\myusername\src, where the C:\ and path separators are Windows, the home first directory in the path is from Linux, the myusername part of the path is from Windows but now in lowercase, the src part of the path is from Linux, while the rest of the path is correct on both systems, except for different path separators.

So, the linux path which I provided as a parameter, which works in all other linux commands such as cat $templatefile, is translated by code within az cli into a path which is not valid any more on linux, nor is it correct or valid if used on Windows either.

To Reproduce Exact steps to reproduce this in the prior section.

Expected behavior Linux paths which include symlinks into /mnt/c/Users (or similar) should work. Whatever code is translating this path into a variant of the path which is invalid on both the Linux and Windows OSes is broken and should not perform this translation when az code is run on Linux.

Environment summary az cli installed via standard published method on WSL2 with Ubuntu 20.04. I have zsh along with oh-my-zsh configured on the Ubuntu shell.

Additional context This is a really annoying thing to workaround, as this will force me to either

Have a separate direct git clone inside the WSL Ubuntu shell, not reached by a symlink, so I could no longer use VS Code to edit and test this code for use in both Ubuntu (bash scripts) and Windows (PowerShell 7 scripts) I'd have to commit each change, then pull on linux. NOT something I want to do.
Translate all absolute paths into relative paths which do not go through the ~/src symlink. This works fine if it's just one repo, but I have separate repos for the templates vs per-client parameters files, so depending on where I am, I have to modify multiple file paths from absolute going through ~/src to get to where I clone all code on Windows.
If someone has a better way to do this sort of cross-platform development, I'm all ears! I want to edit the code in VS Code when working on Windows, and I need to test both bash scripts which must run within the WSL2 and PowerShell scripts which must run within PowerShell, directly in Windows. I need line endings to not get confused in Git. My method worked when using AWS CLI, was surprised it didn't work here with Azure CLI. So, this seems like a bug to me.

@jiasli for awareness

For anyone following this, I had to develop some workaround code in my bash scripts to have them work with the Azure CLI until they get around to fixing this problem.

As mentioned in my second point under additional context, this is complicated when it's not just a single absolute path which goes through a symlink into /mnt/c somewhere. For the first or main path, you can pushd to a known location, such as the HOME of the repo, then convert all relative paths to that starting point.

I use this code at the top to find the repo home based on the location of the directory running the script (so it doesn't matter where I'm running it from or where I clone my repos), and when running on WSL, I perform this adjustment.

# <Skipping comment header, but this is just below>
uname -a | grep "WSL" &> /dev/null && use_relative_paths=1 || use_relative_paths=0 # BUG: Use relative paths on WSL

#  1. Initialize environment
bindir=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
home=${bindir%/*}
[ $use_relative_paths = 1 ] && pushd $home &> /dev/null && home=. && bindir=$home/bin # BUG: Use relative paths on WSL
confdir=$home/conf

So, when on WSL, we move to pwd = /home/myuser/src/MyClient/myclient-azure-prototype for the body of the script, and:

home=.
bindir=./bin
confdir=./conf

But when not on WSL, we see the full absolute path and do not adjust pwd.

This code has the concept of a customization repo, that can augment/override what's in $home/conf with what's in $customhome/conf. This repo is likely to be in a different GitHub organization and not a sibling directory in the filesystem, so converting this path from absolute to relative needs to calculate the relative path from the $home of the first repo, which will probably have some ../ parent references in it. It turns out there's a simple program to do this, available in Ubuntu 20.04, allowing use of this code to convert additional absolute paths into relative paths.

[ $use_relative_paths = 1 ] && customhome=$(realpath --relative-to=. $customhome) # BUG: Use relative paths on WSL
customconfdir=$customhome/conf

So, when on WSL, we might have

confdir=../../AnotherClient/anotherclient-azure-custom/conf

But when not on WSL, we see the full absolute path.

At the end of the script, I add this to pop the directory stack and return to the same directory I was in when I started.

[ $use_relative_paths = 1 ] && popd &> /dev/null # BUG: Use relative paths on WSL

So, this works, but I really shouldn't have to do this - I want to use absolute paths and display them to users to avoid the confusion relative paths is likely to create. Hope this gets fixed in a timely manner!

Thank you for your feedback. This has been routed to the support team for assistance.

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @josephkwchan, @jennyhunter-msft.

Issue Details

**Related command** ``` ls -lAF ~/src lrwxrwxrwx 1 myuser myuser 39 Sep 27 01:49 /home/myuser/src -> /mnt/c/Users/MyUserName/Workspaces/ az deployment group create --name MyClientSandboxDeployment-20221017-1311 \ --resource-group MyClient-Sandbox-Test \ --template-file /home/myuser/src/MyClient/myclient-azure-prototype/deployments/resource-group/tests/resource-group-override-test.bicep An error occurred reading file. Could not find a part of the path 'C:\home\myusername\src\MyClient\myclient-azure-prototype\deployments\resource-group\tests\resource-group-override-test.bicep'. ``` **Describe the bug** The Azure CLI translates a full-path filename from the provided linux path to a **non-existent**, but seemingly equivalent Windows path using Windows path conventions when the linux path contains a symlink that points back into the Windows filesystem. Best explained by example: I am primarily developing some code which includes (initially) a bash-based script to setup parameters and then run Azure deployment commands on a MacBook Pro, with an Azure DevOps repo. To test in bash, because I refuse to install brew (don't like all the changes it makes) on my mac (please create a DMG installer like ALL other similar software! i.e. AWS CLI, PowerShell), and therefore can't test there, I have a separate Windows PC, so I clone/pull the repo into VS Code on Windows 11, then want to run this code within Windows Terminal in a WSL2 Ubuntu 20.04 shell. Once I get any bash variant of these helper scripts working, I usually then create an equivalent version in PowerShell. So, I want to edit in VS Code in Windows, not in WSL2. In order for me to edit in Windows, but run the result in the WSL2 Ubuntu shell, I've created this symlink, which normally works everywhere I've tried it in the past, until now. ``` ls -lAF ~/src lrwxrwxrwx 1 myuser myuser 39 Sep 27 01:49 /home/myuser/src -> /mnt/c/Users/MyUserName/Workspaces/ ``` The repo I'm using is cloned in Windows to ~\Workspaces\MyClient\myclient-azure-prototype, so to run the utility script which prepares and runs the az statement shown above, I would (in an Ubuntu shell window): ``` cd ~/src/MyClient/myclient-azure-prototype/bin ./create-deployment -R MyClient-Sandbox-Test -d tests/resource-group-override-test ``` This creates a templatefile variable which points to: ``` templatefile=/home/myuser/src/MyClient/myclient-azure-prototype/deployments/resource-group/tests/resource-group-override-test.bicep ``` Which due to the ~/src symlink, is really pointing to the equivalent file within the repo on Windows: ``` C:\Users\MyUserName\Workspaces\MyClient\myclient-azure-prototype\deployments\resource-group\tests\resource-group-override-test.bicep ``` Running the code results in this error message: "An error occurred reading file. Could not find a part of the path 'C:\home\myusername\src\MyClient\myclient-azure-prototype\deployments\resource-group\tests\resource-group-override-test.bicep'." You can see how somewhere inside the Azure code which accepts file-based parameters, it is translating the linux full file name, into something which is **not either one** of the two original full path names of this file. It's using `C:\home\myusername\src`, where the `C:\` and path separators are Windows, the `home` first directory in the path is from Linux, the `myusername` part of the path is from Windows but now in lowercase, the `src` part of the path is from Linux, while the rest of the path is correct on both systems, except for different path separators. So, the linux path which I provided as a parameter, which works in all other linux commands such as `cat $templatefile`, is translated by code within az cli into a path which is not valid any more on linux, nor is it correct or valid if used on Windows either. **To Reproduce** Exact steps to reproduce this in the prior section. **Expected behavior** Linux paths which include symlinks into /mnt/c/Users (or similar) should work. Whatever code is translating this path into a variant of the path which is invalid on **both** the Linux and Windows OSes is broken and should not perform this translation when az code is run on Linux. **Environment summary** az cli installed via standard published method on WSL2 with Ubuntu 20.04. I have zsh along with oh-my-zsh configured on the Ubuntu shell. **Additional context** This is a really annoying thing to workaround, as this will force me to either * Have a separate direct git clone inside the WSL Ubuntu shell, not reached by a symlink, so I could no longer use VS Code to edit and test this code for use in both Ubuntu (bash scripts) and Windows (PowerShell 7 scripts) I'd have to commit each change, then pull on linux. NOT something I want to do. * Translate all absolute paths into relative paths which do not go through the ~/src symlink. This works fine if it's just one repo, but I have separate repos for the templates vs per-client parameters files, so depending on where I am, I have to modify multiple file paths from absolute going through ~/src to get to where I clone all code on Windows. * If someone has a better way to do this sort of cross-platform development, I'm all ears! I want to edit the code in VS Code when working on Windows, and I need to test both bash scripts which must run within the WSL2 and PowerShell scripts which must run within PowerShell, directly in Windows. I need line endings to not get confused in Git. My method worked when using AWS CLI, was surprised it didn't work here with Azure CLI. So, this seems like a bug to me.

Author:	michael-crawford
Assignees:	jiasli
Labels:	`Service Attention`, `question`, `ARM`, `customer-reported`, `Installation`, `Auto-Assign`, `Azure CLI Team`
Milestone:	Backlog

Adding Service team to look into this.

Azure / azure-cli

Azure CLI on WSL2 doesn't handle full file paths which include symlinks into /mnt/c/Users #24237