chocolatey / choco

Chocolatey - the package manager for Windows
https://chocolatey.org
Other
10.2k stars 896 forks source link

refreshenv eats percent symbols in environment variable values #1664

Open johnyesberg opened 5 years ago

johnyesberg commented 5 years ago

What You Are Seeing?

refreshenv.cmd doesn't correctly handle environment variables with a value that includes a "%" character.

I'm using Windows 10 and choco v0.10.8.

What is Expected?

I would like the environment variable value after a refreshenv to be the same as if I create a new process.

How Did You Get This To Happen? (Steps to Reproduce)

Output Log

I note that the refreshenv.cmd uses a temporary file %TEMP%\_env.cmd The script does delete some of the files that it creates in this folder, but not _env.cmd, so it is possible to observe the contents of the file after running refreshenv. In my case, the relevant line is

set "test=test%20value"

This seems ok, but the command processor is interpreting the %2 as a command line argument, so the final value of the variable is test0value.

Thoughts about possible fixes

One possibility would be to replace all occurrences of % in the _env.cmd file with %% before calling the cmd file. It does seem to solve the current problem, but it appears to cause a problem: when I edit my path variable using the windows GUI, I see one entry which is %JAVA_HOME%\bin. This variable is expanded by the system at some point, because when I run set in my cmd.exe window, the result does not show %JAVA_HOME%\bin, it shows C:\Program Files\Java\jre1.8.0_181\bin. If I naively replace all single % with %%, then the resulting path does not have that variable replaced. In my case, it might be possible to replace % with %% in all the non-path variables. But it's not clear that would work for other people, who may want variable expansion in non-path variables.

I'm not sure if there is a sensible way to distinguish between cases where the % should be quoted, and cases where it should not.

Another workaround I am considering is to store my environment variable value with %%: setx test test%%20value Then, when I look at the value in a fresh cmd.exe process, it will be test%%20value. After refreshenv, the value will be test%20value. My application that uses the value could then replace occurrences of %% with % before processing the value further. For my application, where the value I'm storing is an encoded URL, I don't need to represent a real value that could have %% in it literally, so the replacement should be safe. But this doesn't seem safe in general.

Interested in peoples' thoughts.

MPagel commented 5 years ago

I like to organize my Path into categorized groupings. To this end, I have many expandable strings in my environment variables

_SysPChoc=C:\ProgramData\chocolatey\bin;C:\ProgramData\Boxstarter;P:\Programs\64bit\Node\
_SysPIntel=C:\Program Files (x86)\Common Files\Intel\Shared Libraries\;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.2.187\windows\mpi\intel64\bin;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.4.210\windows\mpi\intel64\bin;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.5.267\windows\mpi\intel64\bin;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.6.270\windows\mpi\intel64\bin;C:\Program Files (x86)\IntelSWTools\debugger_2017\gdb\intel64_igfx\;C:\Program Files (x86)\IntelSWTools\debugger_2017\gdb\intel64_igfx\bin;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\ia32_win\mpirt;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64_win\compiler;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\ia32_win\compiler;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64_win\mpirt;C:\Program Files (x86)\Common Files\Intel\Licenses
_SysPJava=C:\ProgramData\Oracle\Java\javapath;C:\Program Files\Java\jdk1.8.0_45\bin
_SysPPerl=P:\Programs\64bit\Perl\Strawberry\c\bin;P:\Programs\64bit\Perl\Strawberry\perl\site\bin;P:\Programs\64bit\Perl\Strawberry\perl\bin
_SysPPy=C:\Python27amd64\;C:\Python27amd64\Scripts\;C:\Program Files\Python\27\;C:\Program Files\Python\27\Scripts;C:\Python27\;C:\Python27\Scripts
_SysPSQL=C:\Program Files\Microsoft SQL Server\140\DTS\Binn\;C:\Program Files\Microsoft SQL Server\140\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\140\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\140\Tools\Binn\ManagementStudio\;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files\Microsoft SQL Server\130\DTS\Binn\;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\;C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\110\Tools\Binn\ManagementStudio\;C:\Program Files (x86)\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files\Microsoft SQL Server\100\Tools\Binn\Redist\
_SysPVSdotNET=P:\Programs\64bit\Microsoft VS Code\bin;C:\Program Files\dotnet\;C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\PrivateAssemblies\
_SysPWin=C:\Windows\system32;C:\Windows\;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\
A_ProPath=%_SysPWin%;%_SysPIntel%;%_SysPVSdotNET%;%_SysPPy%;%_SysPPerl%;%_SysPChoc%;%_SysPJava%;%_SysPSQL%;C:\Program Files\Git\cmd;P:\Programs\32bit\ActiveState\Komodo\KE_11\;C:\Program Files (x86)\ActiveState Komodo Edit 9 nightly\;C:\Program Files\Microsoft MPI\Bin\;C:\Program Files (x86)\Graphviz2.38\bin;C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\;C:\Program Files\TortoiseSVN\bin;C:\Program Files (x86)\Common Files\Seagate\SnapAPI\;C:\Program Files (x86)\VEMCO\VUE
Path=%A_ProPath%

My User's Path is Path=%_PyPath%;%20181005_Path%

echo %PATH% gives me

C:\Windows\system32;C:\Windows\;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries
_2017.2.187\windows\mpi\intel64\bin;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.4.210\windows\mpi\intel64\bin;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.5.267\windows\mpi\intel6
4\bin;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.6.270\windows\mpi\intel64\bin;C:\Program Files (x86)\IntelSWTools\debugger_2017\gdb\intel64_igfx\;C:\Program Files (x86)\IntelSWTools\debugger_2017\g
db\intel64_igfx\bin;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\ia32_win\mpirt;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64_win\compiler;C:\Program Files (x86)\Common Fi
les\Intel\Shared Libraries\redist\ia32_win\compiler;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64_win\mpirt;C:\Program Files (x86)\Common Files\Intel\Licenses;P:\Programs\64bit\Microsoft VS Co
de\bin;C:\Program Files\dotnet\;C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\PrivateAssemblies\;C:\Python27amd64\;C:\Python27amd64\Scripts\;C:\Program Files\Python\27\;C:\Program Files\Python\27\Script
s;C:\Python27\;C:\Python27\Scripts;P:\Programs\64bit\Perl\Strawberry\c\bin;P:\Programs\64bit\Perl\Strawberry\perl\site\bin;P:\Programs\64bit\Perl\Strawberry\perl\bin;C:\ProgramData\chocolatey\bin;C:\ProgramData\Boxstarte
r;P:\Programs\64bit\Node\;C:\ProgramData\Oracle\Java\javapath;C:\Program Files\Java\jdk1.8.0_45\bin;C:\Program Files\Microsoft SQL Server\140\DTS\Binn\;C:\Program Files\Microsoft SQL Server\140\Tools\Binn\;C:\Program Fil
es (x86)\Microsoft SQL Server\140\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\140\Tools\Binn\ManagementStudio\;C:\Program Files\Microsoft SQL Server\1
30\Tools\Binn\;C:\Program Files\Microsoft SQL Server\130\DTS\Binn\;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files (x86)\
Microsoft SQL Server\130\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\;C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Bin
n\;C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\110\Tools\Binn\ManagementStudio\;C:\Program Files (x86)\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files\Micro
soft SQL Server\100\Tools\Binn\Redist\;C:\Program Files\Git\cmd;P:\Programs\32bit\ActiveState\Komodo\KE_11\;C:\Program Files (x86)\ActiveState Komodo Edit 9 nightly\;C:\Program Files\Microsoft MPI\Bin\;C:\Program Files (
x86)\Graphviz2.38\bin;C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\;C:\Program Files\TortoiseSVN\bin;C:\Program Files (x86)\Common Files\Seagate\SnapAPI\;C:\Program Files (x86)\VEMCO\VUE

If I do a refreshenv, it balloons in size (and exhibits the reported %2 dropping issue):

C:\Windows\system32;C:\Windows\;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries
_2017.2.187\windows\mpi\intel64\bin;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.4.210\windows\mpi\intel64\bin;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.5.267\windows\mpi\intel6
4\bin;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.6.270\windows\mpi\intel64\bin;C:\Program Files (x86)\IntelSWTools\debugger_2017\gdb\intel64_igfx\;C:\Program Files (x86)\IntelSWTools\debugger_2017\g
db\intel64_igfx\bin;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\ia32_win\mpirt;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64_win\compiler;C:\Program Files (x86)\Common Fi
les\Intel\Shared Libraries\redist\ia32_win\compiler;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64_win\mpirt;C:\Program Files (x86)\Common Files\Intel\Licenses;P:\Programs\64bit\Microsoft VS Co
de\bin;C:\Program Files\dotnet\;C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\PrivateAssemblies\;C:\Python27amd64\;C:\Python27amd64\Scripts\;C:\Program Files\Python\27\;C:\Program Files\Python\27\Script
s;C:\Python27\;C:\Python27\Scripts;P:\Programs\64bit\Perl\Strawberry\c\bin;P:\Programs\64bit\Perl\Strawberry\perl\site\bin;P:\Programs\64bit\Perl\Strawberry\perl\bin;C:\ProgramData\chocolatey\bin;C:\ProgramData\Boxstarte
r;P:\Programs\64bit\Node\;C:\ProgramData\Oracle\Java\javapath;C:\Program Files\Java\jdk1.8.0_45\bin;C:\Program Files\Microsoft SQL Server\140\DTS\Binn\;C:\Program Files\Microsoft SQL Server\140\Tools\Binn\;C:\Program Fil
es (x86)\Microsoft SQL Server\140\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\140\Tools\Binn\ManagementStudio\;C:\Program Files\Microsoft SQL Server\1
30\Tools\Binn\;C:\Program Files\Microsoft SQL Server\130\DTS\Binn\;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files (x86)\
Microsoft SQL Server\130\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\;C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Bin
n\;C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\110\Tools\Binn\ManagementStudio\;C:\Program Files (x86)\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files\Micro
soft SQL Server\100\Tools\Binn\Redist\;C:\Program Files\Git\cmd;P:\Programs\32bit\ActiveState\Komodo\KE_11\;C:\Program Files (x86)\ActiveState Komodo Edit 9 nightly\;C:\Program Files\Microsoft MPI\Bin\;C:\Program Files (
x86)\Graphviz2.38\bin;C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\;C:\Program Files\TortoiseSVN\bin;C:\Program Files (x86)\Common Files\Seagate\SnapAPI\;C:\Program Files (x86)\VEMCO\VUE;C:\Windows
\system32;C:\Windows\;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.2.18
7\windows\mpi\intel64\bin;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.4.210\windows\mpi\intel64\bin;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.5.267\windows\mpi\intel64\bin;C:\P
rogram Files (x86)\IntelSWTools\compilers_and_libraries_2017.6.270\windows\mpi\intel64\bin;C:\Program Files (x86)\IntelSWTools\debugger_2017\gdb\intel64_igfx\;C:\Program Files (x86)\IntelSWTools\debugger_2017\gdb\intel64
_igfx\bin;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\ia32_win\mpirt;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64_win\compiler;C:\Program Files (x86)\Common Files\Intel\
Shared Libraries\redist\ia32_win\compiler;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64_win\mpirt;C:\Program Files (x86)\Common Files\Intel\Licenses;P:\Programs\64bit\Microsoft VS Code\bin;C:\
Program Files\dotnet\;C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\PrivateAssemblies\;C:\Python27amd64\;C:\Python27amd64\Scripts\;C:\Program Files\Python\27\;C:\Program Files\Python\27\Scripts;C:\Pytho
n27\;C:\Python27\Scripts;P:\Programs\64bit\Perl\Strawberry\c\bin;P:\Programs\64bit\Perl\Strawberry\perl\site\bin;P:\Programs\64bit\Perl\Strawberry\perl\bin;C:\ProgramData\chocolatey\bin;C:\ProgramData\Boxstarter;P:\Progr
ams\64bit\Node\;C:\ProgramData\Oracle\Java\javapath;C:\Program Files\Java\jdk1.8.0_45\bin;C:\Program Files\Microsoft SQL Server\140\DTS\Binn\;C:\Program Files\Microsoft SQL Server\140\Tools\Binn\;C:\Program Files (x86)\M
icrosoft SQL Server\140\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\140\Tools\Binn\ManagementStudio\;C:\Program Files\Microsoft SQL Server\130\Tools\B
inn\;C:\Program Files\Microsoft SQL Server\130\DTS\Binn\;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files (x86)\Microsoft
SQL Server\130\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\;C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn\;C:\Prog
ram Files (x86)\Microsoft SQL Server\110\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\110\Tools\Binn\ManagementStudio\;C:\Program Files (x86)\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files\Microsoft SQL S
erver\100\Tools\Binn\Redist\;C:\Program Files\Git\cmd;P:\Programs\32bit\ActiveState\Komodo\KE_11\;C:\Program Files (x86)\ActiveState Komodo Edit 9 nightly\;C:\Program Files\Microsoft MPI\Bin\;C:\Program Files (x86)\Graph
viz2.38\bin;C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\;C:\Program Files\TortoiseSVN\bin;C:\Program Files (x86)\Common Files\Seagate\SnapAPI\;C:\Program Files (x86)\VEMCO\VUE;;C:\Users\mpagel\App
Data\Local\Programs\Python\Python35-32\Scripts\;C:\Users\mpagel\AppData\Local\Programs\Python\Python35-32\;;0181005_Path

So, clearly it's not handling expandable strings correctly, nor literal %s. My guess is that it treats non-expandable strings as if they are expandable. I think they should be doing something akin to call set var=!var:%%=%%%%! but are just doing call set var=!var!

example script

@echo off
setlocal EnableDelayedExpansion
echo Putting value into variable
echo ===========================
set "bobo=%%2018"
echo.%bobo%
REM will output `%2018`
echo.
echo Now re-setting properly
echo =======================
call set bobo=!bobo:%%=%%%%!
echo.%bobo%
REM will output `%2018`
echo.
echo Now re-setting lazily
echo =====================
call set bobo=!bobo!
echo.%bobo%
REM will output `018`