carlos-montiers / enhancedbatch

Enhances your windows command prompt https://www.enhancedbatch.com
Other
5 stars 1 forks source link

EB behavior in command line context #36

Closed DaveBenham closed 4 years ago

DaveBenham commented 4 years ago

Not sure what the behavior is supposed to be, but I decided to investigate how EB behaves within a command line context.

First I created the following bat scripts (note I assume EB exists in the current directory or in PATH):

loadBatchEB.bat

@echo off

:: Return if already loaded
if defined @enhancedbatch exit /b 0

:: Load EB in batch context
rundll32.exe "enhancedbatch_%processor_architecture%.dll" load||(echo Enhanced Batch didn't load.&exit /b 1)

showBatchLine.bat

@echo %$0%:%@batchline%

test.bat

@echo off
call loadBatchEB.bat
call showBatchLine.bat

I then opened a new console running cmd.exe and did the following:

Microsoft Windows [Version 10.0.17763.864]
(c) 2018 Microsoft Corporation. All rights reserved.

C:\test>cd eb

C:\test\eb>loadBatchEB

C:\test\eb>showBatchLine
:

C:\test\eb>echo %@enhancedbatch%
%@enhancedbatch%

C:\test\eb>rundll32.exe "enhancedbatch_%processor_architecture%.dll" load

C:\test\eb>showBatchLine
showBatchLine:1

C:\test\eb>echo %@enhancedbatch%
0

C:\test\eb>

So I learned a few things

Then I opened a fresh console running cmd.exe and tried another experiment:

Microsoft Windows [Version 10.0.17763.864]
(c) 2018 Microsoft Corporation. All rights reserved.

C:\test>cd eb

C:\test\eb>rundll32.exe "enhancedbatch_%processor_architecture%.dll" load

C:\test\eb>echo %@enhancedbatch%
0

C:\test\eb>showBatchLine

After a delay, BOOM - cmd.exe/console silently crashes! That's not good.

EB was partially loaded successfully when loaded from the command line, but not in an entirely healthy way - it can't be used for subsequent batch context.

So this experiment coupled with the previous one demonstrates that some aspect of the batch context EB must persist in a way that allows EB to be loaded in a command line context.

I wonder if EB can be modified to support fully functioning persistence with a single load from batch and/or command line context.

But I decided to work with what I have and try to come up with a nice script to load a healthy persistent EB for the command line and all subsequent batch contexts. The (GOTO)&someCommand feature can be used to terminate the script and execute someCommand from the command line context.

loadCommandLineEBfail.bat

@echo off

:: Return if already loaded
if defined @enhancedbatch exit /b 0

:: Load EB in batch context
rundll32.exe "enhancedbatch_%processor_architecture%.dll" load||(echo Enhanced Batch didn't load.&exit /b 1)

:: Load EB in parent context (presumably command line) only if needed
:: DOES NOT WORK :-(
2>nul (goto) & if not defined @enhancedbatch @enhancedbatch rundll32.exe "enhancedbatch_%processor_architecture%.dll" load

And I tried the following test from a new cmd session:

Microsoft Windows [Version 10.0.17763.864]
(c) 2018 Microsoft Corporation. All rights reserved.

C:\test>cd eb

C:\test\eb>loadCommandLineEBfail

C:\test\eb>echo %@enhancedbatch%
%@enhancedbatch%

C:\test\eb>showBatchLine
:

C:\test\eb>

Well shucks, didn't work :-(

So what happened? I modified the script a bit to debug

loadCommandLineEBdbug.bat

@echo off

:: Return if already loaded
if defined @enhancedbatch exit /b 0

:: Load EB in batch context
rundll32.exe "enhancedbatch_%processor_architecture%.dll" load||(echo Enhanced Batch didn't load.&exit /b 1)

:: Load EB in parent context (presumably command line) only if needed
:: DOES NOT WORK :-(
setlocal enableDelayedExpansion
2>nul (goto) & if not defined @enhancedbatch (
  @enhancedbatch rundll32.exe "enhancedbatch_%processor_architecture%.dll" load
) else echo EB already loaded: %@enhancedbatch% !@enhanced!

From a new cmd session

Microsoft Windows [Version 10.0.17763.864]
(c) 2018 Microsoft Corporation. All rights reserved.

C:\test>cd eb

C:\test\eb>loadCommandLineEBdbug & echo %@enhancedbatch%
EB already loaded: 0 !@enhanced!
%@enhancedbatch%

C:\test\eb>

Interesting - all the code after (GOTO) was definitely executed from a command line context as expected. The batch loaded EB was functionally persistent for the subsequent commands that came from the batch script, but not for the command from the original command line command. The IF statement prevented EB from being loaded after (GOTO)

So lets try something else - Use delayed expansion to test whether we are in command line context after (goto) and if so, then always load EB again.

loadEB.bat

@echo off

:: Return if already loaded
if defined @enhancedbatch exit /b 0

:: Load EB in batch context
rundll32.exe "enhancedbatch_%processor_architecture%.dll" load||(echo Enhanced Batch didn't load.&exit /b 1)

:: Load EB in parent context only if parent is command line
2>nul (goto) & (
  set @delayedExpansion=1
  if "!!" neq "" rundll32.exe "enhancedbatch_%processor_architecture%.dll" load
  set @delayedExpansion=%@delayedExpansion%
)

And here is the final test in a new cmd session

Microsoft Windows [Version 10.0.17763.864]
(c) 2018 Microsoft Corporation. All rights reserved.

C:\test>cd eb

C:\test\eb>loadEB

C:\test\eb>echo %@enhancedbatch%
0

C:\test\eb>showBatchLine
showBatchLine:1

C:\test\eb>

Success!!!

I think this is as good as it can get with EB in its current form.

Not shown, but I verified, that EB is loaded the minimum number of times possible regardless how many times loadEB.bat is called from either the command line or within a batch script.

:)

adoxa commented 4 years ago

Not sure what the behavior is supposed to be, but I decided to investigate how EB behaves within a command line context.

Should work (I do some of my testing that way), but it's not intended for it (it's a batch enhancer).

When loading EB within a batch context, it does not functionally persist after the batch context terminates.

It's explicitly designed to exit when the batch terminates.

When loaded within a command line context, EB persists for the remainder of the cmd session, and is available for both the command line and batch contexts.

Use call @unload to remove it.

After a delay, BOOM - cmd.exe/console silently crashes!

Works for me on Win7; I'll have to remember to try it on 10.

I wonder if EB can be modified to support fully functioning persistence with a single load from batch and/or command line context.

Nifty. :clap:

DaveBenham commented 4 years ago

Works for me on Win7; I'll have to remember to try it on 10.

Are you sure? My failure is very dependent on the order of operations. Windows only crashes if the first time EB is loaded it is from the command line, and then a batch script tries to utilize EB without loading EB itself.

Once a batch script loads EB, then EB can safely be loaded from the command line.

After call @unload EB can still be safely loaded from the command line.

adoxa commented 4 years ago
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

c:\Projects\enhancedbatch>echo %@version%
%@version%

c:\Projects\enhancedbatch>rundll32 enhancedbatch_amd64.dll load

c:\Projects\enhancedbatch>echo %@version%
Dec 30 2019

c:\Projects\enhancedbatch>showBatchLine.bat
showBatchLine.bat:1

c:\Projects\enhancedbatch>

Carlos reported another crash in email, which also works for me, so maybe there's something particular to 10.

carlos-montiers commented 4 years ago

Yes, also on windows 10 and when EB is loaded in interactive mode and maybe only related to .bat files (not .cmd)

DaveBenham commented 4 years ago

Indeed I am running Win 10

adoxa commented 4 years ago

Worked in my virtual 10, too (as did the other reported crash).

Microsoft Windows [Version 10.0.18362.175]
(c) 2019 Microsoft Corporation. All rights reserved.

C:\Users\Jason>z:\eb\

Z:\eb>rundll32 enhancedbatch_amd64.dll load

Z:\eb>showBatchLine.bat
showBatchLine.bat:1

(In case you're wondering, the directory is changed via a CMDread association.)

DaveBenham commented 4 years ago

Argh,

Is your virtual 10 machine 32 or 64 bit? I'm running 10 pro 64 bit.

When I had a Win 8 64 bit machine with Virtual XP, the only option for the virtual machine was 32 bit.

adoxa commented 4 years ago

64-bit (VirtualBox). I also tried CMD versions 10.0.17763.1 & .592, which both worked.

carlos-montiers commented 4 years ago

I reproduced again, happen when you run a .bat or .cmd file after compile the dll and load that dll:

C:\Users\Carlos>cd \

C:>cd enhancedbatch

C:\enhancedbatch>cd enhancedbatch

C:\enhancedbatch\enhancedbatch>del *.dll

C:\enhancedbatch\enhancedbatch>mingw32-make
gcc -m64 -nostartfiles -s -shared -Wl,-e,_dllstart gdi_amd64.o offsets_amd64.o messages_amd64.o util_amd64.o dll_enhancedbatch_amd64.o patch_amd64.o delay_amd64.o extensions_amd64.o say_amd64.o enhancedbatch_info_amd64.o -o enhancedbatch_amd64.dll -lversion -lgdi32
gcc -m32 -nostartfiles -s -shared -Wl,-e,__dllstart,--enable-stdcall-fixup gdi_x86.o offsets_x86.o messages_x86.o util_x86.o dll_enhancedbatch_x86.o patch_x86.o delay_x86.o extensions_x86.o say_x86.o enhancedbatch_info_x86.o -o enhancedbatch_x86.dll -lversion -lgdi32

C:\enhancedbatch\enhancedbatch>rundll32 enhancedbatch_amd64.dll load

C:\enhancedbatch\enhancedbatch>cd ..

C:\enhancedbatch>cd test

C:\enhancedbatch\test>test.bat
adoxa commented 4 years ago

That still works in my 7 (without cd, but I can't see that making any difference).

C:\Projects\enhancedbatch>start cmd /d

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Projects\enhancedbatch>del *.dll

C:\Projects\enhancedbatch>mingw32-make
gcc -m64 -nostartfiles -s -shared -Wl,-e,_dllstart gdi_amd64.o offsets_amd64.o messages_amd64.o dll_enhancedbatch_amd64.o patch_amd64.o delay_amd64.o extensions_amd64.o say_amd64.o util_amd64.o enhancedbatch_info_amd64.o -o enhancedbatch_amd64.dll -lversion -lgdi32
gcc -m32 -nostartfiles -s -shared -Wl,-e,__dllstart,--enable-stdcall-fixup gdi_x86.o offsets_x86.o messages_x86.o dll_enhancedbatch_x86.o patch_x86.o delay_x86.o extensions_x86.o say_x86.o util_x86.o enhancedbatch_info_x86.o -o enhancedbatch_x86.dll -lversion -lgdi32

C:\Projects\enhancedbatch>rundll32 enhancedbatch_amd64.dll load

C:\Projects\enhancedbatch>showBatchLine.bat
showBatchLine.bat:1

C:\Projects\enhancedbatch>
carlos-montiers commented 4 years ago

Yes, maybe is other dynamic fact. I tested on other windows 10 computer, without crash. I will try execute cmd with a debugger in the computer where the crash happen

carlos-montiers commented 4 years ago

Running with WinDbg one time I cannot reproduce, but in other chance I get this:

(3cf4.12e4): Stack overflow - code c00000fd (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
ntdll!LdrpFindLoadedDllByHandle+0x3d:
00007ff9`9b0386c9 e8d20d0200      call    ntdll!RtlAcquireSRWLockExclusive (00007ff9`9b0594a0)
carlos-montiers commented 4 years ago

Stack:

ntdll!LdrpFindLoadedDllByHandle + 0x3d   
ntdll!LdrResolveDelayLoadedAPI + 0x6b   
cmd!_delayLoadHelper2 + 0x31   
cmd!_tailMerge_ext_ms_win_cmd_util_l1_1_0_dll + 0x3f   
enhancedbatch_amd64 + 0xb1cd   
adoxa commented 4 years ago

It was a problem hooking the notification stub before it was imported. That's why it didn't occur on 7, since that doesn't delay load; and it worked in 10 because I changed directory via a batch file, so the import had occurred.

carlos-montiers commented 4 years ago

Oh, I'm glad to know that finally the bug was replicated and fixed, this one was elusive. Many thanks @adoxa

carlos-montiers commented 4 years ago

@DaveBenham about:

The (GOTO)&someCommand feature can be used to terminate the script and execute someCommand from the command line context.

I tried this from in batch file:

2>nul (goto) & (
  for %a in (z x c) do echo %a
)

and it fails because in script mode is needed two %% for the for variables. Thus I understand that the (GOTO)&someCommand feature allow run commands in the parent context but in the current execution mode (script or interactive).

Also in: loadCommandLineEBfail.bat Seems this line: 2>nul (goto) & if not defined @enhancedbatch @enhancedbatch rundll32.exe "enhancedbatch_%processor_architecture%.dll" load should be: 2>nul (goto) & if not defined @enhancedbatch rundll32.exe "enhancedbatch_%processor_architecture%.dll" load

Anyways, EB detect if is already loaded, thus load it is idempotent, is not needed check if is already loaded. Thus the code for load in all the context can be simplified to:

@echo off

:: Load EB in batch context
rundll32.exe "enhancedbatch_%processor_architecture%.dll" load || (echo Enhanced Batch didn't load.&exit /b 1)

:: Load EB in parent context
2>nul (goto) & (
rundll32.exe "enhancedbatch_%processor_architecture%.dll" load || (echo Enhanced Batch didn't load.&exit /b 1)
)

With that I think this issue is really completed.

DaveBenham commented 4 years ago

I tried this from in batch file:

2>nul (goto) & (
 for %a in (z x c) do echo %a
)

and it fails because in script mode is needed two %% for the for variables. Thus I understand that the (GOTO)&someCommand feature allow run commands in the parent context but in the current execution mode (script or interactive).

No, the someCommand will be executed in the parent context with the execution mode of the parent. But either way, all the percents should be doubled because phase 1 of the parser is executed in batch mode, which will convert the %% to %. You can prove this works with the following script:

test.bat

setlocal
set @delayedexpansion=1
set local=local
2>nul (goto) & (
  for %%a in (z x c) do echo %%a  !@enhancedbatch! !local!
)

The (goto) will exit the test.bat script, doing an implicit endlocal, and the local variable will be undefined.

If you run test.bat from the command line, then you get the following:

z 0 !local!
x 0 !local!
c 0 !local!

But if you call test.bat from within another batch script, then you get

z 0
x 0
c 0

There should not be an exit /b after the (goto) because the (goto) takes the place of the exit /b

Now that EB loads properly from the command line, I would write loadEB.bat as

@if not defined @enhancedbatch 2>nul (goto) & (
  rundll32.exe "%~dp0enhancedbatch_%processor_architecture%.dll" load || >&2 echo Enhanced Batch failed to load.
)

I include the if not defined @enhancedbatch because I assume that executes faster than
rundll32.exe ...

That simple loadEB.bat can be called from the command line, or from within a batch script.

I wanted to also have it execute call %* like the EB.CMD, but you cannot use any EB functionality from within the same block that loads EB.

DaveBenham commented 4 years ago

carlos-montiers wrote: Anyways, EB detect if is already loaded, thus load it is idempotent, is not needed check if is already loaded.

DaveBenham responded: I include the if not defined @enhancedbatch because I assume that executes faster than rundll32.exe ...

Confirmed - executing rundll32 ... when EB is already loaded is extremely slow when compared to using if not defined .... I see nearly a factor of 20 difference!

testRUNDLL32.bat

rundll32.exe "%~dp0enhancedbatch_%processor_architecture%.dll" load

testIF.bat

if not defined @enhancedbatch rundll32.exe "%~dp0enhancedbatch_%processor_architecture%.dll" load

test.bat

@echo off
setlocal enableDelayedExpansion
call eb.cmd

for %%A in (RUNDLL32 IF) do (
  call @timer start
  for /l %%N in (1 1 10) do call test%%A.bat
  call @time stop
  echo %%A: !@timer!
)
exit /b

---test.bat output---

RUNDLL32: 891
IF: 46

DaveBenham wrote: I wanted to also have it execute call %* like the EB.CMD, but you cannot use any EB functionality from within the same block that loads EB.

Actually I figured out a simple way to modify loadEB.bat to be able to optionally call an extension:

if defined @enhancedbatch call %* & exit /b
rundll32.exe "%~dp0enhancedbatch_%processor_architecture%.dll" load || >&2 echo Enhanced Batch failed to load. & exit /b 1
call %*
2>nul (goto) & rundll32.exe "%~dp0enhancedbatch_%processor_architecture%.dll" load || >&2 echo Enhanced Batch failed to load.

But it would be simpler if EB could be modified to optionally not unload itself once batch processing terminates. Maybe use install instead of load to signify a "permanent" load for the current cmd.exe session. Anyway, I don't care much about the syntax, just the functionality.

adoxa commented 4 years ago

Automatically unloading is my approach to backwards compatibility, in that there's no EB to create incompatibilities in the first place.

BTW, || won't work with rundll32, it always returns 0.

DaveBenham commented 4 years ago

Automatically unloading is my approach to backwards compatibility, in that there's no EB to create incompatibilities in the first place.

Sure, I figured as such. But imagine the possibility if it were truly backward compatible such that it was safe to include EB load within AutoRun commands. You are so close.

BTW, || won't work with rundll32, it always returns 0.

Thanks. I didn't read the doc page carefully enough and failed to notice Regsvr32 vs Rundll32. But I wouldn't want to use Regsvr32 in an everyday script.

So my script would become

if defined @enhancedbatch call %* & exit /b
rundll32.exe "%~dp0enhancedbatch_%processor_architecture%.dll" load 
if not defined @enhancedbatch >&2 echo Enhanced Batch failed to load. & exit /b 1
call %*
2>nul (goto) & rundll32.exe "%~dp0enhancedbatch_%processor_architecture%.dll" load

I can't include the success check at the end because, as I said before, EB features cannot be accessed within the same block that loads EB. But it would take an extraordinary circumstance for the load to succeed within the batch context, and then immediately fail within the parent command line context.

adoxa commented 4 years ago

Sure, I figured as such. But imagine the possibility if it were truly backward compatible such that it was safe to include EB load within AutoRun commands. You are so close.

Carlos did want EB to automatically load in child processes, which I was going to do then changed my mind, because of compatibility concerns.

But I wouldn't want to use Regsvr32 in an everyday script.

Is there that much stigma associated with the name? There's little difference otherwise.

if defined @enhancedbatch call %* & exit /b

That will fail if %* contains an unterminated quote (unlikely, but possible).

DaveBenham commented 4 years ago

But I wouldn't want to use Regsvr32 in an everyday script.

Is there that much stigma associated with the name? There's little difference otherwise.

The docs say Regsvr32 is intended strictly for installation scripts. And errors are reported through ugly pop up windows instead of stderr.

if defined @enhancedbatch call %* & exit /b

That will fail if %* contains an unterminated quote (unlikely, but possible).

True that, But I think I'm willing to accept that limitation.

Alternatively I could change to IF NOT DEFINED @ENHANCEDBATCH GOTO :LOAD and then put %* and EXIT /B on separate lines.

adoxa commented 4 years ago

The docs say Regsvr32 is intended strictly for installation scripts. And errors are reported through ugly pop up windows instead of stderr.

That needs a bit of elaboration. Rundll32 will always use a dialog if the DLL does not exist or the system couldn't load it; EB will use a dialog for its messages, unless you add /S; there is no way to let the script know what the error is. Regsvr32 has /E to prevent its success dialog (EB's dialogs will still show) and /S to prevent all dialogs (including EB); it will return an error code if the DLL could not be loaded, but not if EB fails to load (de8fe51).

That will fail if %* contains an unterminated quote (unlikely, but possible).

True that, But I think I'm willing to accept that limitation.

If you're the only one using it, sure, but not if we're going to distribute it. I've enhanced eb.cmd my way, so it loads into the command line or runs a command, but not both.

DaveBenham commented 4 years ago

Here is how I would do EB.CMD. It has all the same functionality, with the addition of built in help.

Advantages:

adoxa commented 4 years ago

The original intention of eb.cmd was really as a test harness for me, to save loading and unloading from the command line all the time. I also got tired of copying the load lines from other batches for my test programs, so now I can just call eb. I think authors using EB should load it directly, though.

:: [CALL] EB UNLOAD

Calling that won't work in a batch, as there's nowhere to return to with the DLL unloaded, so, as you put it, BOOM.

::
@if "%~1"=="/?" (
  for /f "delims=: tokens=*" %%A in ('findstr "^::" "%~f0"') do @echo(%%A
  exit /b
)

That has extra lines top and bottom (maybe the comments should, the help shouldn't) and indentation that I don't like. In other programs my approach to inline help was to let the for itself do it, ending with ::~:

::~
@if "%~1"=="/?" (
  for /f "delims=: tokens=* usebackq" %%A in ("%~f0") do @(
    if "%%A"=="" (echo.
    ) else if "%%A"=="~" (exit /b
    ) else echo%%A
  )
)
DaveBenham commented 4 years ago

I think authors using EB should load it directly, though.

Implying not releasing EB.CMD? I think it is generally a good idea to have a simpler load method than calling rundll32.exe. I like having a load script.

But if you mean each script that might be shared should explicitly load EB, then sure.

:: [CALL] EB UNLOAD

Calling that won't work in a batch, as there's nowhere to return to with the DLL unloaded, so, as you put it, BOOM.

Huh? It works just fine for me (both with your EB.CMD or my variant)

@echo off
call eb2 load
echo EB Active..... @EnhancedBatch=%@Enhancedbatch%
call eb2 unload
echo EB Inactive... @EnhancedBatch=%@EnhancedBatch%

--OUTPUT--

EB Active..... @EnhancedBatch=0
EB Inactive... @EnhancedBatch=

That has extra lines top and bottom (maybe the comments should, the help shouldn't) and indentation that I don't like.

Whatever - cosmetic personal preference. I intentionally added the leading and trailing blank lines as well as indents because I find it easier to read. Obviously format and content can be changed.

In other programs my approach to inline help was to let the for itself do it, ending with ::~:

::~
@if "%~1"=="/?" (
  for /f "delims=: tokens=* usebackq" %%A in ("%~f0") do @(
  if "%%A"=="" (echo.
  ) else if "%%A"=="~" (exit /b
  ) else echo%%A
  )
)

Sure, there are any number of ways to do it. I've adopted a general strategy of using FINDSTR because I have some large scripts with help, and I get much better performance filtering with FINDSTR rather than having FOR code do all the work. Not really an issue here.

I've also played around with varying the prefix to allow selectively printing different sections of help.

Regardless how it is ultimately coded, I think built in help is generally a good idea.

adoxa commented 4 years ago

Implying not releasing EB.CMD? I think it is generally a good idea to have a simpler load method than calling rundll32.exe. I like having a load script.

Wouldn't be in git if it wasn't going to be released.

But if you mean each script that might be shared should explicitly load EB, then sure.

Yep. But then, the DLLs are already a dependency so having one more little batch probably doesn't matter. Although I think eb.cmd is overkill for that and a more specific loadeb.cmd should be used.

Huh? It [call unload] works just fine for me (both with your EB.CMD or my variant)

Not for me (still on 7).

c:\Projects\enhancedbatch>type tst.bat
@echo off
call eb
echo loaded
call eb unload
echo unloaded

c:\Projects\enhancedbatch>cmd /c tst.bat
loaded
unloaded

c:\Projects\enhancedbatch>tst.bat
loaded
<crash>
adoxa commented 4 years ago

Fixed it by adding a slight delay before freeing the library, giving the calls a chance to return. Probably not 100% reliable, but simple and it worked well enough in my tests.