Choco-Cleaner - Improve bad shim detection and make deletion optional

[Wowbagger15]

Hi Bill,

Recently got surprised by missing exectuables in my 'bin' directory and found out that choco-cleaner had removed them. Not many people will have such a setup but with me that directory has been around for years containing all kinds of command line and small gui tools that do not require installation. As such, they happily co-exist with Chocolatey generated executables (so long as names are unique of course).

Because the binaries also have other sources than Chocolatey packages, they are not shimmed, fail the shim test miserably and are thus deleted by the cleanup script. Each generated executable has the fact that ShimGenerator created it written in various places in the executable. The most easy way to read this in PowerShell is by getting the FileDescription from the VersionInfo table via get-item.

Also, please consider making the test-and-delete bad shims part configurable (a mere DeleteBadShims or the like addition to the config file and script).

Both suggested changes are in the following diff excerpt (diff -wB org new):

24a25
> $DeleteBadShims = $ConfigFile.Settings.Preferences.DeleteBadShims
51,56c52,58
<     $TargetExists = & "$ShimFile" "--shimgen-help" |
<                      Select-String -pattern "Target Exists: 'True'"
<     if (-not $TargetExists) {
<        Return $false
<     } else {
<        Return $true
---
>     if ( -not ( $ShimFile | get-item -ea silentlyContinue | select -expand versionInfo | select -expand fileDescription |? { $_ -iMatch '(Shim(|Gen(|erator)))' } ) ) {
>         return $true;
>     }
>
>     try {
>         if ( $_result = & "$ShimFile" "--shimgen-help" |? { $_ -iMatch 'target exists' } ) {
>             return [bool]( $_result -iMatch 'true' );
57a60,64
>     } catch {
>         # catching security, antivirus and other execution exceptions;
>         # we do not want to delete everything that behaves unexpected
>       }
>     return $true;
255a263
> if ($DeleteBadShims -eq "True"){
270c278
<
---
> }

[bcurran3]

I'll investigate this. Thanks for the code snippet to try!

Honestly I don't think this will become a configurable option as most people that have Choco-Cleaner installed want it to clean up as best it can and I think you hit a rare edge case that can hopefully be better solved with your code suggestion.

I have a bunch of little CLI tools and such that I use too, but I keep them in my c:\tools dir. I don't want to come off as insensitive, I'm sure it was an unpleasant surprise when you found your tools missing, but you might want to stash them in a different location, at least until I get this better handling put in.

[bcurran3]

Would you like to do a PR for this?

[bcurran3]

Actually I did make deleting shims a configurable option in v1.0. You can test the beta here: choco install choco-cleaner --pre

I have not gone about trying your way of checking the shims but plan to do so before release.

[bcurran3]

BTW: I did like your idea of checking the version of files and doing determinations based on that. Especially because running an .EXE that is not a shim results in really running that program and possibly opening up a window. Unfortunately, there is at least one version of ShimGen out there that does not put 'shim' in the version info. In fact there appears to be two versions of ShimGen 1.0.0, one that adds "Chocolatey Shim: 1.0.0" and one that just adds "1.0.0"... or maybe it's conditional if ShimGen finds a file version in the file, I'm not sure. But this is the results when I get the file version of all the .EXEs in my bin dir:

7z.exe  22.01 - Chocolatey Shim: 1.0.0
7za.exe 22.01 - Chocolatey Shim: 1.0.0
7zFM.exe    22.01 - Chocolatey Shim: 1.0.0
7zG.exe 22.01 - Chocolatey Shim: 1.0.0
AA_v3.exe   3.10 - shim 0.8.1
accesschk.exe   6.15 - Chocolatey Shim: 1.0.0
accesschk64.exe 6.15 - Chocolatey Shim: 1.0.0
AccessEnum.exe  1.35 - Chocolatey Shim: 1.0.0
adb.exe 1.0.0
ADExplorer64.exe    1.52 - Chocolatey Shim: 1.0.0
ADInsight.exe   1.20 - Chocolatey Shim: 1.0.0
ADInsight64.exe 1.20 - Chocolatey Shim: 1.0.0
adrestore.exe   1.2 - Chocolatey Shim: 1.0.0
adrestore64.exe 1.2 - Chocolatey Shim: 1.0.0
adwcleaner.exe  8.4 - Chocolatey Shim: 1.0.0
adwcleaner_8.4.0.exe    8.4 - Chocolatey Shim: 1.0.0
AutoHotkey.exe  2.0.2 - Chocolatey Shim: 1.0.0
Autologon.exe   3.10 - Chocolatey Shim: 1.0.0
Autologon64.exe 3.10 - Chocolatey Shim: 1.0.0
Autoruns.exe    14.09 - Chocolatey Shim: 1.0.0
Autoruns64.exe  14.09 - Chocolatey Shim: 1.0.0
Autoruns64a.exe 14.09 - shim 0.8.1
autorunsc.exe   14.09 - Chocolatey Shim: 1.0.0
autorunsc64.exe 14.09 - Chocolatey Shim: 1.0.0
autorunsc64a.exe    14.09 - shim 0.8.1
BeCyIconGrabber.exe 2.30 - shim 0.8.1
BFG.exe 1.12.5.0 - shim 0.8.1
Bginfo.exe  4.32 - Chocolatey Shim: 1.0.0
Bginfo64.exe    4.32 - Chocolatey Shim: 1.0.0
BRC64.exe   1, 3, 3, 0 - shim 0.8.1
Cacheset.exe    1.02 - Chocolatey Shim: 1.0.0
Cacheset64.exe  1.02 - Chocolatey Shim: 1.0.0
checksum.exe    0.3.1 - Chocolatey Shim: 1.0.0
choco.exe   0.12.1 - Chocolatey Shim: 1.0.0
chocobutler.bat.exe 0.8.1
chocolatey.exe  0.12.1 - Chocolatey Shim: 1.0.0
chocolateygui.exe   1.1.3 - Chocolatey Shim: 1.0.0
chocolateyguicli.exe    1.1.3 - Chocolatey Shim: 1.0.0
cinst.exe   0.12.1 - Chocolatey Shim: 1.0.0
clist.exe   0.12.1 - Chocolatey Shim: 1.0.0
Clockres.exe    2.1 - Chocolatey Shim: 1.0.0
Clockres64.exe  2.1 - Chocolatey Shim: 1.0.0
cmp.exe 2.8.7.1605 - shim 0.8.1
compatibilityupdate.exe 11.00.17134.1 - Chocolatey Shim: 1.0.0
Contig.exe  1.83 - Chocolatey Shim: 1.0.0
Contig64.exe    1.83 - Chocolatey Shim: 1.0.0
Coreinfo.exe    3.6 - Chocolatey Shim: 1.0.0
Coreinfo64.exe  3.6 - Chocolatey Shim: 1.0.0
cports.exe  2.71 - Chocolatey Shim: 1.0.0
cpush.exe   0.12.1 - Chocolatey Shim: 1.0.0
CPUSTRES.EXE    1.0.0
CPUSTRES64.EXE  1.0.0
ctrl2cap.exe    1.0.0
cuninst.exe 0.12.1 - Chocolatey Shim: 1.0.0
cup.exe 0.12.1 - Chocolatey Shim: 1.0.0
curl.exe    8.0.1 - Chocolatey Shim: 1.0.0
Dbgview.exe 4.90 - Chocolatey Shim: 1.0.0
dbgview64.exe   4.90 - Chocolatey Shim: 1.0.0
depends.exe 2.2.6000 - shim 0.8.1
DesktopOK_x64.exe   10, 8, 1, 0 - Chocolatey Shim: 1.0.0
Desktops.exe    2.01 - Chocolatey Shim: 1.0.0
Desktops64.exe  2.01 - Chocolatey Shim: 1.0.0
diff.exe    2.8.7.1605 - shim 0.8.1
diff3.exe   2.8.7.1605 - shim 0.8.1
disk2vhd.exe    2.02 - Chocolatey Shim: 1.0.0
disk2vhd64.exe  2.02 - Chocolatey Shim: 1.0.0
disk2vhd64a.exe 2.02 - Chocolatey Shim: 1.0.0
diskext.exe 1.2 - Chocolatey Shim: 1.0.0
diskext64.exe   1.2 - Chocolatey Shim: 1.0.0
DiskInfo32.exe  8.17.14.0 - Chocolatey Shim: 1.0.0
DiskInfo64.exe  8.17.14.0 - Chocolatey Shim: 1.0.0
DiskInfoA32.exe 8.14.2.0 - Chocolatey Shim: 1.0.0
DiskInfoA64.exe 8.17.14.0 - Chocolatey Shim: 1.0.0
Diskmon.exe 2.02 - Chocolatey Shim: 1.0.0
Diskmon64.exe   2.02 - Chocolatey Shim: 1.0.0
DiskView.exe    2.41 - Chocolatey Shim: 1.0.0
DiskView64.exe  2.41 - Chocolatey Shim: 1.0.0
du.exe  1.62 - Chocolatey Shim: 1.0.0
du64.exe    1.62 - Chocolatey Shim: 1.0.0
efsdump.exe 1.03 - Chocolatey Shim: 1.0.0
egrep.exe   0.8.1
exiftool.exe    12.6.2.0 - Chocolatey Shim: 1.0.0
ExpanDrive_Setup_2021.8.3.exe   2021.8.3.0 - shim 0.8.1
fastboot.exe    1.0.0
ffmpeg.exe  1.0.0
ffplay.exe  1.0.0
ffprobe.exe 1.0.0
fgrep.exe   0.8.1
FindLinks.exe   1.1 - Chocolatey Shim: 1.0.0
FindLinks64.exe 1.1 - Chocolatey Shim: 1.0.0
FlicFlac.exe    1.1.22.09 - shim 0.8.1
ForcePS.exe 1.0.0.1 - shim 0.8.1
gotop.exe   0.8.1
gping.exe   1.0.0
GPU-Z.2.53.0.exe    2.53.0.0 - Chocolatey Shim: 1.0.0
grep.exe    0.8.1
gs.exe  0.8.1
handle.exe  5.0 - Chocolatey Shim: 1.0.0
handle64.exe    5.0 - Chocolatey Shim: 1.0.0
hex2dec.exe 1.1 - Chocolatey Shim: 1.0.0
hex2dec64.exe   1.1 - Chocolatey Shim: 1.0.0
innoextract.exe 0.8.1
innounp.exe 0.50 - shim 0.8.1
InstChoco.exe   2.11.0.0 - shim 0.8.1
jpegoptim.exe   1.0.0
junction.exe    1.07 - Chocolatey Shim: 1.0.0
junction64.exe  1.07 - Chocolatey Shim: 1.0.0
kindlegen.exe   0.8.1
ldmdump.exe 1.0.0
lessmsi-gui.exe 1.10.0 - shim 0.8.1
lessmsi.exe 1.10.0 - shim 0.8.1
Listdlls.exe    3.2 - Chocolatey Shim: 1.0.0
Listdlls64.exe  3.2 - Chocolatey Shim: 1.0.0
livekd.exe  5.63 - Chocolatey Shim: 1.0.0
livekd64.exe    5.63 - Chocolatey Shim: 1.0.0
LoadOrd.exe 1.02 - Chocolatey Shim: 1.0.0
LoadOrd64.exe   1.02 - Chocolatey Shim: 1.0.0
LoadOrdC.exe    1.02 - Chocolatey Shim: 1.0.0
LoadOrdC64.exe  1.02 - Chocolatey Shim: 1.0.0
Log4jScanner.exe    2.1.3.0 - shim 0.8.1
logonsessions.exe   1.41 - Chocolatey Shim: 1.0.0
logonsessions64.exe 1.41 - Chocolatey Shim: 1.0.0
Marker.exe  0.0.6.0 - shim 0.8.1
MediaCreationTool.exe   10.0.19041.572 - shim 0.8.1
MediaInfo.exe   23.04.0.0 - Chocolatey Shim: 1.0.0
mencoder.exe    Redxii-SVN-r37904-4.9.3 (i686) - shim 0.8.1
movdump.exe 0.8.1
movefile.exe    1.02 - Chocolatey Shim: 1.0.0
movefile64.exe  1.02 - Chocolatey Shim: 1.0.0
mplayer.exe Redxii-SVN-r37904-4.9.3 (i686) - shim 0.8.1
nano.exe    1.0.0
NetstatViewer.exe   0.8.1
nirlauncher.exe 1.30 - Chocolatey Shim: 1.0.0
notepad++.exe   8.53 - Chocolatey Shim: 1.0.0
notmyfault.exe  4.21 - Chocolatey Shim: 1.0.0
notmyfault64.exe    4.21 - Chocolatey Shim: 1.0.0
notmyfaultc.exe 4.21 - Chocolatey Shim: 1.0.0
notmyfaultc64.exe   4.21 - Chocolatey Shim: 1.0.0
ntfsinfo.exe    1.2 - Chocolatey Shim: 1.0.0
ntfsinfo64.exe  1.2 - Chocolatey Shim: 1.0.0
nuget.exe   6.5.0+069970c727b254636c1ad29c5a7a767081482a9a.069970c727b254636c1ad29c5a7a767081482a9a - Chocolatey Shim: 1.0.0
osfmount.exe    3.1.1001.0                                         - Chocolatey Shim: 1.0.0
OutlookAddressBookView.exe  2.33 - shim 0.8.1
PAGEANT.EXE 1.0.0
pendmoves.exe   1.3 - Chocolatey Shim: 1.0.0
pendmoves64.exe 1.3 - Chocolatey Shim: 1.0.0
pipelist.exe    1.02 - Chocolatey Shim: 1.0.0
pipelist64.exe  1.02 - Chocolatey Shim: 1.0.0
PlatformTags32W.exe 0.8.1
PlatformTags64W.exe 0.8.1
PLINK.EXE   1.0.0
PngOptimizer.exe    0.8.1
PngOptimizerCL.exe  0.8.1
portmon.exe 3.03 - Chocolatey Shim: 1.0.0
PreviousFilesRecovery.exe   1.10 - shim 0.8.1
procdump.exe    11.0 - Chocolatey Shim: 1.0.0
procdump64.exe  11.0 - Chocolatey Shim: 1.0.0
procexp.exe 17.04 - Chocolatey Shim: 1.0.0
procexp64.exe   17.04 - Chocolatey Shim: 1.0.0
Procmon.exe 3.93 - Chocolatey Shim: 1.0.0
Procmon64.exe   3.93 - Chocolatey Shim: 1.0.0
PSCP.EXE    1.0.0
PsExec.exe  2.43 - Chocolatey Shim: 1.0.0
PsExec64.exe    2.43 - Chocolatey Shim: 1.0.0
psfile.exe  1.04 - Chocolatey Shim: 1.0.0
psfile64.exe    1.04 - Chocolatey Shim: 1.0.0
PSFTP.EXE   1.0.0
PsGetsid.exe    1.46 - Chocolatey Shim: 1.0.0
PsGetsid64.exe  1.46 - Chocolatey Shim: 1.0.0
PsInfo.exe  1.79 - Chocolatey Shim: 1.0.0
PsInfo64.exe    1.79 - Chocolatey Shim: 1.0.0
pskill.exe  1.17 - Chocolatey Shim: 1.0.0
pskill64.exe    1.17 - Chocolatey Shim: 1.0.0
pslist.exe  1.41 - Chocolatey Shim: 1.0.0
pslist64.exe    1.41 - Chocolatey Shim: 1.0.0
PsLoggedon.exe  1.35 - Chocolatey Shim: 1.0.0
PsLoggedon64.exe    1.35 - Chocolatey Shim: 1.0.0
psloglist.exe   2.82 - Chocolatey Shim: 1.0.0
psloglist64.exe 2.82 - Chocolatey Shim: 1.0.0
pspasswd.exe    1.25 - Chocolatey Shim: 1.0.0
pspasswd64.exe  1.25 - Chocolatey Shim: 1.0.0
psping.exe  2.12 - Chocolatey Shim: 1.0.0
psping64.exe    2.12 - Chocolatey Shim: 1.0.0
PsService.exe   2.26 - Chocolatey Shim: 1.0.0
PsService64.exe 2.26 - Chocolatey Shim: 1.0.0
psshutdown.exe  2.6 - Chocolatey Shim: 1.0.0
psshutdown64.exe    2.6 - Chocolatey Shim: 1.0.0
pssuspend.exe   1.08 - Chocolatey Shim: 1.0.0
pssuspend64.exe 1.08 - Chocolatey Shim: 1.0.0
ptime.exe   0.8.1
PUTTY.EXE   1.0.0
PUTTYGEN.EXE    1.0.0
python3.11.exe  3.11.3 - Chocolatey Shim: 1.0.0
qdir.exe    11,2,6,0 - Chocolatey Shim: 1.0.0
QOwnNotes.exe   1.0.0
RAIDar.exe  6, 5, 0, 0 - shim 0.8.1
RAMMap.exe  1.61 - Chocolatey Shim: 1.0.0
RAMMap64.exe    1.61 - Chocolatey Shim: 1.0.0
RDCMan.exe  2.92.1430.0 - Chocolatey Shim: 1.0.0
RegDelNull.exe  1.11 - Chocolatey Shim: 1.0.0
RegDelNull64.exe    1.11 - Chocolatey Shim: 1.0.0
regjump.exe 1.11 - Chocolatey Shim: 1.0.0
Regshot-x64-ANSI.exe    1.9.0 - shim 0.8.1
Regshot-x64-Unicode.exe 1.9.0 - shim 0.8.1
Regshot-x86-ANSI.exe    1.9.0 - shim 0.8.1
Regshot-x86-Unicode.exe 1.9.0 - shim 0.8.1
ru.exe  1.2 - Chocolatey Shim: 1.0.0
ru64.exe    1.2 - Chocolatey Shim: 1.0.0
rufus.exe   4.0.2035 - Chocolatey Shim: 1.0.0
sdelete.exe 2.04 - Chocolatey Shim: 1.0.0
sdelete64.exe   2.04 - Chocolatey Shim: 1.0.0
sdiff.exe   2.8.7.1605 - shim 0.8.1
SetACL.exe  3.1.2.86 - Chocolatey Shim: 1.0.0
ShareEnum.exe   1.61 - Chocolatey Shim: 1.0.0
ShareEnum64.exe 1.61 - Chocolatey Shim: 1.0.0
ShellRunas.exe  1.02 - Chocolatey Shim: 1.0.0
sigcheck.exe    2.90 - Chocolatey Shim: 1.0.0
sigcheck64.exe  2.90 - Chocolatey Shim: 1.0.0
SIV32A.exe  V5.63 - Chocolatey Shim: 1.0.0
StartupList.exe 2.02 - shim 0.8.1
streams.exe 1.60 - Chocolatey Shim: 1.0.0
streams64.exe   1.60 - Chocolatey Shim: 1.0.0
strings.exe 2.54 - Chocolatey Shim: 1.0.0
strings64.exe   2.54 - Chocolatey Shim: 1.0.0
Sudo.exe    1.0.0
sync.exe    2.2 - Chocolatey Shim: 1.0.0
sync64.exe  2.2 - Chocolatey Shim: 1.0.0
Sysmon.exe  14.16 - Chocolatey Shim: 1.0.0
Sysmon64.exe    14.16 - Chocolatey Shim: 1.0.0
tageditor.exe   3.7.5 - Chocolatey Shim: 1.0.0
TaskSchedulerView.exe   1.72 - Chocolatey Shim: 1.0.0
tcpvcon.exe 4.19 - Chocolatey Shim: 1.0.0
tcpvcon64.exe   4.19 - Chocolatey Shim: 1.0.0
tcpview.exe 4.19 - Chocolatey Shim: 1.0.0
tcpview64.exe   4.19 - Chocolatey Shim: 1.0.0
Testlimit.exe   5.24 - Chocolatey Shim: 1.0.0
Testlimit64.exe 5.24 - Chocolatey Shim: 1.0.0
tgpt.exe    1.0.0
ts.exe  1, 0, 0, 1 - shim 0.8.1
ussf.exe    0.8.1
vfw2menc.exe    0.8.1
vmmap.exe   3.32 - Chocolatey Shim: 1.0.0
vmmap64.exe 3.32 - Chocolatey Shim: 1.0.0
Volumeid.exe    2.1 - Chocolatey Shim: 1.0.0
Volumeid64.exe  2.1 - Chocolatey Shim: 1.0.0
WakeMeOnLan.exe 1.90 - Chocolatey Shim: 1.0.0
wget.exe    0.8.1
whois.exe   1.21 - Chocolatey Shim: 1.0.0
whois64.exe 1.21 - Chocolatey Shim: 1.0.0
WinCrashReport.exe  1.25 - shim 0.8.1
Winobj.exe  3.14 - Chocolatey Shim: 1.0.0
Winobj64.exe    3.14 - Chocolatey Shim: 1.0.0
yumi.exe    0.8.1
ZoomIt.exe  6.12 - Chocolatey Shim: 1.0.0
ZoomIt64.exe    6.12 - Chocolatey Shim: 1.0.0

So there's some weirdness/inconsistencies created by ShimGen that makes this unfortunately unreliable.

I completely understand the desire to have a directory on the path to store misc small utilities to access easily on demand. I've been doing that for years as well. I just can't recommend dropping these in Chocolatey's bin dir. I've noticed during recent upgrades of Chocolatey a "regenerating shims" message. I don't know what Chocolatey is actually doing to the .EXEs to regenerate them, but it's possible that it might also harm your utilities put there. The Chocolatey Team got protective of the bin folder about two years ago and forbade me, in my scripts, doing anything in there. So I would warn it's best practice to not store anything not Chocolatey native there.

With that said $env:ChocolateyToolsLocation (default c:\tools) is a pretty safe location to drop these misc. utilities into assuming that package maintainers that use it are careful about removing their package items from the dir in thier uninstall scripts.

Proof of concept is simple as you're using Choco-Cleaner and it's stored there. :-)