search

PowerShell / PSResourceGet

PSResourceGet is the package manager for PowerShell
https://www.powershellgallery.com/packages/Microsoft.PowerShell.PSResourceGet
MIT License
475 stars 90 forks source link

"The process cannot access the file" during `Save-PSResource` #1662

Open o-l-a-v opened 4 weeks ago

o-l-a-v commented 4 weeks ago

Prerequisites

Steps to reproduce

# Assets
$ModuleName = [string] 'AWSPowerShell.NetCore'
$SavePath   = [string] [System.IO.Path]::Combine(
    $env:LOCALAPPDATA,
    'Microsoft',
    'PowerShell',
    'Modules'
)

# Create $SavePath if it does not already exist
if (-not [System.IO.Directory]::Exists($SavePath)) {
    $null = [System.IO.Directory]::CreateDirectory($SavePath)
}

# Save module
Save-PSResource -Repository 'PSGallery' -TrustRepository -IncludeXml -SkipDependencyCheck -Path $SavePath -Name $ModuleName

Lately PSResourceGet has had problems installing AWSPowerShell.NetCore on my system. I use Save-PSResource because I want modules in user context, but not synced with OneDrive (Known Folder Move).

It seems that the installation succeeds, as in it managed to move all files to -Path. But then it fails at deleting the temp directory used for downloading and extracting the files.

If a service like an antimalware scan is the reason these files can't be deleted: Detect it and wait for it to finish? Or try twice, wait five seconds before second attempt?

Expected behavior

PSResourceGet should succeed in deleting temp path.

Actual behavior

PSResourceGet fails to delete temp path.

Error details

PS C:\Users\<redacted>> Save-PSResource -Repository 'PSGallery' -TrustRepository -IncludeXml -SkipDependencyCheck -Path 'C:\Users\<redacted>\AppData\Local\Microsoft\PowerShell\Modules' -Name 'awspowershell.netcore'
Save-PSResource: Unable to successfully install package 'AWSPowerShell.NetCore': 'The process cannot access the file 'C:\Users\<redacted>\AppData\Local\Temp\9ae5cbb5-18d0-4daf-ae7d-fd4abed1818d\awspowershell.netcore\4.1.597\AWSPowerShell.TypeExtensions.ps1xml' because it is being used by another process.'
Save-PSResource: Operation is not valid due to the current state of the object.
Save-PSResource: The process cannot access the file 'C:\Users\<redacted>\AppData\Local\Temp\9ae5cbb5-18d0-4daf-ae7d-fd4abed1818d\awspowershell.netcore\4.1.597\AWSSDK.AWSMarketplaceMetering.dll' because it is being used by another process.
Save-PSResource: Package(s) 'awspowershell.netcore' could not be installed from repository 'PSGallery'.
PS C:\Users\<redacted>> Save-PSResource -Repository 'PSGallery' -TrustRepository -IncludeXml -SkipDependencyCheck -Path 'C:\Users\<redacted>\AppData\Local\Microsoft\PowerShell\Modules' -Name 'awspowershell.netcore' -Verbose
VERBOSE: Attempting to search for packages in 'PSGallery'
VERBOSE: Performing the operation "Save-PSResource" on target "Package to save: 'AWSPowerShell.NetCore', version: '4.1.597'".
VERBOSE: Performing the operation "Save-PSResource" on target "Package to install: 'AWSPowerShell.NetCore', version: '4.1.597'".
VERBOSE: Performing the operation "Save-PSResource" on target "Exit ShouldProcess".
VERBOSE: Installation source path is: 'C:\Users\<redacted>\AppData\Local\Temp\101cb6bc-d191-4747-a56f-4304b1bbe1eb\awspowershell.netcore\4.1.597'
VERBOSE: Installation destination path is: 'C:\Users\<redacted>\AppData\Local\Microsoft\PowerShell\Modules\AWSPowerShell.NetCore\4.1.597'
VERBOSE: Temporary module version directory is: 'C:\Users\<redacted>\AppData\Local\Temp\101cb6bc-d191-4747-a56f-4304b1bbe1eb\awspowershell.netcore\4.1.597'
VERBOSE: Attempting to delete with restore on failure.'C:\Users\<redacted>\AppData\Local\Microsoft\PowerShell\Modules\AWSPowerShell.NetCore\4.1.597'
VERBOSE: Attempting to move 'C:\Users\<redacted>\AppData\Local\Temp\101cb6bc-d191-4747-a56f-4304b1bbe1eb\awspowershell.netcore\4.1.597' to 'C:\Users\<redacted>\AppData\Local\Microsoft\PowerShell\Modules\AWSPowerShell.NetCore\4.1.597'
Save-PSResource: Unable to successfully install package 'AWSPowerShell.NetCore': 'The process cannot access the file 'C:\Users\<redacted>\AppData\Local\Temp\101cb6bc-d191-4747-a56f-4304b1bbe1eb\awspowershell.netcore\4.1.597\AWSPowerShellLegacyAliases.psm1' because it is being used by another process.'
Save-PSResource: Operation is not valid due to the current state of the object.
VERBOSE: Attempting to delete 'C:\Users\<redacted>\AppData\Local\Temp\101cb6bc-d191-4747-a56f-4304b1bbe1eb'
Save-PSResource: The process cannot access the file 'C:\Users\<redacted>\AppData\Local\Temp\101cb6bc-d191-4747-a56f-4304b1bbe1eb\awspowershell.netcore\4.1.597\AWSSDK.AppConfig.dll' because it is being used by another process.
Save-PSResource: Package(s) 'awspowershell.netcore' could not be installed from repository 'PSGallery'.
PS C:\Users\<redacted>> Save-PSResource -Repository 'PSGallery' -TrustRepository -IncludeXml -SkipDependencyCheck -Path 'C:\Users\<redacted>\AppData\Local\Microsoft\PowerShell\Modules' -Name 'awspowershell.netcore' -Verbose -Debug
DEBUG: In SavePSResource::ProcessSaveHelper()
DEBUG: In InstallHelper::BeginInstallPackages()
DEBUG: Parameters passed in >>> Name: 'awspowershell.netcore'; VersionRange: ''; NuGetVersion: ''; VersionType: 'NoVersion'; Version: ''; Prerelease: 'False'; Repository: 'PSGallery'; AcceptLicense: 'True'; Quiet: 'False'; Reinstall: 'True'; TrustRepository: 'True'; NoClobber: 'False'; AsNupkg: 'False'; IncludeXml 'True'; SavePackage 'True'; TemporaryPath ''; SkipDependencyCheck: 'True'; AuthenticodeCheck: 'False'; PathsToInstallPkg: 'C:\Users\<redacted>\AppData\Local\Microsoft\PowerShell\Modules'; Scope ''
DEBUG: In InstallHelper::ProcessRepositories()
VERBOSE: Attempting to search for packages in 'PSGallery'
DEBUG: In InstallHelper::InstallPackages()
DEBUG: In InstallHelper::InstallPackage()
DEBUG: In V2ServerAPICalls::FindName()
DEBUG: In V2ServerAPICalls::HttpRequestCall()
DEBUG: Request url is 'https://www.powershellgallery.com/api/v2/FindPackagesById()?$filter=Id+eq+%27awspowershell.netcore%27+and+IsLatestVersion&$inlinecount=allpages&id=%27awspowershell.netcore%27'

Confirm
Are you sure you want to perform this action?
Performing the operation "Save-PSResource" on target "Package to save: 'AWSPowerShell.NetCore', version: '4.1.597'".
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "Y"): a
DEBUG: In V2ServerAPICalls::InstallVersion()
DEBUG: In V2ServerAPICalls::HttpRequestCallForContent()
DEBUG: Request url is 'https://www.powershellgallery.com/api/v2/package/AWSPowerShell.NetCore/4.1.597'
DEBUG: In InstallHelper::TryInstallToTempPath()
DEBUG: In InstallHelper::DeleteExtraneousFiles()
DEBUG: Deleting 'C:\Users\<redacted>\AppData\Local\Temp\757b37dd-9480-4a6d-938c-9e702e642d3d\awspowershell.netcore\4.1.597\AWSPowerShell.NetCore.nuspec'
DEBUG: Deleting 'C:\Users\<redacted>\AppData\Local\Temp\757b37dd-9480-4a6d-938c-9e702e642d3d\awspowershell.netcore\4.1.597\[Content_Types].xml'
DEBUG: Deleting 'C:\Users\<redacted>\AppData\Local\Temp\757b37dd-9480-4a6d-938c-9e702e642d3d\awspowershell.netcore\4.1.597\_rels'
DEBUG: Deleting 'C:\Users\<redacted>\AppData\Local\Temp\757b37dd-9480-4a6d-938c-9e702e642d3d\awspowershell.netcore\4.1.597\package'
DEBUG: In InstallHelper::CreateMetadataXMLFile()
DEBUG: In InstallHelper::TryMoveInstallContent()
DEBUG: In InstallHelper::MoveFilesIntoInstallPath()
VERBOSE: Installation source path is: 'C:\Users\<redacted>\AppData\Local\Temp\757b37dd-9480-4a6d-938c-9e702e642d3d\awspowershell.netcore\4.1.597'
VERBOSE: Installation destination path is: 'C:\Users\<redacted>\AppData\Local\Microsoft\PowerShell\Modules\AWSPowerShell.NetCore\4.1.597'
VERBOSE: Temporary module version directory is: 'C:\Users\<redacted>\AppData\Local\Temp\757b37dd-9480-4a6d-938c-9e702e642d3d\awspowershell.netcore\4.1.597'
VERBOSE: Attempting to delete with restore on failure.'C:\Users\<redacted>\AppData\Local\Microsoft\PowerShell\Modules\AWSPowerShell.NetCore\4.1.597'
VERBOSE: Attempting to move 'C:\Users\<redacted>\AppData\Local\Temp\757b37dd-9480-4a6d-938c-9e702e642d3d\awspowershell.netcore\4.1.597' to 'C:\Users\<redacted>\AppData\Local\Microsoft\PowerShell\Modules\AWSPowerShell.NetCore\4.1.597'
Save-PSResource: Unable to successfully install package 'AWSPowerShell.NetCore': 'The process cannot access the file 'C:\Users\<redacted>\AppData\Local\Temp\757b37dd-9480-4a6d-938c-9e702e642d3d\awspowershell.netcore\4.1.597\AWSPowerShell.TypeExtensions.ps1xml' because it is being used by another process.'
Save-PSResource: Operation is not valid due to the current state of the object.
VERBOSE: Attempting to delete 'C:\Users\<redacted>\AppData\Local\Temp\757b37dd-9480-4a6d-938c-9e702e642d3d'
Save-PSResource: The process cannot access the file 'C:\Users\<redacted>\AppData\Local\Temp\757b37dd-9480-4a6d-938c-9e702e642d3d\awspowershell.netcore\4.1.597\AWSSDK.ARCZonalShift.dll' because it is being used by another process.
Save-PSResource: Package(s) 'awspowershell.netcore' could not be installed from repository 'PSGallery'.
PS C:\Users\<redacted>> Get-Module

ModuleType Version    PreRelease Name                                ExportedCommands
---------- -------    ---------- ----                                ----------------
Manifest   7.0.0.0               Microsoft.PowerShell.Management     {Add-Content, Clear-Content, Clear-Item, Clear-ItemProperty…}
Binary     1.0.5                 Microsoft.PowerShell.PSResourceGet  {Find-PSResource, Get-InstalledPSResource, Get-PSResourceRepository, Get-PSScriptFileInfo…}
Manifest   7.0.0.0               Microsoft.PowerShell.Utility        {Add-Member, Add-Type, Clear-Variable, Compare-Object…}
Script     2.3.5                 PSReadLine                          {Get-PSReadLineKeyHandler, Get-PSReadLineOption, Remove-PSReadLineKeyHandler, Set-PSReadLineKeyHandler…}

PS C:\Users\<redacted>>

Environment data

Visuals

No response

o-l-a-v commented 4 weeks ago

From the logs it's hard to say whether module was actually successfully installed. This is another case where feature request #1646 would be useful. 😊

o-l-a-v commented 4 weeks ago

Just tested on my personal computer which worked flawlessly.

This only fails on my work computer were we have CrowdStrike Falcon sensor instead of Windows built in antimalware. I suspect it might have something to do with it, but I don't know for sure.

evelyn-bi commented 3 weeks ago

You could try running handle.exe as administrator to spit out all the open handles and ctrl+f to find what is holding onto the temp file, as long as the handle is still held when you run it then you can see if it's CrowdStrike or powershell or some other thing. You can also use process explorer as administrator and go to "Find -> Find Handle or DLL" to search for which process is holding onto the file or folder.

o-l-a-v commented 3 weeks ago

The time window for when the files are not deletable is so brief, I could not get handle.exe to show anything.

Edit1: What I can see in Task Manager -> Details sorted by CPU usage, is that MsSense.exe (Windows Defender Advanced Threat Protection Service Executable), SenseNdr.exe, CSFalconService.exe and CSFalconContainer.exe all start working.

Edit 2: With Process Explorer, selecting the MsSense.exe process, then the Handles tab, I can see that it adds handles to DLLs in temp. Which are still open when Save-PSResource fails to delete the temp files. Interesting.

Edit 3: Same with CSFalconService.exe ^.

Edit 4: This seems to be the code for deleting a directory. Maybe it can be made more robust in regards to handles from antimalware processes?

https://github.com/PowerShell/PSResourceGet/blob/48d16c0fac61f29a15e2327f462e2eec315822d7/src/code/Utils.cs#L1531-L1555

SydneyhSmith commented 3 weeks ago

I wonder if it would be best to not fail if the temp directory can't be installed, just a verbose message

o-l-a-v commented 3 weeks ago

It seems that PSResourceGet currently does what people in this stackoverflow thread refers to as "nonsense":

Seems many agree that trying multiple times before failing with a small delay is a good idea:

o-l-a-v commented 1 week ago

I tried using Dev Drive as -TemporaryPath D:\Temp for animalware "performance mode" ( https://learn.microsoft.com/en-us/defender-endpoint/microsoft-defender-endpoint-antivirus-performance-mode ), which seems to work fine. But if installing the module twice I get a strange error message:

# Save package with success using Dev Drive as temp
PS > Save-PSResource -Repository 'PSGallery' -TrustRepository -IncludeXml -SkipDependencyCheck -Path $SavePath -Name $ModuleName -TemporaryPath 'D:\Temp'
VERBOSE: Attempting to search for packages in 'PSGallery'
VERBOSE: Installation source path is: 'D:\Temp\685afbd2-002c-4c9d-acce-c5ff2dda6c2f\awspowershell.netcore\4.1.612'
VERBOSE: Installation destination path is: 'C:\Users\olav.birkeland\AppData\Local\Microsoft\PowerShell\Modules\AWSPowerShell.NetCore\4.1.612'
VERBOSE: Temporary module version directory is: 'D:\Temp\685afbd2-002c-4c9d-acce-c5ff2dda6c2f\awspowershell.netcore\4.1.612'
VERBOSE: Attempting to move 'D:\Temp\685afbd2-002c-4c9d-acce-c5ff2dda6c2f\awspowershell.netcore\4.1.612' to 'C:\Users\olav.birkeland\AppData\Local\Microsoft\PowerShell\Modules\AWSPowerShell.NetCore\4.1.612'VERBOSE: Successfully installed package 'AWSPowerShell.NetCore' to location 'C:\Users\olav.birkeland\AppData\Local\Microsoft\PowerShell\Modules'
VERBOSE: Attempting to delete 'D:\Temp\685afbd2-002c-4c9d-acce-c5ff2dda6c2f'
VERBOSE: Successfully deleted 'D:\Temp\685afbd2-002c-4c9d-acce-c5ff2dda6c2f'

# Again, right after last command finished
PS > Save-PSResource -Repository 'PSGallery' -TrustRepository -IncludeXml -SkipDependencyCheck -Path $SavePath -Name $ModuleName -TemporaryPath 'D:\Temp'
VERBOSE: Attempting to search for packages in 'PSGallery'
VERBOSE: Installation source path is: 'D:\Temp\82a3807e-c267-4fed-adde-7ab312444f73\awspowershell.netcore\4.1.612'
VERBOSE: Installation destination path is: 'C:\Users\olav.birkeland\AppData\Local\Microsoft\PowerShell\Modules\AWSPowerShell.NetCore\4.1.612'
VERBOSE: Temporary module version directory is: 'D:\Temp\82a3807e-c267-4fed-adde-7ab312444f73\awspowershell.netcore\4.1.612'
VERBOSE: Attempting to delete with restore on failure.'C:\Users\olav.birkeland\AppData\Local\Microsoft\PowerShell\Modules\AWSPowerShell.NetCore\4.1.612'
Save-PSResource: Unable to successfully install package 'AWSPowerShell.NetCore': 'The process cannot access the file 'C:\Users\olav.birkeland\AppData\Local\Temp\ad0b094f-01e3-4305-be6e-7de0496d382b\AWSSDK.CloudFormation.dll' because it is being used by another process.'
Save-PSResource: Operation is not valid due to the current state of the object.
VERBOSE: Attempting to delete 'D:\Temp\82a3807e-c267-4fed-adde-7ab312444f73'
VERBOSE: Successfully deleted 'D:\Temp\82a3807e-c267-4fed-adde-7ab312444f73'
Save-PSResource: Package(s) 'AWSPowerShell.NetCore' could not be installed from repository 'PSGallery'.
PS >

The directory in the error message does not exist, I've made sure of it. Seems the error throws when deleting the module which will be overriden, it was just installed and antimalware is probably still scanning all the content and placing file handles. But it is not located in %TEMP%.