SHA256 not matching while installing docker

jaydeepch15 commented 7 years ago

I am able to find package

Find-Package -providerName DockerMsftProvider -AllVersions

Name Version Source Summary

Docker 1.12.2-cs2-ws... DockerDefault Contains the CS Docker Engine for use with Windows ...

But when I tried to install I am getting SHA256 mismatch error

PS C:> Install-Package -Name docker -ProviderName DockerMsftProvider -Verbose VERBOSE: Using the provider 'DockerMsftProvider' for searching packages. VERBOSE: Download size: 0MB VERBOSE: Free space on the drive: 40384.86MB VERBOSE: Downloading https://dockermsft.blob.core.windows.net/dockercontainer/DockerMsftIndex.json to C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\DockerDefault_DockerSearchIndex.json VERBOSE: About to download VERBOSE: Finished downloading VERBOSE: Downloaded in 0 hours, 0 minutes, 2 seconds. VERBOSE: Performing the operation "Install Package" on target "Package 'Docker' version '1.12.2-cs2-ws-beta' from 'DockerDefault'.".

The package(s) come(s) from a package source that is not marked as trusted. Are you sure you want to install software from 'DockerDefault'? [Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "N"): A VERBOSE: Containers feature is already installed. Skipping the install. VERBOSE: Verifying Hash of the downloaded file. WARNING: Cannot verify the file SHA256. Deleting the file. VERBOSE: Hash verified! WARNING: C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip does not exist Install-Package : Cannot find path 'C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip' because it does not exist. At line:1 char:1

Install-Package -Name docker -ProviderName DockerMsftProvider -Verbos ...


+ CategoryInfo          : ObjectNotFound: (C:\Users\ADMINI...cs2-ws-beta.zip:String) [Install-Package], Exception
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand,Microsoft.PowerShell.Packag
eManagement.Cmdlets.InstallPackage

What to Do?

jayshah-msft commented 7 years ago

@jaydeepjc, Are you using FIPS enabled machine?

jaydeepch15 commented 7 years ago

@jayshah-msft No, I am not using FIPS enabled machine, Its simple Windows server 2016 Evalution (Desktop Experience)

jayshah-msft commented 7 years ago

@nampdn Can you please run the following commands and provide the output:

Start-BitsTransfer -Source https://dockermsft.blob.core.windows.net/dockercontainer/docker-1-12-2-cs2-ws-beta.zip -Destination /docker.zip

Get-FileHash -Path /docker.zip -Algorithm SHA256

Please provide the output.

Thanks!

jayshah-msft commented 7 years ago

I see that your SHA256 is as expected and I am glad to see that your install works fine. I would still like to know why your first attempt said SHA did not match. I will investigate on my end. Let me know if you face any further issues.

Cheers, Jay!

jayshah-msft commented 7 years ago

@jaydeepjc, Can you please try to download the file and manually check the SHA?

$destinationFolder = "Path to the destination folder" Start-BitsTransfer -Source https://dockermsft.blob.core.windows.net/dockercontainer/docker-1-12-2-cs2-ws-beta.zip -Destination $destinationFolder/docker.zip Get-FileHash -Path $destinationFolder/docker.zip -Algorithm SHA256

Thanks for your help.

jaydeepch15 commented 7 years ago

@jayshah-msft

I can Download and SHA is exactly the same which is in JSON file

    PS C:\> Start-BitsTransfer -Source https://dockermsft.blob.core.windows.net/dockercontainer/docker-1-12-2-cs2-ws-beta.zip -Destination $destinationFolder/docker.zip
   PS C:\> Get-FileHash -Path $destinationFolder/docker.zip -Algorithm SHA256

    Algorithm       Hash                                                                   Path
    ---------       ----                                                                   ----
    SHA256          FEFEB0CD5566550E9ECAFB87CF26C78AFC8643B2450F2AC0203F6B88BE15B68D        C:\test\docker.zip

However when tried to install, issue still persists

    PS C:\> Install-Package -Name docker -ProviderName DockerMsftProvider

    The package(s) come(s) from a package source that is not marked as trusted.
    Are you sure you want to install software from 'DockerDefault'?
    [Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): A
    WARNING: Cannot verify the file SHA256. Deleting the file.
    WARNING: C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-       ws-beta.zip does not exist
    Install-Package : Cannot find path
    'C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip'    because it does not exist.
    At line:1 char:1
    + Install-Package -Name docker -ProviderName DockerMsftProvider
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (C:\Users\ADMINI...cs2-ws-beta.zip:String) [Install-Package], Exception
+ FullyQualifiedErrorId :    PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand,Microsoft.PowerShell.Packag
       eManagement.Cmdlets.InstallPackage

DmitryKakurin commented 7 years ago

I've had exactly the same problem at first and I've identified the reason: I had a file Get-FileHash.ps1 in my path, and it was called by powershell instead of built-in command. Since my .ps1 was returning hash in a different format, Install-Package was thinking that hash is incorrect. The [obvious] fix was to remove my Get-FileHash.ps1.

jayshah-msft commented 7 years ago

@DmitryKakurin, Thanks for the info. Appreciate your help. @jaydeepjc, do you have a file named Get-FileHash.ps1 in your path?

Cheers, Jay!

jaydeepch15 commented 7 years ago

@jayshah-msft

I don't have Get-FileHash.ps1 file in my path.

jianyunt commented 7 years ago

@jaydeepjc, I recommend you change Get-FileHash to Microsoft.PowerShell.Utility\Get-FileHash in DockerMsftProvider.psm1. Relaunch PowerShell to see how it goes.

jaydeepch15 commented 7 years ago

@jianyunt

I did the changes you suggested, still problem persists. Is there any way to stop verifying SHA256?

jianyunt commented 7 years ago

You can, but not recommended. Can you add something like the following in VerifyHashCheck()/DockerMsftProvider.psm1 line 1651, 1654? Write-Verbose "$hash $Destination $fileHash"

Write-Verbose "$hash $Destination $fileHash $fileSha256"

jaydeepch15 commented 7 years ago

I did added above lines on mentioned line numbers.

Output is as below,

PS C:\Users\Administrator> Install-Package -Name docker -ProviderName DockerMsftProvider -Verbose
VERBOSE: Importing package provider 'DockerMsftProvider'.
VERBOSE: Using the provider 'DockerMsftProvider' for searching packages.
VERBOSE: Download size: 0MB
VERBOSE: Free space on the drive: 37105.95MB
VERBOSE: Downloading     https://dockermsft.blob.core.windows.net/dockercontainer/DockerMsftIndex.json to
 C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\DockerDefault_DockerSearchIndex.json
VERBOSE: About to download
VERBOSE: Finished downloading
VERBOSE: Downloaded in 0 hours, 0 minutes, 2 seconds.
VERBOSE: Performing the operation "Install Package" on target "Package 'Docker' version '1.12.2-cs2-    ws-beta' from
'DockerDefault'.".

The package(s) come(s) from a package source that is not marked as trusted.
Are you sure you want to install software from 'DockerDefault'?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): A
VERBOSE: Containers feature is already installed. Skipping the install.
VERBOSE: Verifying Hash of the downloaded file.
VERBOSE: FEFEB0CD5566550E9ECAFB87CF26C78AFC8643B2450F2AC0203F6B88BE15B68D
C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip
WARNING: Cannot verify the file SHA256. Deleting the file.
VERBOSE: Hash verified!
WARNING: C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-    beta.zip does not exist
Install-Package : Cannot find path
'C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip'     because it does not exist.
 At line:1 char:1
+ Install-Package -Name docker -ProviderName DockerMsftProvider -Verbos ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (C:\Users\ADMINI...cs2-ws-beta.zip:String) [Install-Package], Exception
+ FullyQualifiedErrorId :       PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand,Microsoft.PowerShell.Packag
   eManagement.Cmdlets.InstallPackage

jianyunt commented 7 years ago

Please try adding these and run again. Thanks! Write-Verbose "hash=$hash" Write-Verbose "Destination =$Destination" Write-Verbose "fileHash =$fileHash" Write-Verbose "fileSha256 =$fileSha256"

jaydeepch15 commented 7 years ago

Code looks like below, is it correct one which I have written?

    $fileHash = Get-FileHash -Path $Destination `
                            -Algorithm SHA256
    Write-Verbose "hash=$hash"
    Write-Verbose "Destination =$Destination"
    Write-Verbose "fileHash =$fileHash"

if($fileHash.Psobject.properties.name -Contains "Hash")
{
    $fileSha256 = $fileHash.Hash
    Write-Verbose "fileSha256 =$fileSha256"
}

OUTPUT is as below

 PS C:\Users\Administrator> Install-Package -Name docker -ProviderName DockerMsftProvider -Verbose
 VERBOSE: Importing package provider 'DockerMsftProvider'.
 VERBOSE: Using the provider 'DockerMsftProvider' for searching packages.
 VERBOSE: Download size: 0MB
 VERBOSE: Free space on the drive: 37091.31MB
 VERBOSE: Downloading https://dockermsft.blob.core.windows.net/dockercontainer/DockerMsftIndex.json to
 C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\DockerDefault_DockerSearchIndex.json
 VERBOSE: About to download
 VERBOSE: Finished downloading
 VERBOSE: Downloaded in 0 hours, 0 minutes, 2 seconds.
 VERBOSE: Performing the operation "Install Package" on target "Package 'Docker' version '1.12.2-cs2-ws-beta' from
 'DockerDefault'.".

 The package(s) come(s) from a package source that is not marked as trusted.
 Are you sure you want to install software from 'DockerDefault'?
 [Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): A
 VERBOSE: Containers feature is already installed. Skipping the install.
 VERBOSE: Verifying Hash of the downloaded file.
 VERBOSE: hash=FEFEB0CD5566550E9ECAFB87CF26C78AFC8643B2450F2AC0203F6B88BE15B68D
 VERBOSE: Destination =C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip
 VERBOSE: fileHash =
 WARNING: Cannot verify the file SHA256. Deleting the file.
 VERBOSE: Hash verified!
 WARNING: C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip does not exist
 Install-Package : Cannot find path
 'C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip' because it does not exist.
 At line:1 char:1
 + Install-Package -Name docker -ProviderName DockerMsftProvider -Verbos ...
 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 + CategoryInfo          : ObjectNotFound: (C:\Users\ADMINI...cs2-ws-beta.zip:String) [Install-Package], Exception
 + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand,Microsoft.PowerShell.Packag
 eManagement.Cmdlets.InstallPackage

jianyunt commented 7 years ago

fileHash is null, meaning Get-FileHash returns null. That;s the reason hash does not match.

can you please add the following?

if(-not (Test-Path -Path $Destination))
{
    Write-Verbose  "File $Destination does not exist!!!"
}

$fileHash = Get-FileHash -Path $Destination `
                    -Algorithm SHA256

if (-not $fileHash)
{
    Write-Verbose  "fileHash with path is null"
}

$fileHash = Get-FileHash -LiteralPath $Destination `
                    -Algorithm SHA256

if (-not $fileHash)
{
    Write-Verbose  "fileHash with LiteralPath is null"
}

jaydeepch15 commented 7 years ago

Below is output

PS C:\Users\Administrator> Install-Package -Name docker -ProviderName DockerMsftProvider -Verbose
VERBOSE: Importing package provider 'DockerMsftProvider'.
VERBOSE: Using the provider 'DockerMsftProvider' for searching packages.
VERBOSE: Download size: 0MB
VERBOSE: Free space on the drive: 37045.36MB
VERBOSE: Downloading https://dockermsft.blob.core.windows.net/dockercontainer/DockerMsftIndex.json to
C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\DockerDefault_DockerSearchIndex.json
VERBOSE: About to download
VERBOSE: Finished downloading
VERBOSE: Downloaded in 0 hours, 0 minutes, 2 seconds.
VERBOSE: Performing the operation "Install Package" on target "Package 'Docker' version '1.12.2-cs2-ws-beta' from
'DockerDefault'.".

The package(s) come(s) from a package source that is not marked as trusted.
Are you sure you want to install software from 'DockerDefault'?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): A
VERBOSE: Containers feature is already installed. Skipping the install.
VERBOSE: Verifying Hash of the downloaded file.
VERBOSE: hash=FEFEB0CD5566550E9ECAFB87CF26C78AFC8643B2450F2AC0203F6B88BE15B68D
VERBOSE: Destination =C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip
VERBOSE: fileHash =
VERBOSE: File C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip does not exist!!!
VERBOSE: fileHash with path is null
VERBOSE: fileHash with LiteralPath is null
WARNING: Cannot verify the file SHA256. Deleting the file.
VERBOSE: Hash verified!
WARNING: C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip does not exist
Install-Package : Cannot find path
'C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip' because it does not exist.
At line:1 char:1
+ Install-Package -Name docker -ProviderName DockerMsftProvider -Verbos ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (C:\Users\ADMINI...cs2-ws-beta.zip:String) [Install-Package], Exception
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand,Microsoft.PowerShell.Packag
eManagement.Cmdlets.InstallPackage

jianyunt commented 7 years ago

It looks like Get-FileHash returns null on your machine. Run gcm get-filehash

jaydeepch15 commented 7 years ago

PS C:\Users\Administrator> gcm get-filehash

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Function        Get-FileHash                                       3.1.0.0    Microsoft.PowerShell.Utility

jianyunt commented 7 years ago

hmm... need to debug it.

change Get-FileHash to Microsoft.PowerShell.Utility\Get-FileHash
add "wait-debugger" before Microsoft.PowerShell.Utility\Get-FileHash
save and relaunch powershell
run the install package
launch PowerShell ISE, following the debugging provider instruction to exam each variables mentioned above, such as filehash, Destination, etc.

jaydeepch15 commented 7 years ago

I debug hash , filehash, destination, filehashsha256 however doesnt get any error message one ex is as below

PS C:\Windows\system32> Get-PSHostProcessInfo

ProcessName    ProcessId AppDomainName    MainWindowTitle                      
-----------    --------- -------------    ---------------                      
powershell          3160 DefaultAppDomain Administrator: Windows PowerShell    
powershell_ise      4720 DefaultAppDomain Administrator: Windows PowerShell ISE

PS C:\Windows\system32> Enter-PSHostProcess 3160

[Process:3160]: PS C:\Users\Administrator\Documents> Get-Runspace

 Id Name            ComputerName    Type          State         Availability   
 -- ----            ------------    ----          -----         ------------   
  1 Runspace1       localhost       Local         Opened        Available      
  2 Runspace2       localhost       Local         Opened        Available      
  3 Runspace3       localhost       Local         Opened        Available      
  4 Runspace4       localhost       Local         Opened        InBreakpoint   
  5 Runspace5       localhost       Local         Opened        Available      
  6 RemoteHost      localhost       Local         Opened        Busy           

[Process:3160]: PS C:\Users\Administrator\Documents> Debug-Runspace 4
Debugging Runspace: Runspace4
[DBG]: [Process:3160]: [Runspace4]: PS C:\Users\Administrator>> $fileHash

[DBG]: [Process:3160]: [Runspace4]: PS C:\Users\Administrator>>

jianyunt commented 7 years ago

So you can confirm after executing (Press F10) Microsoft.PowerShell.Utility\Get-FileHash, it returns null, right?
During the debugging can you check if Docker-1-12-2-cs2-ws-beta.zip exists? and run Microsoft.PowerShell.Utility\Get-FileHash c:\....Docker-1-12-2-cs2-ws-beta.zip
you can run Microsoft.PowerShell.Utility\Get-FileHash on your PowerShell console no problem, right? e.g., Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe works?
Just curious, add "$a=Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe" in the DockerMsftProvider.psm1 and check to see if $a is null. If $a is not null, get-filehash has no problem. it is likely the docker.zip does not get downloaded.
Also, you may run install-package -debug -verbose ...
install-package -debug -verbose -force
Debugging further Can you add wait-debugger in line 280, e.g. right before calling DownloadPackageHelper. then F11 step into DownloadPackageHelper function. Watch out the code between line 1428 and 1452. Step into the function DownloadFile and check if the code line 1526-1538 executed.
by the way, which OS you are running? $PSVersionTable to check your PowerShell Version.
We may need a live meeting with you.

jaydeepch15 commented 7 years ago

My OS IS Windows 2016 Server I have attached the debug file.

1) I ran wait-debugger again with f10 and found Microsoft.PowerShell.Utility module not found 2) then ran Import-Module Microsoft.PowerShell.Utility 3) Then ran Install-package with --debug -verbose 4) $a= showed null value 5) Closed poweshell and re-run InstallModule this time $a shows value 6) Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe works fine, however when tried to check Microsoft.PowerShell.Utility\Get-FileHash c:....Docker-1-12-2-cs2-ws-beta.zip, its not there

7) My doubt is how come hash file entry shows in verbose mode.

PS C:\Users\Administrator> $PSVersionTable

    Name                           Value
    ----                           -----
    PSVersion                      5.1.14393.206
    PSEdition                      Desktop
    PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
    BuildVersion                   10.0.14393.206
    CLRVersion                     4.0.30319.42000
    WSManStackVersion              3.0
    PSRemotingProtocolVersion      2.3
    SerializationVersion           1.1.0.1

Details are in attached file

Debug.txt

jianyunt commented 7 years ago

I can see two potential issues. Let’s try to isolate problems whether it is related to download zip file issue or Get-FileHash issue. Step0: launch a new PowerShell console, Please confirm Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe works for you.

PS C:\> get-module

ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Manifest   3.1.0.0    Microsoft.PowerShell.Utility        {Add-Member, Add-Type, Clear-Variable, Compare-Object...}
Script     1.2        PSReadline                          {Get-PSReadlineKeyHandler, Get-PSReadlineOption, Remove-PSReadlineKeyHandler, ...

PS C:\> Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe

Algorithm       Hash                                                                   Path
---------       ----                                                                   ----
SHA256          BA4038FD20E474C047BE8AAD5BFACDB1BFC1DDBE12F803F473B7918D8D819436       C:\Windows\System32\WindowsPowerShell\v1.0\powers...

Step1: Close all PowerShell consoles rename your current MicrosoftDockerProvider under C:\Program Files\WindowsPowerShell\Modules to something else launch PowerShell install-module MicrosoftDockerProvider -force -verbose <== this will download the module.

Step 2: Open the DockerMsftProvider.psm1 from C:\Program Files\WindowsPowerShell\Modules\ and add the following code right after line 1647, i.e. “Write-Verbose "Verifying Hash of the downloaded file."; save it

$a=Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe

if (-not $a)
{
    Write-verbose  "Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe returns null"
}
else
{
    Write-verbose "Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe Successed! $a"
}

Step 3: Import-Packageprovider MicrosoftDockerProvider -force -verbose or you can simply relaunch PowerShell Console Step 4:

Install-package  …. -verbose -force

If $a is null, it is the get-filehash issue on your machine. If $a is not null, It may be the downloading Docker-1-12-2-cs2-ws-beta.zip' issue. let’s continue.

Step 5. Remove the code added in Step 2. Add the following code right after line 1647, i.e. “Write-Verbose "Verifying Hash of the downloaded file."

        if(-not (Test-Path -Path $Destination))
       {
              Write-Verbose  "File $Destination does not exist!!!"
       }

Step6: Import-Packageprovider MicrosoftDockerProvider -force -verbose Install-package …. -verbose -force

Please confirm you see the message `File …. Docker-1-12-2-cs2-ws-beta.zip does not exist!!!`.
If so, meaning the Docker-1-12-2-cs2-ws-beta.zip does not get downloaded. Let’s continue.

Step 7: Looking for “function DownloadFile” in the MicrosoftDockerProvider .psm1. Make changes as suggested below (added write-verbose in 4 places). Save it. Import-Packageprovider MicrosoftDockerProvider -force -verbose Install-package …. -verbose -force

function DownloadFile
{
    [CmdletBinding()]
    param
    (
        [Parameter(Mandatory=$true)]
        [System.String]
        $downloadURL, 

    [Parameter(Mandatory=$true)]
    [System.String]
    $destination
)

try
{

Write-Verbose “downloadURL = $downloadURL” Write-Verbose “destination = $destination”

    if(-not (CheckDiskSpace -Destination $destination -URL $downloadURL))
    {

 Write-Verbose “failed in CheckDiskSpace!!”

return }

    # Download the file
    if($downloadURL.StartsWith("https://"))
    {
        Write-Verbose "Downloading $downloadUrl to $destination"
        $saveItemPath = $PSScriptRoot + "\SaveHTTPItemUsingBITS.psm1"
        Import-Module "$saveItemPath"
        $startTime = Get-Date
        Write-Verbose "About to download"
        Save-HTTPItemUsingBitsTransfer -Uri $downloadURL `
                        -Destination $destination

        Write-Verbose "Finished downloading"
        $endTime = Get-Date
        $difference = New-TimeSpan -Start $startTime -End $endTime
        $downloadTime = "Downloaded in " + $difference.Hours + " hours, " + $difference.Minutes + " minutes, " + $difference.Seconds + " seconds."
        Write-Verbose $downloadTime
    }
else
{

   Write-Verbose “downloadURL does not StartsWith https!!!”
} catch { ThrowError -CallerPSCmdlet $PSCmdlet -ExceptionName $_.Exception.GetType().FullName -ExceptionMessage $_.Exception.Message -ExceptionObject $downloadURL -ErrorId FailedToDownload ` -ErrorCategory InvalidOperation
} }

jaydeepch15 commented 7 years ago

get-module and Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe working properly as you mention above. However when I rename C:\Program Files\WindowsPowerShell\Modules to C:\Program Files\WindowsPowerShell\Modules.org

I got below error

PS C:\Users\Administrator> install-module MicrosoftDockerProvider -force -verbose
install-module : The term 'install-module' is not recognized as the name of a cmdlet, function, script file,   or
operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try
again.
   At line:1 char:1
    + install-module MicrosoftDockerProvider -force -verbose
    + ~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (install-module:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

jianyunt commented 7 years ago

Rename C:\Program Files\WindowsPowerShell\Modules\MicrosoftDockerProvider to C:\Program Files\WindowsPowerShell\Modules\MicrosoftDockerProvider .org

jaydeepch15 commented 7 years ago

There no folder MicroSoftDockerProvider, I renamed DockerMsftProvider to DockerMsftProvider.org

then below Output

 PS C:\Users\Administrator> install-module MicrosoftDockerProvider -force -verbose
 VERBOSE: Using the provider 'PowerShellGet' for searching packages.
 VERBOSE: The -Repository parameter was not specified.  PowerShellGet will use all of the registered repositories.
 VERBOSE: Getting the provider object for the PackageManagement Provider 'NuGet'.
 VERBOSE: The specified Location is 'https://www.powershellgallery.com/api/v2/' and   PackageManagementProvider is
 'NuGet'.
 VERBOSE: Searching repository
 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='MicrosoftDockerProvider'' for ''.
 VERBOSE: Total package yield:'0' for the specified package 'MicrosoftDockerProvider'.
 PackageManagement\Install-Package : No match was found for the specified search criteria and module name
 'MicrosoftDockerProvider'. Try Get-PSRepository to see all available registered module repositories.
 At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1:1661   char:21
 + ...          $null = PackageManagement\Install-Package @PSBoundParameters
 +                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 + CategoryInfo          : ObjectNotFound: (Microsoft.Power....InstallPackage:InstallPackage) [Install- Package], Ex
    ception
 + FullyQualifiedErrorId :   NoMatchFoundForCriteria,Microsoft.PowerShell.PackageManagement.Cmdlets.InstallPackage

jianyunt commented 7 years ago

install-module DockerMsftProvider -force -verbose

jaydeepch15 commented 7 years ago

My Code is

    Write-Verbose "Verifying Hash of the downloaded file."
    $a=Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe
    if (-not $a)
    {
        Write-verbose  "Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe returns null"
    }
    else
    {
        Write-verbose "Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe Successed! $a"
    } 
   $fileHash = Get-FileHash -Path $Destination `
                            -Algorithm SHA256

And below is output

 PS C:\Users\Administrator> Import-Packageprovider DockerMsftProvider -force -verbose
VERBOSE: Importing package provider 'DockerMsftProvider'.

Name                     Version          DynamicOptions
----                     -------          --------------
DockerMsftProvider       1.0.0.1          Update

PS C:\Users\Administrator> Install-Package -Name docker -ProviderName DockerMsftProvider -Verbose -force
VERBOSE: Using the provider 'DockerMsftProvider' for searching packages.
VERBOSE: Download size: 0MB
VERBOSE: Free space on the drive: 34595.54MB
VERBOSE: Downloading https://dockermsft.blob.core.windows.net/dockercontainer/DockerMsftIndex.json to
C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\DockerDefault_DockerSearchIndex.json
VERBOSE: About to download
VERBOSE: Finished downloading
VERBOSE: Downloaded in 0 hours, 0 minutes, 2 seconds.
VERBOSE: Performing the operation "Install Package" on target "Package 'Docker' version '1.12.2-cs2-ws-beta' from
'DockerDefault'.".
VERBOSE: Containers feature is already installed. Skipping the install.
VERBOSE: Verifying Hash of the downloaded file.
 VERBOSE: Microsoft.PowerShell.Utility\Get-FileHash C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe returns
 null
 WARNING: Cannot verify the file SHA256. Deleting the file.
 VERBOSE: Hash verified!
 WARNING: C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip does not exist
 Install-Package : Cannot find path
 'C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip'   because it does not exist.
 At line:1 char:1
+ Install-Package -Name docker -ProviderName DockerMsftProvider -Verbos ...
+       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~     ~~~~~
   + CategoryInfo          : ObjectNotFound: (C:\Users\ADMINI...cs2-ws-beta.zip:String) [Install-Package],  Exception
  + FullyQualifiedErrorId :   PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand,Microsoft.PowerShell.Packag
eManagement.Cmdlets.InstallPackage

It look likes get-filehash issue is there

jianyunt commented 7 years ago

@KrishnaV-MSFT, @lzybkr, @bmanikm, any ideas why Microsoft.PowerShell.Utility\Get-FileHash $pshome\powershell.exe returns null?

lucas-simao commented 7 years ago

1- Download docker Start-BitsTransfer -Source https://dockermsft.blob.core.windows.net/dockercontainer/docker-1-12-2-cs2-ws-beta.zip -Destination /docker.zip

2- Chek file Get-FileHash -Path /docker.zip -Algorithm SHA256

3- Copy file for folder docker cp .\docker.zip C:\Users\Administrator\AppData\Local\Temp\DockerMsftProvider\

4- Get in folder cd C:\Users\Administrator\AppData\Local\Temp\DockerMsftProvider\

5- Rename file cp .\docker.zip Docker-1-12-2-cs2-ws-beta.zip

6- Install Docker Install-Package -Name docker -ProviderName DockerMsftProvider -Verbose

7- reboot VM Restart-Computer -Force

jaydeepch15 commented 7 years ago

PS C:\> Start-BitsTransfer -Source https://dockermsft.blob.core.windows.net/dockercontainer/docker-1-12-2-cs2-ws-beta.zip -Destination dc\docker.zip
PS C:\> cd dc
PS C:\dc> ls

    Directory: C:\dc

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----       10/13/2016   5:18 AM       14183015 docker.zip

PS C:\Users\Administrator\AppData\Local\Temp\2\DockerMsftProvider> cp .\docker.zip Docker-1-12-2-cs2-ws-beta.zip
PS C:\Users\Administrator\AppData\Local\Temp\2\DockerMsftProvider> ls

    Directory: C:\Users\Administrator\AppData\Local\Temp\2\DockerMsftProvider

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----       10/13/2016   5:18 AM       14183015 Docker-1-12-2-cs2-ws-beta.zip
-a----       10/13/2016   5:18 AM       14183015 docker.zip
-a----       10/13/2016  11:00 PM            589 DockerDefault_DockerSearchIndex.json

PS C:\Users\Administrator\AppData\Local\Temp\2\DockerMsftProvider> Get-FileHash -Path c:\dc\docker.zip -Algorithm SHA256

Algorithm       Hash                                                                   Path
---------       ----                                                                   ----
SHA256          FEFEB0CD5566550E9ECAFB87CF26C78AFC8643B2450F2AC0203F6B88BE15B68D       C:\dc\docker.zip

PS C:\Users\Administrator\AppData\Local\Temp\2\DockerMsftProvider> Install-Package -Name docker -ProviderName DockerMsft
Provider -Verbose
VERBOSE: Importing package provider 'DockerMsftProvider'.
VERBOSE: Using the provider 'DockerMsftProvider' for searching packages.
VERBOSE: Download size: 0MB
VERBOSE: Free space on the drive: 34546.27MB
VERBOSE: Downloading https://dockermsft.blob.core.windows.net/dockercontainer/DockerMsftIndex.json to
C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\DockerDefault_DockerSearchIndex.json
VERBOSE: About to download
VERBOSE: Finished downloading
VERBOSE: Downloaded in 0 hours, 0 minutes, 2 seconds.
VERBOSE: Performing the operation "Install Package" on target "Package 'Docker' version '1.12.2-cs2-ws-beta' from
'DockerDefault'.".

The package(s) come(s) from a package source that is not marked as trusted.
Are you sure you want to install software from 'DockerDefault'?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): A
VERBOSE: Containers feature is already installed. Skipping the install.
VERBOSE: C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip already exists.
Skipping save. Use -Force to overwrite.
VERBOSE: Verifying Hash of the downloaded file.
VERBOSE: Microsoft.PowerShell.Utility\Get-FileHash C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe returns
null
VERBOSE: Hash verified!
VERBOSE: Found C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip to install.
VERBOSE: Trying to unzip : C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip
VERBOSE: Preparing to expand...
VERBOSE: Created 'C:\Program Files\docker\docker.exe'.
VERBOSE: Created 'C:\Program Files\docker\dockerd.exe'.
VERBOSE: Trying to enable the docker service...
VERBOSE: Removing the archive: C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-1-12-2-cs2-ws-beta.zip
Install-Package : Cannot rename because item at 'C:\Program Files\dummyName' does not exist.
At line:1 char:1
+ Install-Package -Name docker -ProviderName DockerMsftProvider -Verbos ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (Microsoft.Power....InstallPackage:InstallPackage) [Install-Package],
   Exception
    + FullyQualifiedErrorId : InvalidOperation,Microsoft.PowerShell.Commands.RenameItemCommand,Microsoft.PowerShell.Pa
   ckageManagement.Cmdlets.InstallPackage

Post Reboot

  PS C:\Users\Administrator> docker ps
  error during connect: Get http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.25/containers/json: open //./pipe/docker_engine: The
 system cannot find the file specified.

danyhoron commented 7 years ago

After hours of digging, I found a hack. I opened registry editor, and changed the path of the Windows Service (HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Docker\ImagePath) to be: C:\Program Files\Docker\dockerd.exe --run-service -H npipe:// -b "none" After that the service started working.

niranjanch19 commented 7 years ago

@danyhoron Can you please explain why did you make this change and what exactly it does?

Did you had any issues with creating nat network with "fixed-cidr" after making the change you mentioned?

danyhoron commented 7 years ago

@niranjanch19 I made that change to be able to start the docker service. It did not worked before that.

I had no issues w/ nat network. Everything worked flawless.

kagarlickij commented 7 years ago

Folks, I've faced the same issue and it turned out that if you execute Install-Package -Name docker -ProviderName DockerMsftProvider -Force and it fails you don't have to give up and try to execute the same command a few more times, for me it had been installed from third or fourth attempt.

gaya3chandran1 commented 7 years ago

I too encounter the same error. Any solution? PS C:\Users\Administrator> Install-Package -Name docker -ProviderName DockerMsftProvider

The package(s) come(s) from a package source that is not marked as trusted. Are you sure you want to install software from 'DockerDefault'? [Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "N"): A WARNING: A restart is required to start docker service. Please restart your machine. WARNING: After the restart please start the docker service. Install-Package : Cannot rename because item at 'C:\Program Files\dummyName' does not exist. At line:1 char:1

Install-Package -Name docker -ProviderName DockerMsftProvider


+ CategoryInfo          : InvalidOperation: (Microsoft.Power....InstallPackage:InstallPackage) [Install-Package],
Exception
+ FullyQualifiedErrorId : InvalidOperation,Microsoft.PowerShell.Commands.RenameItemCommand,Microsoft.PowerShell.Pa
ckageManagement.Cmdlets.InstallPackage

InteXX commented 7 years ago

The solution, in my case at least, was to modify the Get-HttpResponse() function.

Beginning at line 1133, change this:

1133    $httpClient = New-Object System.Net.Http.HttpClient
1134    $response = $httpclient.GetAsync($Uri)

to this:

1133    $message = New-Object System.Net.Http.HttpRequestMessage
1134    $message.Method = [System.Net.Http.HttpMethod]::Head
1135    $message.RequestUri = $Uri
1136    $httpClient = New-Object System.Net.Http.HttpClient
1137    $response = $httpclient.SendAsync($message)

When the request sends the GET verb, the entire content is downloaded. On low-bandwidth connections such as mine, this results in a timeout. The Response.Result.Content property is null, which causes the subsequent package download to fail.

On the other hand, when we send the HEAD verb only the headers are returned. Since we're only looking for the Content-Length header in order to get the file size, this is all we need.

HTH

laoshancun commented 6 years ago

any updates? still got Docker-17-06-2-ee-13.zip does not exist on windows server 1709 update on 2018-06-11: it works fine on windows server 1803.

pablodav commented 6 years ago

Got same issue on Windows server 1803, still trying to find a workaround.

pablodav commented 6 years ago

What I have confirmed is that the hash is not verified because the file is not at destination, but couldn't find the line that downloads the docker...zip file as it looks like it's not the same as the function that downloads the .json file.

Doing:


$downloadURL = 'https://dockermsft.blob.core.windows.net/dockercontainer/docker-17-06-2-ee-13.zip'
$destination = 'C:\Users\ADMINI~1\AppData\Local\Temp\2\DockerMsftProvider\Docker-17-06-2-ee-13.zip'
Invoke-WebRequest -Uri $downloadURL -OutFile $destination
Install-Package Docker -ProviderName DockerMsftProvider -RequiredVersion $RequiredVersion -Verbose

It will work only without -Force, because -Force flag also removes the file before downloading and then it doesn't downloads there.

File that I have been checking for debugging:

C:\Program Files\WindowsPowerShell\Modules\DockerMsftProvider\1.0.0.4\DockerMsftProvider.psm1

I have never seen the package in the temp folder $destination, looks like some bug in the script doesn't download it or downloads it to some other destination.

pablodav commented 6 years ago

My workaround script:

https://github.com/pablodav/kubernetes-for-windows/blob/feature/kubespray_k8win1803/ansible/roles/windows/docker/files/install_docker_msft.ps1

vinicius91 commented 6 years ago

First of all I would like to say thank you for all you guys for the info provided in this Issue. I had the same problems installing Docker on a Windows Server 2016, but unfortunately @pablodav script didn't work for me. I created another one based on the solution that was giben by @danyhoron .

Here is it: https://github.com/vinicius91/docker-windows-containers/blob/master/PowerShell-Scripts/install_docker_windows_server.ps1

Cheers!!!

MagicJohnJang commented 5 years ago

Fix the script:

New temp path.
Temp path shows only after failed installation.
Passing parameters.

The script tested on WindowsServer2016core-hyper 14393.0.161119-1705.RS1

work script: https://gist.github.com/MagicJohnJang/46655c4b3228ca2f636e38fc7c84eb11

PlagueHO commented 5 years ago

This still occurs on a fresh install of Windows Server 2019:

Yes, the methods above do work around the problem. We know this is some issue with the binary file transfer and is resolved by using BITS to do the transfer.

So why is this still a problem after 3 years and why hasn't it been fixed yet?

sgf commented 5 years ago

i have the same problem. Cannot verify the file SHA256. Deleting the file.....

pull the 18.03 is ok just on pull the 18.09 version.

AdamWyzgol commented 5 years ago

to fix it just start BITS service

rcaunt commented 5 years ago

Similar issue with DockerProvider for 19.03.0-rc2 except in this case the hash doesn't match:

https://github.com/MicrosoftDocs/Virtualization-Documentation/issues/1158

StefanScherer commented 5 years ago

So, the general fix seems to update the package provider to the latest version available. This can be done with this command:

Find-PackageProvider DockerMsftProvider | Install-PackageProvider

Then try to install a new Docker package.

hades200082 commented 5 years ago

Same issue here - none of the solutions provided above had worked for me.

Windows Server 2019.

OneGet / MicrosoftDockerProvider

SHA256 not matching while installing docker #15