[bootstrapper] Improve handling of invalid hash

jamescrowley commented 7 years ago

Description

I receive a

The file 'C:\Windows\system32\config\systemprofile\AppData\Local\NuGet\Cache\Paket\5.92.3\paket.exe' already exists.

error when calling "paket.exe restore" having bumped the version to 5.92.3.

Have never seen this before. It goes away on a brand new build agent (they're automated), but comes back again.

Repro steps

Not yet able to reproduce. Known aspects:

Occurs when calling paket.exe restore
Most branches on build server running on 5.85.1 with paket.exe in magic mode, with version passed as argument.
New branch on build server running 5.92.3 with paket.exe in magic mode, with version passed in paket.dependencies.
Build servers rebuilt from scratch each day, and happened two days running so far.
Works successfully on first run of the day.
Subsequent builds of branch fail on that build server. Not sure if there is interaction with other builds going on here.
Happened once on local machine but haven't been able to reproduce since after clearing cache folder.

Expected behavior

paket.exe restores packages as in previous versions.

Actual behavior

The file 'C:\Windows\system32\config\systemprofile\AppData\Local\NuGet\Cache\Paket\5.92.3\paket.exe' already exists.

Packages are then not restored.

Known workarounds

Kill the build server and force a fresh instance.

matthid commented 7 years ago

system32??? So you are running this stuff on a system account? What I would expect is that the bootstrapper is cached and then all are using the same exe in the Cache.

In your specific scenario it seems the machines are cleared from time to time? We really need to repro this or at least track it down. Can you enable logs for the bootstrapper by setting the verbose flag or the PAKET_BOOTSTRAPPER_TRACE environment variable to true (ie PAKET_BOOTSTRAPPER_TRACE=true) and report back the results?

miloszes commented 7 years ago

Hi,

We've runned with the same issue. The problem arises on the continues integration machines (TeamCity). We're running paket (paket.boostrap latest version (checked an half hour ago) in the magic mode) with the following arguments:

.paket\paket.exe restore --verbose The environment variable has been also set with command (before paket execution :) ) setx PAKET_BOOTSTRAPPER_TRACE true

The only message We've received was:

The file 'C:\Users\XXXX\AppData\Local\NuGet\Cache\Paket\5.92.4\paket.exe' already exists. (Github - cached (temporarily ignore updates))

We're using the user dedicated for TeamCity builder.

The problem also occurs with the install command.

When you remove manually the file C:\Users\XXXX\AppData\Local\NuGet\Cache\Paket\5.92.4\paket.exe the build process runs correctly once. After the next paket restoration the same problem exists, which is not easilly seen, because bootstrap finishes its work with 0 code and the solution is compiled with old references :/.

Best Regards, MiloszeS

matthid commented 7 years ago

I don't even know what we changed there recently, could that be a side effect from signing the .exe?

miloszes commented 7 years ago

I don't know whether it helps. I've switched to the paket.bootstrap (execution of paket.bootstrapper.exe, then paket.exe). When I executed it manually I've received the following messages:

Checking Paket version (downloading latest stable)... Version 5.92.4 found in cache but it's hash isn't valid. Starting download from https://github.com/fsprojects/Paket/releases/download/5.92.4/paket.exe Hash of downloaded paket.exe is invalid, retrying once Hash of downloaded paket.exe still invalid (Using the file anyway) The file 'C:\Users\XXXXX\AppData\Local\NuGet\Cache\Paket\5.92.4\paket.exe' already exists. (Github - cached)

But the paket.exe has been downloaded into the same folder as paket.boostrap

It looks like a problem with hash validation and exception handling...

miloszes commented 7 years ago

@matthid it looks like the side offect of signing.

matthid commented 7 years ago

@forki?

matthid commented 7 years ago

see https://github.com/fsprojects/Paket/issues/2729 wtf

forki commented 7 years ago

Ok can you please push an unsigned version for now as quick fix?

Will need to investigate next week. /cc @agross

Am 06.09.2017 15:05 schrieb "Matthias Dittrich" notifications@github.com:

see #2729 https://github.com/fsprojects/Paket/issues/2729 wtf

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/fsprojects/Paket/issues/2725#issuecomment-327476761, or mute the thread https://github.com/notifications/unsubscribe-auth/AADgNKrrUYHBxE1Y9SrE8-YLWaVOBKnJks5sfpiAgaJpZM4PNBe7 .

agross commented 7 years ago

We've signed before, also with timestamping. Not sure why a new certificate should change semantics.

matthid commented 7 years ago

probably a bug in the build process. we calculate the hash before signing the exe? also a bug in the bootstrapper because it doesn't report the actual problem. it should either always warn and work or never work and fail with the correct message imho

matthid commented 7 years ago

I'll leave this open for the bootstrapper problem, as https://github.com/fsprojects/Paket/issues/2729 better describes the invalid hash...

matthid commented 7 years ago

So the immediate problem has been solved #2729 and the hashes match again, please retry. I'll leave this open as the bootstrapper still needs to be fixed in case this happens again. A PR is appreciated.

fsprojects / Paket