NuGet / NuGetGallery

NuGet Gallery is a package repository that powers https://www.nuget.org. Use this repo for reporting NuGet.org issues.
https://www.nuget.org/
Apache License 2.0
1.55k stars 644 forks source link

[Self-hosted Bug]: Build Issues in Allsigned environment #9750

Open cmcknz77 opened 11 months ago

cmcknz77 commented 11 months ago

Impact

I'm unable to use this version

Describe the bug

I'm in an AllSigned PowerShell environment (locked at Domain level) which means that none of the PowerShell scripts that are included with this project will run in my environment without having been signed first. As a result, in order for me to be able to run "build.ps1" I'm finding that I not only need to codesign all the included .ps files, I'm also having to make an amendment to "build.ps1" in so that it signs the NuGet/NuGet/ServerCommon/build/init.ps1 file before it tries to import it and use functions from it during the build process.

I'm accomplishing this by adding the following line in between that file being downloaded and it being run: Set-AuthenticodeSignature -FilePath "$PSScriptRoot/build/init.ps1" -Certificate (@(ls Cert:\CurrentUser\My\ -code)[-1])

But I'm still having a problem building the project afterwards. I have had it running since the start of 2022 and just tried to update it and I when I try to get it to build I'm receiving errors and the :

"BUILD FAILED: The term 'Remove-EditorconfigFile' is not recognized as the name of a cmdlet, function, script file".

Now, I've found the 'Remove-EditorconfigFile' function in https://github.com/NuGet/ServerCommon/blob/main/build/common.ps1 So I'm assuming that that's being pulled down by the ServerCommon/init.ps1 file but because it's being built on the fly, I can't sign it, so that function never gets imported and the build fails.

Repro Steps

  1. Using build commit e6a38a882007374b320420645f63cc30f2a93e4d
  2. . ./build.ps1
  3. PS C:\Users\user\Documents\GitHub\NuGetGallery> . .\build.ps1 VERBOSE: Performing the operation "Set-AuthenticodeSignature" on target "C:\Users\user\Documents\GitHub\NuGetGallery\build\init.ps1".

    Directory: C:\Users\user\Documents\GitHub\NuGetGallery\build

SignerCertificate Status Path


DFD24E9A4927B66CFE8F3685DF0FE8D65DBB34AC Valid init.ps1

PSPath : Microsoft.PowerShell.Core\FileSystem::C:\Users\user\Documents\GitHub\NuGetGallery\ServerComm on PSParentPath : Microsoft.PowerShell.Core\FileSystem::C:\Users\user\Documents\GitHub\NuGetGallery PSChildName : ServerCommon PSDrive : C PSProvider : Microsoft.PowerShell.Core\FileSystem PSIsContainer : True Name : ServerCommon FullName : C:\Users\user\Documents\GitHub\NuGetGallery\ServerCommon Parent : NuGetGallery Exists : True Root : C:\ Extension : CreationTime : 12/12/2023 00:35:47 CreationTimeUtc : 12/12/2023 00:35:47 LastAccessTime : 12/12/2023 00:35:47 LastAccessTimeUtc : 12/12/2023 00:35:47 LastWriteTime : 12/12/2023 00:35:47 LastWriteTimeUtc : 12/12/2023 00:35:47 Attributes : Directory Mode : d----- BaseName : ServerCommon Target : {} LinkType :

Getting ServerCommon repository... Initialized empty Git repository in C:/Users/user/Documents/GitHub/NuGetGallery/ServerCommon/.git/ remote: Enumerating objects: 3994, done. remote: Counting objects: 100% (682/682), done. remote: Compressing objects: 100% (448/448), done. Receiving objects: 100% (3994/3994), 1.60 MiB | 23.80 MiB/s, done.d 3312

Resolving deltas: 100% (2827/2827), done. From https://github.com/NuGet/ServerCommon

[00:35:52 +0] ============================================================ [00:35:52 +0] Build #785959 started at 12/12/2023 00:35:52 [00:35:52 +0] [BEGIN] Getting private build tools [00:35:52 +0] No private build tools are configured. Use the 'PRIVATE_BUILD_TOOLS_REPO' and 'PRIVATE_BUILD_TOOLS_COMMI T' environment variables. [00:35:52 +0] [DONE +0:00] Getting private build tools [00:35:52 +0] [BEGIN] Cleaning test results [00:35:52 +0] Cleaning test results [00:35:52 +0] [DONE +0:00] Cleaning test results [00:35:52 +0] [BEGIN] Installing NuGet.exe [00:35:52 +0] nuget.exe is already installed [00:35:52 +0] Marker file exists: C:\Users\user\Documents\GitHub\NuGetGallery.nuget.marker.v1 [00:35:52 +0] Setting NuGet .NET Framework credential path [00:35:52 +0] Increasing NuGet plug-in timeout values to 30 seconds. [00:35:52 +0] [DONE +0:00] Installing NuGet.exe WARNING: [00:35:52 +0] [SKIP] Clearing package cache [00:35:52 +0] [BEGIN] Clearing artifacts [00:35:52 +0] Clearing the Artifacts folder [00:35:52 +0] [DONE +0:00] Clearing artifacts [00:35:52 +0] [BEGIN] Restoring solution packages [00:35:52 +0] Installing packages @"C:\Users\user\Documents\GitHub\NuGetGallery.nuget" [00:35:52 +0] C:\Users\user\Documents\GitHub\NuGetGallery.nuget\nuget.exe install C:\Users\user\Documen ts\GitHub\NuGetGallery.nuget\packages.config -NonInteractive -ExcludeVersion -OutputDirectory C:\Users\user\Docu ments\GitHub\NuGetGallery\packages Feeds used: C:\Users\user.nuget\packages\ https://api.nuget.org/v3/index.json https://pkgs.dev.azure.com/dnceng/public/_packaging/nuget-build/nuget/v3/index.json

All packages listed in C:\Users\user\Documents\GitHub\NuGetGallery.nuget\packages.config are already installed. [00:35:54 +2] [DONE +0:01] Restoring solution packages [00:35:54 +0] [BEGIN] Set version metadata in AssemblyInfo.cs

LastWriteTime : 12/12/2023 00:35:54 Length : 0 Name : AssemblyInfo.g.cs

[00:35:54 +0] Getting version info in @"C:\Users\user\Documents\GitHub\NuGetGallery\src\NuGetGallery\Properties \AssemblyInfo.g.cs" [00:35:54 +0] [assembly: AssemblyVersion("4.4.5")] [00:35:54 +0] [assembly: AssemblyInformationalVersion("4.4.5-HEAD")] [00:35:54 +0] [assembly: AssemblyMetadata("Branch", "HEAD")] [00:35:54 +0] [assembly: AssemblyMetadata("CommitId", "e6a38a882")] [00:35:54 +0] [assembly: AssemblyMetadata("BuildDateUtc", "12/12/2023 00:35:54 +00:00")]

LastWriteTime : 12/12/2023 00:35:54 Length : 0 Name : AssemblyInfo.g.cs

[00:35:54 +0] Getting version info in @"C:\Users\user\Documents\GitHub\NuGetGallery\src\NuGetGallery.Core\Prope rties\AssemblyInfo.g.cs" [00:35:54 +0] [assembly: AssemblyVersion("4.4.5")] [00:35:54 +0] [assembly: AssemblyInformationalVersion("4.4.5-HEAD")] [00:35:54 +0] [assembly: AssemblyMetadata("Branch", "HEAD")] [00:35:54 +0] [assembly: AssemblyMetadata("CommitId", "e6a38a882")] [00:35:54 +0] [assembly: AssemblyMetadata("BuildDateUtc", "12/12/2023 00:35:54 +00:00")]

LastWriteTime : 12/12/2023 00:35:55 Length : 0 Name : AssemblyInfo.g.cs

[00:35:55 +0] Getting version info in @"C:\Users\user\Documents\GitHub\NuGetGallery\src\NuGetGallery.Services\P roperties\AssemblyInfo.g.cs" [00:35:55 +0] [assembly: AssemblyVersion("4.4.5")] [00:35:55 +0] [assembly: AssemblyInformationalVersion("4.4.5-HEAD")] [00:35:55 +0] [assembly: AssemblyMetadata("Branch", "HEAD")] [00:35:55 +0] [assembly: AssemblyMetadata("CommitId", "e6a38a882")] [00:35:55 +0] [assembly: AssemblyMetadata("BuildDateUtc", "12/12/2023 00:35:55 +00:00")]

LastWriteTime : 12/12/2023 00:35:55 Length : 0 Name : AssemblyInfo.g.cs

[00:35:55 +0] Getting version info in @"C:\Users\user\Documents\GitHub\NuGetGallery\src\NuGet.Services.Entities \Properties\AssemblyInfo.g.cs" [00:35:55 +0] [assembly: AssemblyVersion("4.4.5")] [00:35:55 +0] [assembly: AssemblyInformationalVersion("4.4.5-HEAD")] [00:35:55 +0] [assembly: AssemblyMetadata("Branch", "HEAD")] [00:35:55 +0] [assembly: AssemblyMetadata("CommitId", "e6a38a882")] [00:35:55 +0] [assembly: AssemblyMetadata("BuildDateUtc", "12/12/2023 00:35:55 +00:00")]

LastWriteTime : 12/12/2023 00:35:57 Length : 0 Name : AssemblyInfo.g.cs

[00:35:57 +2] Getting version info in @"C:\Users\user\Documents\GitHub\NuGetGallery\src\NuGet.Services.Database Migration\Properties\AssemblyInfo.g.cs" [00:35:57 +0] [assembly: AssemblyVersion("4.4.5")] [00:35:57 +0] [assembly: AssemblyInformationalVersion("4.4.5-HEAD")] [00:35:57 +0] [assembly: AssemblyMetadata("Branch", "HEAD")] [00:35:57 +0] [assembly: AssemblyMetadata("CommitId", "e6a38a882")] [00:35:57 +0] [assembly: AssemblyMetadata("BuildDateUtc", "12/12/2023 00:35:57 +00:00")]

LastWriteTime : 12/12/2023 00:35:57 Length : 0 Name : AssemblyInfo.g.cs

[00:35:57 +0] Getting version info in @"C:\Users\user\Documents\GitHub\NuGetGallery\src\DatabaseMigrationTools\ Properties\AssemblyInfo.g.cs" [00:35:57 +0] [assembly: AssemblyVersion("4.4.5")] [00:35:57 +0] [assembly: AssemblyInformationalVersion("4.4.5-HEAD")] [00:35:57 +0] [assembly: AssemblyMetadata("Branch", "HEAD")] [00:35:57 +0] [assembly: AssemblyMetadata("CommitId", "e6a38a882")] [00:35:57 +0] [assembly: AssemblyMetadata("BuildDateUtc", "12/12/2023 00:35:57 +00:00")]

LastWriteTime : 12/12/2023 00:35:57 Length : 0 Name : AssemblyInfo.g.cs

[00:35:57 +0] Getting version info in @"C:\Users\user\Documents\GitHub\NuGetGallery\src\AccountDeleter\Properti es\AssemblyInfo.g.cs" [00:35:58 +0] [assembly: AssemblyVersion("4.4.5")] [00:35:58 +0] [assembly: AssemblyInformationalVersion("4.4.5-HEAD")] [00:35:58 +0] [assembly: AssemblyMetadata("Branch", "HEAD")] [00:35:58 +0] [assembly: AssemblyMetadata("CommitId", "e6a38a882")] [00:35:58 +0] [assembly: AssemblyMetadata("BuildDateUtc", "12/12/2023 00:35:58 +00:00")]

LastWriteTime : 12/12/2023 00:35:58 Length : 0 Name : AssemblyInfo.g.cs

[00:35:58 +0] Getting version info in @"C:\Users\user\Documents\GitHub\NuGetGallery\src\GitHubVulnerabilities2D b\Properties\AssemblyInfo.g.cs" [00:35:58 +0] [assembly: AssemblyVersion("4.4.5")] [00:35:58 +0] [assembly: AssemblyInformationalVersion("4.4.5-HEAD")] [00:35:58 +0] [assembly: AssemblyMetadata("Branch", "HEAD")] [00:35:58 +0] [assembly: AssemblyMetadata("CommitId", "e6a38a882")] [00:35:58 +0] [assembly: AssemblyMetadata("BuildDateUtc", "12/12/2023 00:35:58 +00:00")]

LastWriteTime : 12/12/2023 00:35:58 Length : 0 Name : AssemblyInfo.g.cs

[00:35:58 +0] Getting version info in @"C:\Users\user\Documents\GitHub\NuGetGallery\src\GalleryTools\Properties \AssemblyInfo.g.cs" [00:35:58 +0] [assembly: AssemblyVersion("4.4.5")] [00:35:58 +0] [assembly: AssemblyInformationalVersion("4.4.5-HEAD")] [00:35:58 +0] [assembly: AssemblyMetadata("Branch", "HEAD")] [00:35:58 +0] [assembly: AssemblyMetadata("CommitId", "e6a38a882")] [00:35:58 +0] [assembly: AssemblyMetadata("BuildDateUtc", "12/12/2023 00:35:58 +00:00")]

LastWriteTime : 12/12/2023 00:35:58 Length : 0 Name : AssemblyInfo.g.cs

[00:35:58 +0] Getting version info in @"C:\Users\user\Documents\GitHub\NuGetGallery\src\VerifyMicrosoftPackage\ Properties\AssemblyInfo.g.cs" [00:35:58 +0] [assembly: AssemblyVersion("4.4.5")] [00:35:58 +0] [assembly: AssemblyInformationalVersion("4.4.5-HEAD")] [00:35:58 +0] [assembly: AssemblyMetadata("Branch", "HEAD")] [00:35:58 +0] [assembly: AssemblyMetadata("CommitId", "e6a38a882")] [00:35:58 +0] [assembly: AssemblyMetadata("BuildDateUtc", "12/12/2023 00:35:58 +00:00")] [00:35:58 +0] [DONE +0:04] Set version metadata in AssemblyInfo.cs [00:35:58 +0] [BEGIN] Removing .editorconfig file in NuGetGallery Error-Log : [00:35:59 +1] [FAILED +0:00] Removing .editorconfig file in NuGetGallery At C:\Users\user\Documents\GitHub\NuGetGallery\build\common.ps1:169 char:21

BUILD FAILED: The term 'Remove-EditorconfigFile' is not recognized as the name of a cmdlet, function, script file, or op erable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again . ERROR DETAILS: System.Management.Automation.CommandNotFoundException: The term 'Remove-EditorconfigFile' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, veri fy that the path is correct and try again. at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(FunctionContext funcContext, Exception exc eption) at System.Management.Automation.Interpreter.ActionCallInstruction2.Run(InterpretedFrame frame) at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame) at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame) at System.Management.Automation.Interpreter.Interpreter.Run(InterpretedFrame frame) at System.Management.Automation.Interpreter.LightLambda.RunVoid1[T0](T0 arg0) at System.Management.Automation.ScriptBlock.InvokeWithPipeImpl(ScriptBlockClauseToInvoke clauseToInvoke, Boolean crea teLocalScope, Dictionary2 functionsToDefine, List1 variablesToDefine, ErrorHandlingBehavior errorHandlingBehavior, Obj ect dollarUnder, Object input, Object scriptThis, Pipe outputPipe, InvocationInfo invocationInfo, Object[] args) at System.Management.Automation.ScriptBlock.<>c__DisplayClass57_0.<InvokeWithPipe>b__0() at System.Management.Automation.Runspaces.RunspaceBase.RunActionIfNoRunningPipelinesWithThreadCheck(Action action) at System.Management.Automation.ScriptBlock.InvokeWithPipe(Boolean useLocalScope, ErrorHandlingBehavior errorHandling Behavior, Object dollarUnder, Object input, Object scriptThis, Pipe outputPipe, InvocationInfo invocationInfo, Boolean p ropagateAllExceptionsToTop, List1 variablesToDefine, Dictionary`2 functionsToDefine, Object[] args) at System.Management.Automation.ScriptBlock.InvokeUsingCmdlet(Cmdlet contextCmdlet, Boolean useLocalScope, ErrorHandl ingBehavior errorHandlingBehavior, Object dollarUnder, Object input, Object scriptThis, Object[] args) at Microsoft.PowerShell.Commands.InvokeCommandCommand.EndProcessing() at System.Management.Automation.CommandProcessorBase.Complete()

Expected Behavior

Build succeeds. Gallery website available at expected server URL

Screenshots

No response

Additional Context and logs

No response

joelverhagen commented 7 months ago

Hey @cmcknz77, apologies on the delay. There's another user encountering a similar issue who I've spoken to over Twitter. https://twitter.com/joelverhagen/status/1773731527861404011

As a simple workaround for this problem, could you try my jver-noop branch on NuGetGallery? It should leave init.ps1 and the other PowerShell scripts as is (not overwrite them) if you set $env:UseExistingBuildTools = "true".

If this meets your needs, then we can consider merging this into our main branch if it doesn't break anything else on our side.

joelverhagen commented 5 months ago

I haven't heard back from you about this case.

We could introduce a feature to our build tools to disable the .ps1 getting reset every build (per my last comment). This would allow you to sign the build tools in place in your own fork and not have our script overwrite your signed copy each time.

If you're interested in getting this change in the main branches, consider opening a PR on ServerCommon and NuGetGallery using the jver-noop as a base (with whatever modifications you need) and we can discuss the approach.