search

actions / runner-images

GitHub Actions runner images
MIT License
10.28k stars 3.09k forks source link

wuauserv cannot be enabled or started #4821

Closed lonevvolf closed 2 years ago

lonevvolf commented 2 years ago

Description

Using the default scripts to build the image for win2019, there appears to be a difference between the MS-hosted pool image and the one produced by the scripts. When trying to enable wuauserv, the user does not have permissions. Simply changing to the MS-hosted pool works immediately.

Virtual environments affected

Image version and build link

Image version in logs is "dev", but release label used to build the image was "win19/20211219.1"

Is it regression?

unknown

Expected behavior

[debug]Entering Assert-SdtAtLeastOneServiceIsValid.

[debug] ExpectedServices: 'System.Collections.Hashtable'

[debug] MessageForServiceNotFound: 'Windows Update is required to download and install up-to-date Antimalware signatures.'

[debug] MessageForServiceDisabled: 'If the group policy does not disable the service and the account this build is running under has admin privileges, you can check the "Enable Required Services for Windows Update if Disabled" checkbox in the build task UI to have the service enabled.'

[debug] AllServices: 'True'

[debug] Throw: 'True'

[debug]Try getting service: wuauserv

[debug]Found service: Windows Update (wuauserv), with Status: Stopped, StartType: Disabled

[debug]Service Windows Update (wuauserv) is disabled

[debug]Try enabling the Windows Update (wuauserv) service...

[debug]Entering Set-SdtService.

[debug] Service: 'System.ServiceProcess.ServiceController'

[debug] StartupType: 'Manual'

[debug]Leaving Set-SdtService.

[debug]Starting the Windows Update (wuauserv) service...

[debug]The Windows Update (wuauserv) service current Status: Running, StartType: Manual

[debug]Leaving Assert-SdtAtLeastOneServiceIsValid.

Actual behavior

[debug]Script stack trace:

[debug]at Write-DteError, C:\a_tasks\AntiMalware_02f6ca63-3d4a-4c14-833a-737a4ab099dd\3.308.0\DevToolsEngine\Write-DteError.psm1: line 16

[debug]at Write-DteMessage, C:\a_tasks\AntiMalware_02f6ca63-3d4a-4c14-833a-737a4ab099dd\3.308.0\DevToolsEngine\Write-DteMessage.psm1: line 28

[debug]at Write-SdtAntiMalwareMessage, C:\a_tasks\AntiMalware_02f6ca63-3d4a-4c14-833a-737a4ab099dd\3.308.0\Assert-SdtAtLeastOneServiceIsValid.psm1: line 234

[debug]at Assert-SdtAtLeastOneServiceIsValid, C:\a_tasks\AntiMalware_02f6ca63-3d4a-4c14-833a-737a4ab099dd\3.308.0\Assert-SdtAtLeastOneServiceIsValid.psm1: line 113

[debug]at Invoke-SdtAntiMalware, C:\a_tasks\AntiMalware_02f6ca63-3d4a-4c14-833a-737a4ab099dd\3.308.0\SdtAntiMalware.psm1: line 171

[debug]at , C:\a_tasks\AntiMalware_02f6ca63-3d4a-4c14-833a-737a4ab099dd\3.308.0\AntiMalwarePSV3.ps1: line 24

[debug]at , : line 1

[debug]at , : line 22

[debug]at , : line 18

[debug]at , : line 1

[debug]Exception:

[debug]Microsoft.PowerShell.Commands.WriteErrorException: If the group policy does not disable the service and the account this build is running under has admin privileges, you can check the "Enable Required Services for Windows Update if Disabled" checkbox in the build task UI to have the service enabled.

[error]If the group policy does not disable the service and the account this build is running under has admin privileges, you can check the "Enable Required Services for Windows Update if Disabled" checkbox in the build task UI to have the service enabled.

[debug]Processed: ##vso[task.logissue type=error]If the group policy does not disable the service and the account this build is running under has admin privileges, you can check the "Enable Required Services for Windows Update if Disabled" checkbox in the build task UI to have the service enabled.

Repro steps

Run a build against the default image/agent with the AntiMalware Scan Azure DevOps task.

shilovmaksim commented 2 years ago

Hi @lonevvolf We will take a look at this.

al-cheb commented 2 years ago

@lonevvolf , Could you please provide the output of the whoami /groups command from a self-hosted agent?

lonevvolf commented 2 years ago 
Starting: CmdLine
==============================================================================
Task         : Command line
Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version      : 2.182.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
==============================================================================
Generating script.
Script contents:
whoami /groups
========================== Starting Command Output ===========================
"C:\Windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "C:\a\_temp\c8c1af35-c67d-4c03-8dcb-5b9b521bd1f8.cmd""

GROUP INFORMATION
-----------------

Group Name                                                    Type             SID                                           Attributes                                        
============================================================= ================ ============================================= ==================================================
Everyone                                                      Well-known group S-1-1-0                                       Mandatory group, Enabled by default, Enabled group
NT AUTHORITY\Local account and member of Administrators group Well-known group S-1-5-114                                     Group used for deny only                          
<machineprefix>-0000M5\VSTS_AgentService_G39071                      Alias            S-1-5-21-1965340077-819459311-2943365379-1004 Mandatory group, Enabled by default, Enabled group
BUILTIN\Administrators                                        Alias            S-1-5-32-544                                  Group used for deny only                          
BUILTIN\Users                                                 Alias            S-1-5-32-545                                  Mandatory group, Enabled by default, Enabled group
BUILTIN\Performance Log Users                                 Alias            S-1-5-32-559                                  Mandatory group, Enabled by default, Enabled group
NT AUTHORITY\INTERACTIVE                                      Well-known group S-1-5-4                                       Mandatory group, Enabled by default, Enabled group
CONSOLE LOGON                                                 Well-known group S-1-2-1                                       Mandatory group, Enabled by default, Enabled group
NT AUTHORITY\Authenticated Users                              Well-known group S-1-5-11                                      Mandatory group, Enabled by default, Enabled group
NT AUTHORITY\This Organization                                Well-known group S-1-5-15                                      Mandatory group, Enabled by default, Enabled group
NT AUTHORITY\Local account                                    Well-known group S-1-5-113                                     Mandatory group, Enabled by default, Enabled group
LOCAL                                                         Well-known group S-1-2-0                                       Mandatory group, Enabled by default, Enabled group
NT AUTHORITY\NTLM Authentication                              Well-known group S-1-5-64-10                                   Mandatory group, Enabled by default, Enabled group
Mandatory Label\Medium Mandatory Level                        Label            S-1-16-8192                                                                                     
Finishing: CmdLine
al-cheb commented 2 years ago

@lonevvolf , As I can see you have limited token Mandatory Label\Medium Mandatory Level UAC is enabled.

Configure VMs to run interactive tests (Windows Server OS Only) - Windows agents can either be configured to run unelevated with autologon and with interactive UI, or they can be configured to run with elevated permissions. Check this box to run unelevated with interactive UI. In either case, the agent user is a member of the Administrators group. image

lonevvolf commented 2 years ago

Do you mean to say that we cannot mix interactive tests with the malware scans? This is working on the MS-hosted pipelines.

lonevvolf commented 2 years ago

How can we have the same behavior as with MS-hosted pipelines?

lonevvolf commented 2 years ago

Apologies, I don't follow what is meant here by capacity or system behavior.

We are using the MS-provided scripts to build the system image. They claim these are the same scripts used to build the MS-hosted images, yet they are exhibiting different behavior. How can we align our image to act the same as the official one?

lonevvolf commented 2 years ago

And can u also see what was the last error during this failure of the image?

The error is listed in the original report.

al-cheb commented 2 years ago

Do you mean to say that we cannot mix interactive tests with the malware scans? This is working on the MS-hosted pipelines.

This works on Hosted Agents, because the runner user has full token and an agent runs via script in the user session.

image

lonevvolf commented 2 years ago

Do you mean to say that we cannot mix interactive tests with the malware scans? This is working on the MS-hosted pipelines.

This works on Hosted Agents, because the runner user has full token and an agent runs via script in the user session.

image

Is there a way to change the self-hosted behavior to the same without changing that checkbox? Again, both interactive tests and the scanner tasks work on the MS-hosted without any changes, but not on self-hosted.

al-cheb commented 2 years ago

Do you mean to say that we cannot mix interactive tests with the malware scans? This is working on the MS-hosted pipelines.

This works on Hosted Agents, because the runner user has full token and an agent runs via script in the user session. image

Is there a way to change the self-hosted behavior to the same without changing that checkbox? Again, both interactive tests and the scanner tasks work on the MS-hosted without any changes, but not on self-hosted.

We don't use VMSS based pool, as I mentioned above we run an agent in the user session via task scheduler in interactive mode, that's why we don't have such configuration.

lonevvolf commented 2 years ago

@freddy123098 I think a main reason for my confusion is you need to check your comment visibility - it looks like you are having internal conversations and I am seeing half of it.

lonevvolf commented 2 years ago

Since there seems to be some confusion based on the comments I have seen, I will just link to the MS article here which are following to setup the VMSS as an agent pool: Create the scale set agent pool

Note that the agents are automatically provisioned - we have little to no control here over that part.

al-cheb commented 2 years ago

Since there seems to be some confusion based on the comments I have seen, I will just link to the MS article here which are following to setup the VMSS as an agent pool: Create the scale set agent pool

Note that the agents are automatically provisioned - we have little to no control here over that part.

In that case you have 2 options:

  1. Uncheck Configure VMs to run interactive tests
  2. Don't use VMSS and manually provision a self-hosted pool in the interactive mode
lonevvolf commented 2 years ago

2. Don't use VMSS and manually provision a self-hosted pool in the interactive mode

If we were to do this, what would we configure differently to get the feature to work?

al-cheb commented 2 years ago

If we were to do this, what would we configure differently to get the feature to work?

  1. Deploy a vm
  2. Configure autologon
  3. Install vsts agent in interactive mode(configure.cmd & run.cmd)
  4. Run a build
lonevvolf commented 2 years ago

BTW, option #1 is not an option for us due to: Failure to get lock fails installation (yes I realize it's a different team, just want to mention)

I guess in this case, the issue comes down to a difference in the implementation of agent pools in Azure DevOps between self- and MS-hosted. A third option in our case would be to just enable wuauserv by default in the image, but I'm not sure how to do that yet. Any tip would be greatly appreciated. Otherwise, I guess you can close this issue.

al-cheb commented 2 years ago

BTW, option #1 is not an option for us due to: Failure to get lock fails installation (yes I realize it's a different team, just want to mention)

I guess in this case, the issue comes down to a difference in the implementation of agent pools in Azure DevOps between self- and MS-hosted. A third option in our case would be to just enable wuauserv by default in the image, but I'm not sure how to do that yet. Any tip would be greatly appreciated. Otherwise, I guess you can close this issue.

  1. Failure to get lock fails installation <- As I understand this issue occurs at an agent installation time and not at the running build time.

  2. We have intentionally disabled "Windows Updates" to prevent unexpected situations and by default wuauserv service is set to manual start mode. Maybe, CSE could help - https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/scale-set-agents?view=azure-devops#customizing-virtual-machine-startup-via-the-custom-script-extension

Close as external.