windows webroot: This security ID may not be assigned as the owner of this object

This is a reopening of #8597. I suspect there is more than one cause to this error message. I've struggled with this while testing #9054.

My operating system is (include version):

Windows Server 2019

I installed Certbot with (snap, OS package manager, pip, certbot-auto, etc):

git master @ https://github.com/certbot/certbot/commit/d250d341934448e9b4a712f9c7801d655256cbd8

I ran this command and it produced this output:

Certbot's behavior differed from what I expected because:

The webroot plugin shouldn't be crashing with a permissions error.

It looks like in _copy_win_ownership, when Certbot tries to copy the owner (SYSTEM) from the source folder (C:\inetpub\wwwroot) to the destination folder (C:\inetpub\wwwroot\.well-known), Windows spits out this error.

If I repeat the same Certbot command 2 or 3 times more, it will issue the certificate, because it will create .well-known, crash, create acme-challenge, crash, then finally create the challenge file and succeed.

This is a stock standard Windows 2019 server from binarylane.com.au with IIS installed. If it's helpful, I can provide a virtual machine where this issue occurs.

cc @adferrand

Here is a Certbot log showing the issue (if available):

2021-10-16 08:54:24,530:DEBUG:certbot._internal.main:certbot version: 1.20.0.dev0
2021-10-16 08:54:24,530:DEBUG:certbot._internal.main:Location of certbot entry point: C:\Users\Administrator\Desktop\certbot\venv\Scripts\certbot
2021-10-16 08:54:24,530:DEBUG:certbot._internal.main:Arguments: ['--webroot', '--register-unsafely-without-email', '--dry-run', '-w', 'C:\\inetpub\\wwwroot', '-d', 'ethnic-baron.bnr.la']
2021-10-16 08:54:24,530:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#certbot-route53:auth,PluginEntryPoint#dns-cloudflare,PluginEntryPoint#dns-cloudxns,PluginEntryPoint#dns-digitalocean,PluginEntryPoint#dns-dnsimple,PluginEntryPoint#dns-dnsmadeeasy,PluginEntryPoint#dns-gehirn,PluginEntryPoint#dns-google,PluginEntryPoint#dns-linode,PluginEntryPoint#dns-luadns,PluginEntryPoint#dns-nsone,PluginEntryPoint#dns-ovh,PluginEntryPoint#dns-rfc2136,PluginEntryPoint#dns-route53,PluginEntryPoint#dns-sakuracloud,PluginEntryPoint#manual,PluginEntryPoint#nginx,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2021-10-16 08:54:24,564:DEBUG:certbot._internal.log:Root logging level set at 30
2021-10-16 08:54:24,564:DEBUG:certbot._internal.plugins.selection:Requested authenticator webroot and installer None
2021-10-16 08:54:24,564:DEBUG:certbot._internal.plugins.selection:Single candidate plugin: * webroot
Description: Place files in webroot directory
Interfaces: Authenticator, Plugin
Entry point: webroot = certbot._internal.plugins.webroot:Authenticator
Initialized: <certbot._internal.plugins.webroot.Authenticator object at 0x053E7F10>
Prep: True
2021-10-16 08:54:24,564:DEBUG:certbot._internal.plugins.selection:Selected authenticator <certbot._internal.plugins.webroot.Authenticator object at 0x053E7F10> and installer None
2021-10-16 08:54:24,564:INFO:certbot._internal.plugins.selection:Plugins selected: Authenticator webroot, Installer None
2021-10-16 08:54:24,595:DEBUG:certbot._internal.main:Picked account: <Account(RegistrationResource(body=Registration(key=None, contact=(), agreement=None, status=None, terms_of_service_agreed=None, only_return_existing=None, external_account_binding=None), uri='https://acme-staging-v02.api.letsencrypt.org/acme/acct/30264438', new_authzr_uri=None, terms_of_service=None), 0b8b32ff60434a3293352d02d1cc0ead, Meta(creation_dt=datetime.datetime(2021, 10, 15, 21, 30, 20, tzinfo=<UTC>), creation_host='ethnic-baron.bnr.la', register_to_eff=None))>
2021-10-16 08:54:24,595:DEBUG:acme.client:Sending GET request to https://acme-staging-v02.api.letsencrypt.org/directory.
2021-10-16 08:54:24,595:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org:443
2021-10-16 08:54:25,167:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "GET /directory HTTP/1.1" 200 724
2021-10-16 08:54:25,167:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Fri, 15 Oct 2021 21:54:25 GMT
Content-Type: application/json
Content-Length: 724
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "Dk2GWFNIoPc": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
  "keyChange": "https://acme-staging-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
    "website": "https://letsencrypt.org/docs/staging-environment/"
  },
  "newAccount": "https://acme-staging-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-staging-v02.api.letsencrypt.org/acme/new-order",
  "revokeCert": "https://acme-staging-v02.api.letsencrypt.org/acme/revoke-cert"
}
2021-10-16 08:54:25,168:DEBUG:certbot._internal.display.obj:Notifying user: Simulating a certificate request for ethnic-baron.bnr.la
2021-10-16 08:54:25,297:DEBUG:acme.client:Requesting fresh nonce
2021-10-16 08:54:25,297:DEBUG:acme.client:Sending HEAD request to https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce.
2021-10-16 08:54:25,497:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "HEAD /acme/new-nonce HTTP/1.1" 200 0
2021-10-16 08:54:25,497:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Fri, 15 Oct 2021 21:54:25 GMT
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: 0002EkQC_XMAVSvImeE-VPE-7BTw9bRIFPXFNKEJF-KwhY4
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

2021-10-16 08:54:25,498:DEBUG:acme.client:Storing nonce: 0002EkQC_XMAVSvImeE-VPE-7BTw9bRIFPXFNKEJF-KwhY4
2021-10-16 08:54:25,498:DEBUG:acme.client:JWS payload:
b'{\n  "identifiers": [\n    {\n      "type": "dns",\n      "value": "ethnic-baron.bnr.la"\n    }\n  ]\n}'
2021-10-16 08:54:25,502:DEBUG:acme.client:Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/new-order:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS1zdGFnaW5nLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYWNjdC8zMDI2NDQzOCIsICJub25jZSI6ICIwMDAyRWtRQ19YTUFWU3ZJbWVFLVZQRS03QlR3OWJSSUZQWEZOS0VKRi1Ld2hZNCIsICJ1cmwiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9uZXctb3JkZXIifQ",
  "signature": "N-OnIpw995ajn8LSeDow39Cs96IVMQyY4eg9AYVZ9ZzPUoeQh5wWgoAik4OxwLRtN86nAZ0VaqRZzaAIQJyzfCFzbHJLg1QaYIPYbEUlr8WUrNKNdemhOou_heDm8oAs-XI5V44RUdh7Za7xjQDHt6Q3DdYr1ypUp9BwBjsn9Xd1QXHzgAhQBP_AqUcKCQj09qaDyanjp5QIeMEVA4io2_IkfeSKJGIoBd3WX4AV5PkKX78Ll04Y9k4LMl89JUNs-PgQJQ2NokgJhEaAncDW2hfM_uu0hRfKRWHopjuAazBPd-lKCooO6sR_izs8esmbKOmWB5HeYUzRLcMDzMQxAg",
  "payload": "ewogICJpZGVudGlmaWVycyI6IFsKICAgIHsKICAgICAgInR5cGUiOiAiZG5zIiwKICAgICAgInZhbHVlIjogImV0aG5pYy1iYXJvbi5ibnIubGEiCiAgICB9CiAgXQp9"
}
2021-10-16 08:54:25,697:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/new-order HTTP/1.1" 201 353
2021-10-16 08:54:25,698:DEBUG:acme.client:Received response:
HTTP 201
Server: nginx
Date: Fri, 15 Oct 2021 21:54:25 GMT
Content-Type: application/json
Content-Length: 353
Connection: keep-alive
Boulder-Requester: 30264438
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Location: https://acme-staging-v02.api.letsencrypt.org/acme/order/30264438/779532848
Replay-Nonce: 0001VxPTPNlia7WAN8PeXX7yEsLCzHno_N3_z5Dd8OHtTPI
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "status": "pending",
  "expires": "2021-10-22T21:30:30Z",
  "identifiers": [
    {
      "type": "dns",
      "value": "ethnic-baron.bnr.la"
    }
  ],
  "authorizations": [
    "https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/715919928"
  ],
  "finalize": "https://acme-staging-v02.api.letsencrypt.org/acme/finalize/30264438/779532848"
}
2021-10-16 08:54:25,699:DEBUG:acme.client:Storing nonce: 0001VxPTPNlia7WAN8PeXX7yEsLCzHno_N3_z5Dd8OHtTPI
2021-10-16 08:54:25,699:DEBUG:acme.client:JWS payload:
b''
2021-10-16 08:54:25,706:DEBUG:acme.client:Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/715919928:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS1zdGFnaW5nLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYWNjdC8zMDI2NDQzOCIsICJub25jZSI6ICIwMDAxVnhQVFBObGlhN1dBTjhQZVhYN3lFc0xDekhub19OM196NURkOE9IdFRQSSIsICJ1cmwiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9hdXRoei12My83MTU5MTk5MjgifQ",
  "signature": "wNQ5SNbHCKLfMU6oMU5S2ONKG9q-7-jByK3A0IjAqAaydWrT2n1kYtIuGjssH1iUinM7mBBnxD-VyMtN-JkiX20NRMhmA9ncZfYvGiYXjaZzsAtsGd_RLHvgRl55KoQlizRTC_zalEd0VCIqRLqCicIkSYDw7fPN1AtSl-OlypFIA2a6vejuD_u5jYJTX6Bv0u4RwfYH-JQosY8e3CRgi4-rcFp56qg7ILUk1mXXTEJNihXrXqVA6iP4V7M9Ma-rM7OtYAHILqu0SWf0UuxMwZ_dBQnv4hUBMOpWQjC4y11hwT2R6-R7FrhOI4NOnqWHrQXPImyzhcq4gcT9CguHNg",
  "payload": ""
}
2021-10-16 08:54:25,898:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/715919928 HTTP/1.1" 200 818
2021-10-16 08:54:25,898:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Fri, 15 Oct 2021 21:54:25 GMT
Content-Type: application/json
Content-Length: 818
Connection: keep-alive
Boulder-Requester: 30264438
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: 0002a2MFn6Ie1bPAU2RSJ-LN3Q9nt5GDJLPBqhcz_Uw9v-0
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "identifier": {
    "type": "dns",
    "value": "ethnic-baron.bnr.la"
  },
  "status": "pending",
  "expires": "2021-10-22T21:30:30Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/715919928/36IZWQ",
      "token": "waq1Ys_sxpvU3VQPCy6A6Z4yqM4pEWluGk0tk69xeZo"
    },
    {
      "type": "dns-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/715919928/cxU4eA",
      "token": "waq1Ys_sxpvU3VQPCy6A6Z4yqM4pEWluGk0tk69xeZo"
    },
    {
      "type": "tls-alpn-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/715919928/O3dfmQ",
      "token": "waq1Ys_sxpvU3VQPCy6A6Z4yqM4pEWluGk0tk69xeZo"
    }
  ]
}
2021-10-16 08:54:25,898:DEBUG:acme.client:Storing nonce: 0002a2MFn6Ie1bPAU2RSJ-LN3Q9nt5GDJLPBqhcz_Uw9v-0
2021-10-16 08:54:25,899:INFO:certbot._internal.auth_handler:Performing the following challenges:
2021-10-16 08:54:25,899:INFO:certbot._internal.auth_handler:http-01 challenge for ethnic-baron.bnr.la
2021-10-16 08:54:25,899:INFO:certbot._internal.plugins.webroot:Using the webroot path C:\inetpub\wwwroot for all unmatched domains.
2021-10-16 08:54:25,899:DEBUG:certbot._internal.plugins.webroot:Creating root challenges validation dir at C:\inetpub\wwwroot\.well-known\acme-challenge
2021-10-16 08:54:25,901:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "c:\users\administrator\desktop\certbot\certbot\certbot\_internal\auth_handler.py", line 70, in handle_authorizations
    resps = self.auth.perform(achalls)
  File "c:\users\administrator\desktop\certbot\certbot\certbot\_internal\plugins\webroot.py", line 100, in perform
    self._create_challenge_dirs()
  File "c:\users\administrator\desktop\certbot\certbot\certbot\_internal\plugins\webroot.py", line 201, in _create_challenge_dirs
    filesystem.copy_ownership_and_apply_mode(
  File "c:\users\administrator\desktop\certbot\certbot\certbot\compat\filesystem.py", line 108, in copy_ownership_and_apply_mode
    _copy_win_ownership(src, dst)
  File "c:\users\administrator\desktop\certbot\certbot\certbot\compat\filesystem.py", line 632, in _copy_win_ownership
    win32security.SetFileSecurity(dst, win32security.OWNER_SECURITY_INFORMATION, security_dst)
pywintypes.error: (1307, 'SetFileSecurity', 'This security ID may not be assigned as the owner of this object.')

2021-10-16 08:54:25,901:DEBUG:certbot._internal.error_handler:Calling registered functions
2021-10-16 08:54:25,901:INFO:certbot._internal.auth_handler:Cleaning up challenges
2021-10-16 08:54:25,901:DEBUG:certbot._internal.plugins.webroot:Removing C:\inetpub\wwwroot\.well-known\acme-challenge\waq1Ys_sxpvU3VQPCy6A6Z4yqM4pEWluGk0tk69xeZo
2021-10-16 08:54:25,902:ERROR:certbot._internal.error_handler:Encountered exception during recovery: FileNotFoundError: [WinError 2] The system cannot find the file specified: 'C:\\inetpub\\wwwroot\\.well-known\\acme-challenge\\waq1Ys_sxpvU3VQPCy6A6Z4yqM4pEWluGk0tk69xeZo'
2021-10-16 08:54:25,902:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "C:\Users\Administrator\Desktop\certbot\venv\Scripts\certbot-script.py", line 33, in <module>
    sys.exit(load_entry_point('certbot', 'console_scripts', 'certbot')())
  File "c:\users\administrator\desktop\certbot\certbot\certbot\main.py", line 15, in main
    return internal_main.main(cli_args)
  File "c:\users\administrator\desktop\certbot\certbot\certbot\_internal\main.py", line 1574, in main
    return config.func(config, plugins)
  File "c:\users\administrator\desktop\certbot\certbot\certbot\_internal\main.py", line 1434, in certonly
    lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
  File "c:\users\administrator\desktop\certbot\certbot\certbot\_internal\main.py", line 133, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
  File "c:\users\administrator\desktop\certbot\certbot\certbot\_internal\client.py", line 454, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
  File "c:\users\administrator\desktop\certbot\certbot\certbot\_internal\client.py", line 384, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "c:\users\administrator\desktop\certbot\certbot\certbot\_internal\client.py", line 434, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
  File "c:\users\administrator\desktop\certbot\certbot\certbot\_internal\auth_handler.py", line 70, in handle_authorizations
    resps = self.auth.perform(achalls)
  File "c:\users\administrator\desktop\certbot\certbot\certbot\_internal\plugins\webroot.py", line 100, in perform
    self._create_challenge_dirs()
  File "c:\users\administrator\desktop\certbot\certbot\certbot\_internal\plugins\webroot.py", line 201, in _create_challenge_dirs
    filesystem.copy_ownership_and_apply_mode(
  File "c:\users\administrator\desktop\certbot\certbot\certbot\compat\filesystem.py", line 108, in copy_ownership_and_apply_mode
    _copy_win_ownership(src, dst)
  File "c:\users\administrator\desktop\certbot\certbot\certbot\compat\filesystem.py", line 632, in _copy_win_ownership
    win32security.SetFileSecurity(dst, win32security.OWNER_SECURITY_INFORMATION, security_dst)
pywintypes.error: (1307, 'SetFileSecurity', 'This security ID may not be assigned as the owner of this object.')
2021-10-16 08:54:25,903:ERROR:certbot._internal.log:An unexpected error occurred:
2021-10-16 08:54:25,903:ERROR:certbot._internal.log:pywintypes.error: (1307, 'SetFileSecurity', 'This security ID may not be assigned as the owner of this object.')

certbot / certbot