Pulumi: "key error" encountered while trying to deploy an SRE

[craddm]

Possibly because I've missed out something that's required to be supplied as an option to the dsh deploy sre command?

(data-safe-haven) deploydsh@a3e2d2920f17:~/.config/data_safe_haven$ dsh deploy sre raphael -d 193.60.220.253 -u 193.60.220.253 
2023-09-20 15:02:51 [    INFO] Reading project settings from '/home/deploydsh/.config/data_safe_haven/config.yaml'. backend_settings.py:110
2023-09-20 15:02:52 [    INFO] IP addresses used by data providers will be ['193.60.220.253/32'].                             config.py:292
2023-09-20 15:02:52 [    INFO] Databases available to users will be [].                                                       config.py:301
2023-09-20 15:02:52 [    INFO] Copying text out of the SRE will be forbidden.                                                 config.py:228
2023-09-20 15:02:52 [    INFO] Pasting text into the SRE will be forbidden.                                                   config.py:234
2023-09-20 15:02:52 [    INFO] Workspace SKUs will be [].                                                                     config.py:311
2023-09-20 15:02:52 [    INFO] Software packages from none sources will be installable.                                       config.py:316
2023-09-20 15:02:52 [    INFO] IP addresses used by users will be ['193.60.220.253/32'].                                      config.py:323
2023-09-20 15:02:53 [    INFO] Administrator approval is needed in order to interact with Azure Active Directory.          graph_api.py:382
2023-09-20 15:02:53 [    INFO] Please sign-in with global administrator credentials for Azure Active Directory             graph_api.py:385
cb94a6f6-ef7a-42ab-bcad-4f0b887cfd3e.                                                                                                      
2023-09-20 15:02:53 [    INFO] Note that the sign-in screen will prompt you to sign-in to Microsoft Graph Command Line     graph_api.py:389
Tools - this is expected.                                                                                                                  
2023-09-20 15:02:53 [    INFO] To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter  graph_api.py:393
the code G3ALE9LVN to authenticate.                                                                                                        
2023-09-20 15:03:36 [    INFO] Creating/loading stack shm-green.                                                        stack_manager.py:77
2023-09-20 15:03:39 [    INFO] Logged in to a3e2d2920f17 as deploydsh (azblob://pulumi)                                stack_manager.py:290

2023-09-20 15:03:39 [    INFO] Creating/loading stack shm-green-sre-raphael.                                            stack_manager.py:77
2023-09-20 15:03:42 [    INFO] Loaded stack shm-green-sre-raphael.                                                      stack_manager.py:89
2023-09-20 15:03:42 [    INFO] Logged into Pulumi as deploydsh                                                         stack_manager.py:272
2023-09-20 15:03:42 [    INFO] Creating/loading stack shm-green.                                                        stack_manager.py:77
2023-09-20 15:03:44 [    INFO] Loaded stack shm-green.                                                                  stack_manager.py:89
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /workspaces/data-safe-haven/data_safe_haven/commands/deploy.py:168 in sre                        │
│                                                                                                  │
│   165 │   ] = None,                                                                              │
│   166 ) -> None:                                                                                 │
│   167 │   """Deploy a Secure Research Environment"""                                             │
│ ❱ 168 │   deploy_sre(                                                                            │
│   169 │   │   name,                                                                              │
│   170 │   │   allow_copy=allow_copy,                                                             │
│   171 │   │   allow_paste=allow_paste,                                                           │
│                                                                                                  │
│ ╭────────────────────── locals ──────────────────────╮                                           │
│ │                 allow_copy = None                  │                                           │
│ │                allow_paste = None                  │                                           │
│ │ data_provider_ip_addresses = ['193.60.220.253/32'] │                                           │
│ │                  databases = []                    │                                           │
│ │                      force = None                  │                                           │
│ │                       name = 'raphael'             │                                           │
│ │          software_packages = None                  │                                           │
│ │          user_ip_addresses = ['193.60.220.253/32'] │                                           │
│ │             workspace_skus = []                    │                                           │
│ ╰────────────────────────────────────────────────────╯                                           │
│                                                                                                  │
│ /workspaces/data-safe-haven/data_safe_haven/commands/deploy_sre.py:64 in deploy_sre              │
│                                                                                                  │
│    61 │   │   # Load SHM stack outputs                                                           │
│    62 │   │   stack.add_option(                                                                  │
│    63 │   │   │   "shm-domain_controllers-domain_sid",                                           │
│ ❱  64 │   │   │   shm_stack.output("domain_controllers")["domain_sid"],                          │
│    65 │   │   │   replace=True,                                                                  │
│    66 │   │   )                                                                                  │
│    67 │   │   stack.add_option(                                                                  │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │                 allow_copy = None                                                            │ │
│ │                allow_paste = None                                                            │ │
│ │                     config = <data_safe_haven.config.config.Config object at 0xffff90f7caf0> │ │
│ │ data_provider_ip_addresses = ['193.60.220.253/32']                                           │ │
│ │                  databases = []                                                              │ │
│ │                      force = None                                                            │ │
│ │                  graph_api = <data_safe_haven.external.api.graph_api.GraphApi object at      │ │
│ │                              0xffff90f156c0>                                                 │ │
│ │                       name = 'raphael'                                                       │ │
│ │                  shm_stack = <data_safe_haven.infrastructure.stack_manager.SHMStackManager   │ │
│ │                              object at 0xffff90f17a00>                                       │ │
│ │          software_packages = None                                                            │ │
│ │                   sre_name = 'raphael'                                                       │ │
│ │                      stack = <data_safe_haven.infrastructure.stack_manager.SREStackManager   │ │
│ │                              object at 0xffff90eee320>                                       │ │
│ │          user_ip_addresses = ['193.60.220.253/32']                                           │ │
│ │             workspace_skus = []                                                              │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /workspaces/data-safe-haven/data_safe_haven/infrastructure/stack_manager.py:302 in output        │
│                                                                                                  │
│   299 │   │   """Get a named output value from a stack"""                                        │
│   300 │   │   if not self.stack_outputs_:                                                        │
│   301 │   │   │   self.stack_outputs_ = self.stack.outputs()                                     │
│ ❱ 302 │   │   return self.stack_outputs_[name].value                                             │
│   303 │                                                                                          │
│   304 │   def preview(self) -> None:                                                             │
│   305 │   │   """Preview the Pulumi stack."""                                                    │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ name = 'domain_controllers'                                                                  │ │
│ │ self = <data_safe_haven.infrastructure.stack_manager.SHMStackManager object at               │ │
│ │        0xffff90f17a00>                                                                       │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
KeyError: 'domain_controllers'

[jemrobinson]

No, this is due to the way that I implemented passing information between stacks. There's an SHM stack and one or more SRE stacks. The SRE stack wants to load some information from the SHM stack. The SHM stack outputs should be available as JSON but it looks like this isn't working. I'll take a look.

[JimMadge]

I can reproduce this but only when logged into the wrong Pulumi account.

[craddm]

I'd started to guess at this being the reason, as it says I'm logged in to pulumi as deploydsh. What isn't clear to me is how to make sure I'm logged in to the right pulumi account.

[JimMadge]

Looking at how Pulumi login is handled https://github.com/alan-turing-institute/data-safe-haven/blob/64a55d832e9a4426bc880a379b9b54193e84dcb3/data_safe_haven/infrastructure/stack_manager.py#L260

Presumably your ability to use the storage account as a backend depends on which AZ account you are authenticated with. Maybe here the dsploydsh user doesn't have the correct permissions and isn't the user we want to use.

[craddm]

This was due to using an old version of Pulumi. After updating to 3.80.0, which is what the SHM was deployed with, this error doesn't occur.