Unable to delete an `azure-native.dbforpostgresql.Configuration` resource: PUT vs PATCH

[onordberg]

What happened?

I seem not to be able to delete an azure-native.dbforpostgresql.Configuration resource. As a consequence I can also not destroy a stack without manual intervention. As you can see in the reproducible example below I am only doing a minor configuration change.

Thank you for your great work on this project.

Example

// index.ts

import {
  Configuration,
  Server,
  SkuTier,
} from "@pulumi/azure-native/dbforpostgresql";
import { ResourceGroup } from "@pulumi/azure-native/resources";

const resourceGroup = new ResourceGroup(`deleteme`);

const server = new Server(
  `deleteme`,
  {
    resourceGroupName: resourceGroup.name,
    administratorLogin: "NotForProduction",
    administratorLoginPassword: "NotForProduction",
    sku: {
      name: "Standard_B1ms",
      tier: SkuTier.Burstable,
    },
    version: "16",
    storage: {
      storageSizeGB: 32,
    },
  },
  { parent: resourceGroup },
);

new Configuration(
  "deleteme",
  {
    resourceGroupName: resourceGroup.name,
    serverName: server.name,
    configurationName: "azure.extensions",
    value: "vector",
    source: "user-override",
  },
  { parent: server },
);

root@c38eaf6a2211:/dev/workspace/pulumi/bugs# pulumi up
Please choose a stack, or create a new one: <create a new stack>
Please enter your desired stack name: dev 
Created stack 'dev'
Previewing update (dev):
     Type                                                 Name      Plan       Info
 +   pulumi:pulumi:Stack                                  bugs-dev  create     1 message
 +   └─ azure-native:resources:ResourceGroup              deleteme  create     
 +      └─ azure-native:dbforpostgresql:Server            deleteme  create     
 +         └─ azure-native:dbforpostgresql:Configuration  deleteme  create     

Diagnostics:
  pulumi:pulumi:Stack (bugs-dev):
    Downloading provider: azure-native

Resources:
    + 4 to create

Do you want to perform this update? yes
Updating (dev):
     Type                                                 Name      Status             
 +   pulumi:pulumi:Stack                                  bugs-dev  created (391s)     
 +   └─ azure-native:resources:ResourceGroup              deleteme  created (1s)       
 +      └─ azure-native:dbforpostgresql:Server            deleteme  created (363s)     
 +         └─ azure-native:dbforpostgresql:Configuration  deleteme  created (17s)      

Resources:
    + 4 created

Duration: 6m33s

I have verified that the configuration change has happened in Azure.

After commenting out the Configuration code block and running pulumi up I get the following error:

root@c38eaf6a2211:/dev/workspace/pulumi/bugs# pulumi up
Previewing update (dev):
     Type                                                 Name      Plan       
     pulumi:pulumi:Stack                                  bugs-dev             
     └─ azure-native:resources:ResourceGroup              deleteme             
        └─ azure-native:dbforpostgresql:Server            deleteme             
 -         └─ azure-native:dbforpostgresql:Configuration  deleteme  delete     

Resources:
    - 1 to delete
    3 unchanged

Do you want to perform this update? yes
Updating (dev):
     Type                                                 Name      Status                  Info
     pulumi:pulumi:Stack                                  bugs-dev  **failed**              1 error
     └─ azure-native:resources:ResourceGroup              deleteme                          
        └─ azure-native:dbforpostgresql:Server            deleteme                          
 -         └─ azure-native:dbforpostgresql:Configuration  deleteme  **deleting failed**     1 error

Diagnostics:
  azure-native:dbforpostgresql:Configuration (deleteme):
    error: Code="InternalServerError" Message="An unexpected error occured while processing the request. Tracking ID: '38555a33-74b0-4a0c-86ad-34b644d0ee0f'"

  pulumi:pulumi:Stack (bugs-dev):
    error: update failed

Resources:
    3 unchanged

Duration: 7m42s

The same also happens when running pulumi destroy.

Output of pulumi about

root@c38eaf6a2211:/dev/workspace/pulumi/bugs# pulumi about
CLI          
Version      3.127.0
Go Version   go1.22.5
Go Compiler  gc

Plugins
KIND      NAME    VERSION
language  nodejs  unknown

Host     
OS       debian
Version  12.6
Arch     x86_64

This project is written in nodejs: executable='/usr/local/bin/node' version='v20.16.0'

Current Stack: organization/bugs/dev

TYPE                                        URN
pulumi:pulumi:Stack                         urn:pulumi:dev::bugs::pulumi:pulumi:Stack::bugs-dev
pulumi:providers:azure-native               urn:pulumi:dev::bugs::pulumi:providers:azure-native::default_2_45_0
azure-native:resources:ResourceGroup        urn:pulumi:dev::bugs::azure-native:resources:ResourceGroup::deleteme
azure-native:dbforpostgresql:Server         urn:pulumi:dev::bugs::azure-native:resources:ResourceGroup$azure-native:dbforpostgresql:Server::deleteme
azure-native:dbforpostgresql:Configuration  urn:pulumi:dev::bugs::azure-native:resources:ResourceGroup$azure-native:dbforpostgresql:Server$azure-native:dbforpostgresql:Configuration::deleteme

Found no pending operations associated with dev

Backend        
Name           c38eaf6a2211
URL            azblob://pulumi-state?storage_account=plmiprodst6fe3d9bd
User           root
Organizations  
Token type     personal

Pulumi locates its logs in /tmp by default

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[thomas11]

Thank you for reporting this issue! I can reproduce it. This is the Azure HTTP request and response.

HTTP Request Begin PUT https://management.azure.com/subscriptions/123/resourceGroups/deleteme488fffb2/providers/Microsoft.DBforPostgreSQL/flexibleServers/deletemefbe4ffe8/configurations/azure.extensions?api-version=2022-12-01 
{"properties":{"source":"system-default"}}

HTTP Response Begin PUT [https://management.azure.com/subscriptions/123/resourceGroups/deleteme488fffb2/providers/Microsoft.DBforPostgreSQL/flexibleServers/deletemefbe4ffe8/configurations/azure.extensions?api-version=2022-12-01 
HTTP/2.0 500 Internal Server Error
{"error":{"code":"InternalServerError","message":"An unexpected error occured while processing the request. Tracking ID: '57b96ac5-25a1-41ae-bfae-b7fdeaf36ef8'"}}

[thomas11]

The Azure docs state

If you want to reset the value of a parameter, you simply choose to leave out the optional --value parameter, and the service applies the default value.

The Pulumi provider does that (not sending a value on reset of the property), but it made me wonder whether the provider maybe shouldn't send {"source":"system-default"} either. So I tried that locally but got the same InternalServerError.

I tried both sending {} and {"properties": {}}, same result.

At this point I think there might be an Azure server-side issue here. The only thing I haven't tried yet is following the cli instructions I linked above and checking whether that works and, if yes, what request it sends.

[onordberg]

Thank you for checking and investigating, @thomas11! Much appreciated. Based on what you are writing it seems strange. If it is an Azure server-side issue, how do you typically proceed with a fix?

[thomas11]

I took a second, fresh look and compared our provider's request with what the Azure CLI does, via az postgres flexible-server parameter set --name azure.extensions -g $rg --server-name $server --debug. Turns out this does a PATCH request, not PUT. The Azure spec defines both without guidance that only PATCH works for resetting a config (except a slight hint in the name "Configurations_Update" vs. "Configurations_Put").

We'll have to look into adding an exception for updating this resource to the provider.

[thomas11]

Also filed [BUG] Cannot reset a dbforpostgresql Configuration via PUT #30143 against Azure.

[onordberg]

Belated thanks for following up on this and filing the bug, @thomas11

[penenkel]

Hi, is there a workaround for this issue?

I tried it with new Pulumi.CustomResourceOptions { DeletedWith = server } (pulumi-dotnet) but that seems to be ignored?

[thomas11]

Unfortunately, there's no known workaround currently. We're still discussing with the Azure team on the upstream issue.

[penenkel]

@thomas11, could you explain to me why new Pulumi.CustomResourceOptions { DeletedWith = server } does not work? I thought that this is a Pulumi-side option that causes the subordinate resource to be skipped when deleting the master resource. In my mind that should side-step the issue with Azure. At least for my specific case where I want to delete the entire server including the configuration.

[thomas11]

Hi @penenkel, sorry I didn't respond to that part of your message. I don't see at first glance why DeletedWith wouldn't work. To confirm, your code using it looks roughly like this?

import {
  Configuration,
  Server,
  SkuTier,
} from "@pulumi/azure-native/dbforpostgresql";
import { ResourceGroup } from "@pulumi/azure-native/resources";

const resourceGroup = new ResourceGroup(`deleteme`);

const server = new Server(
  `deleteme`,
  {
    resourceGroupName: resourceGroup.name,
    administratorLogin: "NotForProduction",
    administratorLoginPassword: "NotForProduction",
    sku: {
      name: "Standard_B1ms",
      tier: SkuTier.Burstable,
    },
    version: "16",
    storage: {
      storageSizeGB: 32,
    },
  },
  { parent: resourceGroup },
);

new Configuration(
  "deleteme",
  {
    resourceGroupName: resourceGroup.name,
    serverName: server.name,
    configurationName: "azure.extensions",
    value: "vector",
    source: "user-override",
  },
  { parent: server, deletedWith:server },
);

Are you using parent: as well or only deletedWith?

[penenkel]

I'm using the dotnet variant, but otherwise, my code looks similar. Though I'm not using parent, is that required?

[thomas11]

I don't think parent is required, no. I've asked some team members for more input.

[thomas11]

Hi @penenkel, I've just tried to reproduce the issue but the deletion worked fine with DeletedWith. Here's the complete program I used.

using Pulumi.AzureNative.Resources;
using Pulumi.AzureNative.DBforPostgreSQL;
using Pulumi.AzureNative.DBforPostgreSQL.Inputs;
using System.Collections.Generic;

return await Pulumi.Deployment.RunAsync(() =>
{
    var resourceGroup = new ResourceGroup("resourceGroup");

    var server = new Server("server", new ServerArgs
    {
        ResourceGroupName = resourceGroup.Name,
        Location = resourceGroup.Location,
        Version = "14",
        Sku = new SkuArgs
        {
            Name = "Standard_B1ms",
            Tier = SkuTier.Burstable
        },
        Storage = new StorageArgs
        {
            StorageSizeGB = 32
        },
        AdministratorLogin = "pulumiadminuser",
        AdministratorLoginPassword = "pulumiadminpassword"
    });

    var conf = new Configuration("deleteme", new ConfigurationArgs
    {
        ResourceGroupName = resourceGroup.Name,
        ServerName = server.Name,
        ConfigurationName = "azure.extensions",
        Value = "vector",
        Source = "user-override",
    },
    new Pulumi.CustomResourceOptions { DeletedWith = server }
    );

    return new Dictionary<string, object?>
    {
        ["serverAddress"] = server.FullyQualifiedDomainName
    };
});

I ran pulumi up, then pulumi down.

[penenkel]

hm, perhaps my stack had devolved into a broken state by that point. I will try again and report back.