`pulumi refresh` doesn't show a resource change, only in the following `pulumi up`

What happened?

This issue is related to #3521

When I deploy the Pulumi app below, everything works as expected during the 1st deployment and the stack output outboundIpAddresses shows the same value as the Azure portal ✔.

However as explained in #3521, changing the sku of the AppServicePlan causes the WebApp outbound IP addresses to change but not having the new value shown in the stack output ❌.

So, I tried to run a refresh to force the resource state to be updated with the latest details present in the Azure Portal but no changes were displayed ❌ despite the operation completing correctly.

Only running an up after the refresh, and despite no code changes, forced the stack output to be updated with the new outbound IP addresses.

Example

Steps to reproduce:

p up --yes --skip-preview --logtostderr --verbose=9 2> pulumi-up-1.log ✔
Go to the Azure portal and take note of the Outbound IP addresses
Comment/Uncomment the sku in the AppServicePlan resource
p up --yes --skip-preview --logtostderr --verbose=9 2> pulumi-up-2.log ✔
Go to the Azure Portal and compare the new Outbound IP addresses with the ones noted before
The stack output hasn't change despite new IP addresses shown in the portal (see #3521)
p up --yes --skip-preview --logtostderr --verbose=9 2> pulumi-up-3.log (with no code changes) ✔
The stack output hasn't change ❌
p refresh --yes --skip-preview --logtostderr --verbose=9 2> pulumi-refresh-4.log ✔
The stack output hasn't change ❌
p up --yes --skip-preview --logtostderr --verbose=9 2> pulumi-up-5.log (with no code changes) ✔
The stack output shows the updated Outbound IP addresses ✔

import * as pulumi from "@pulumi/pulumi";
import * as resources from "@pulumi/azure-native/resources";
import * as web from "@pulumi/azure-native/web/v20231201";

export = async () => {
    const resourceGroup = new resources.ResourceGroup("resourceGroup");

    const plan = new web.AppServicePlan("appServicePlan", {
        resourceGroupName: resourceGroup.name,
        kind: "app,linux", // https://github.com/Azure/app-service-linux-docs/blob/master/Things_You_Should_Know/kind_property.md
        // sku: {
        //     tier: "Basic",
        //     name: "B1",
        //     capacity: 1,
        // },
        sku: {
            tier: "Premium",
            name: "P2V3",
            capacity: 1,
        }
    }, {
        parent: resourceGroup,
    });

    const app = new web.WebApp("webApp", {
        resourceGroupName: resourceGroup.name,
        serverFarmId: plan.id,
        siteConfig: {
            use32BitWorkerProcess: false,
            netFrameworkVersion: "v8.0",
            appSettings: [
                { name: "FUNCTIONS_WORKER_RUNTIME", value: "dotnet" },
                { name: "WEBSITE_RUN_FROM_PACKAGE", value: "0" }

            ]
        },
        // httpsOnly: false,
    }, {
        parent: resourceGroup
    });

    return {
        "outboundIpAddresses": app.outboundIpAddresses,
        "possibleOutboundIpAddresses": app.possibleOutboundIpAddresses,
    }
}

Output of `pulumi about`

CLI          
Version      3.129.0
Go Version   go1.22.6
Go Compiler  gc

Plugins
KIND      NAME          VERSION
resource  azure-native  2.56.0
language  nodejs        unknown

Host     
OS       debian
Version  12.6
Arch     x86_64

This project is written in nodejs: executable='/usr/local/bin/node' version='v20.16.0'

Current Stack: menfin/zendesk/5534

TYPE                                       URN
pulumi:pulumi:Stack                        urn:pulumi:5534::zendesk::pulumi:pulumi:Stack::zendesk-5534
pulumi:providers:azure-native              urn:pulumi:5534::zendesk::pulumi:providers:azure-native::default_2_56_0
azure-native:resources:ResourceGroup       urn:pulumi:5534::zendesk::azure-native:resources:ResourceGroup::resourceGroup
azure-native:web/v20231201:AppServicePlan  urn:pulumi:5534::zendesk::azure-native:resources:ResourceGroup$azure-native:web/v20231201:AppServicePlan::appServicePlan
azure-native:web/v20231201:WebApp          urn:pulumi:5534::zendesk::azure-native:resources:ResourceGroup$azure-native:web/v20231201:WebApp::webApp

Found no pending operations associated with 5534

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/aureq
User           aureq
Organizations  aureq, team-ce, menfin, menfin-team, demo
Token type     personal

Dependencies:
NAME                  VERSION
@pulumi/pulumi        3.129.0
@types/node           18.19.44
typescript            5.5.4
@pulumi/azure-native  2.56.0

Pulumi locates its logs in /tmp by default

Additional context

The verbose logs are in this logs.tar.gz archive.

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

pulumi / pulumi-azure-native