azurerm_web_pubsub_hub resource `Allow anonymous WebSocket clients` not set to true

gonzalojaubert commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

1.2.1

AzureRM Provider Version

3.37.0

Affected Resource(s)/Data Source(s)

azurerm_web_pubsub_hub

Terraform Configuration Files

terraform {
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = ">= 3.37.0"
    }
  }
}

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "tissuerg" {
  name     = "ggj-terraform-issue"
  location = "east us"
}

resource "azurerm_web_pubsub" "tissuewpb" {
  name                = "ggj-webpubsub"
  location            = azurerm_resource_group.tissuerg.location
  resource_group_name = azurerm_resource_group.tissuerg.name

  sku      = "Standard_S1"
  capacity = 1
}

resource "azurerm_web_pubsub_hub" "tissuehub" {
  name          = "ggj_wpsh"
  web_pubsub_id = azurerm_web_pubsub.tissuewpb.id

  event_handler {
    url_template       = "https://test.com/api/{hub}/{event}"
    user_event_pattern = "*"
    system_events      = ["connect", "connected"]
  }

  anonymous_connections_enabled = true

  depends_on = [
    azurerm_web_pubsub.tissuewpb
  ]
}

Debug Output/Panic Output

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Expected Behaviour

Web PusbSub Hub Allow anonymous WebSocket clients value should be true (documentation: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/web_pubsub_hub#anonymous_connections_enabled)

Actual Behaviour

Web PusbSub Hub Allow anonymous WebSocket clients value is false.

In Azure Portal the behaviour is:

In the Settings page, we can see the property as: Allow
In the hub details, the property is disabled and anonymous connection will not work

It looks like that the value anonymousPolicyEnabled is not properly set in https://github.com/hashicorp/terraform-provider-azurerm/blob/main/internal/services/signalr/web_pubsub_hub_resource.go

For deny: Current value is Deny but I think it should be deny
For allow: Current value is Allow but I think it should be allow

Steps to Reproduce

terraform apply

Important Factoids

No response

References

No response

xiaxyi commented 1 year ago

Thanks @gonzalojaubert for raising this issue. From your terraform config, I can see the anonymous_connections_enabled is set to true, then it should allow the anonymous connections, which is the expected behavior.

As for the Allow or allow, I tested the api, it has the same result which allows the anonymous connection.

Let me know if you have any questions.

gonzalojaubert commented 1 year ago

Hey @xiaxyi, yes, the anonymous_connections_enabled is set to true to allow the anonymous connections and in fact, az cli returns Allow as you can see running the following command:

> az webpubsub hub show --name ggj-webpubsub --hub-name ggj_wpsh --resource-group ggj-terraform-issue | jq -r ‘.properties.anonymousConnectPolicy’
Allow

but if you navigate to: "Portal -> Web PubSub Service (ggj-webpubsub) -> Settings -> Edit (ggj_wpsh)", you can see that the option is disabled:

And this is disallowing anonymous connections to the hub.

You can see the same behaviour using ARM templates. If you set anonymousConnectPolicy to Allow in the template, az cli will return Allow but the "Allow anonymous WebSocket clients" option on the hub is still disabled and all the anonymous connections will be rejected. If you change the value to allow in the template, then the option will shown as enabled and the hub will allow anonymous connections.

I guess the value Allow is an accepted value, but the hub is not correctly set by azure until you use allow

Makes sense?

xiaxyi commented 1 year ago

Thanks @gonzalojaubert , I see what you meant, I tested the behavior, part of it does matches with your experience, the option is turned off, but the connection won't be rejected from my side. The service team also confirmed that the property is case-insensitive from the service side, just the portal is not picking up the value with the capital letter. Can you help to check the live trace to see why did the connection got rejected? You can also send us the resource uri and we can try checking from our side about the reject reason.

gonzalojaubert commented 1 year ago

Thanks @xiaxyi! I'm thinking there is another issue in my terraform then. Another issue could be when I update the url-template field in the hub. Not sure about it, but when I change the allow anonymous websocket clients value it start working and the only difference I can see is in the Allow/allow value.

I created a gist with a full example here: https://gist.github.com/gonzalojaubert/aef9d67055071c9bdc6df32df74098f4. You can run it and it should give you the same issue.

This is the live trace output:

"Time","Log Level","Event Name","Message","IP","Http Method","URL","Status Code","Duration","Headers"
"2023/01/11 14:26:01:165","Information","RequestStarted","Request started.","xxxxx","GET","http://ggj-test-webpubsub.webpubsub.azure.com/client/hubs/ggj_test_wpsh","","","{""Connection"":[""Upgrade""],""Upgrade"":[""websocket""],""Content-Length"":[""0""],""x-request-id"":[""xxxxxx""],""Sec-WebSocket-Version"":[""13""],""Sec-WebSocket-Extensions"":[""permessage-deflate; client_max_window_bits""]}"
"2023/01/11 14:26:01:183","Information","RequestProcessed","Request processed.","xxxxx","GET","http://ggj-test-webpubsub.webpubsub.azure.com/client/hubs/ggj_test_wpsh","401","20.083516","{""Connection"":[""Upgrade""],""Upgrade"":[""websocket""],""Content-Length"":[""0""],""x-request-id"":[""xxxxx""],""Sec-WebSocket-Version"":[""13""],""Sec-WebSocket-Extensions"":[""permessage-deflate; client_max_window_bits""]}"

xiaxyi commented 1 year ago

Thanks @gonzalojaubert for the information, we tested the your deployment code and the hub can be connected as expected. However, during our testing, there is one tiny step that may cause the hub to get disconnected: when you update the functions key to the hub's URL template in azure portal, the anonymous connection may be turned off since portal only matches the value "allow", so can you confirm anonymous connection does not showed as below screenshot after you updated the url tempate? (it's ok for the Allow value)

xiaxyi commented 1 year ago

Our web pubsub engineering team also aware of this portal issue and they are working on the fix, unfortunately, we may not be able to provide any ETA of the fix, I will keep you updated. Feel free to let us know if you have any questions or concerns.

gonzalogarciajaubert commented 1 year ago

Thank you very much for reviewing this @xiaxyi .

I can confirm that the same thing is happening to me and it would explain why updating the url-template from the portal disables anonymous access. As in the portal, the anonymous access is disabled, when updating the url-template and clicking on save the value changes to Deny.

If the engineering team manages to solve this problem it would be great.

Sorry to keep bothering you, but in the meantime, the other option I have tried is not to change the URL template from the portal but from the Terraform file itself. It seems that this is not possible since the value of the key webpubsub_key is always null.

Here is an example:

data "azurerm_function_app_host_keys" "tissuehk" {
  name                = azurerm_windows_function_app.tissuefa.name
  resource_group_name = azurerm_resource_group.tissuerg.name
  depends_on = [
    azurerm_function_app_function.tissuefaf
  ]
}

output "data_key" {
  sensitive = true
  value = data.azurerm_function_app_host_keys.tissuehk.webpubsub_extension_key
}

The data_key value is empty.

> terraform output -raw data_key

I tried with a time_sleep also:

resource "time_sleep" "tissuehksleep" {
  create_duration = "60s"

  triggers = {
    sleep_extension_key = data.azurerm_function_app_host_keys.tissuehk.webpubsub_extension_key
  }
}

output "sleep_key" {
  sensitive = true
  value = time_sleep.tissuehksleep.triggers["sleep_extension_key"]
}

And the sleep_key is also null.

> terraform output -raw sleep_key

however, az-cli returns the expected value:

> az functionapp keys list -g ggj-test-rg -n ggjfunction-app | jq -r '.systemKeys.webpubsub_extension'
XXXXXXXXXX

Is it the expected behavior?

xiaxyi commented 1 year ago

No problems @gonzalogarciajaubert

I tried the data source and it's working as expected. Can you try my TF config:


resource "azurerm_web_pubsub_hub" "tissuehub" {
  name          = "xiaxin_test_wpsh"
  web_pubsub_id = azurerm_web_pubsub.tissuewpb.id

  event_handler {
    url_template       = "https://xiaxinfunction-app.azurewebsites.net/runtime/webhooks/webpubsub?code=${data.azurerm_function_app_host_keys.test.webpubsub_extension_key}"
    user_event_pattern = "*"
    system_events      = ["connect", "connected", "disconnected"]
  }

  anonymous_connections_enabled = true

  depends_on = [
    azurerm_web_pubsub.tissuewpb
  ]
}

gonzalogarciajaubert commented 1 year ago

@xiaxyi, I tried it, but the webpubsub_extension_key attribute is still null. Have you tried starting from scratch? I mean, with a new functionApp or a new resource group?. If I try to use it from scratch the first time is null. And then, if I apply it again, then the attribute has the expected value.

gonzalojaubert commented 1 year ago

@xiaxyi I made it works but I had to use some workarounds to get the code from the function.

This doesn’t looks like to work properly:

// Data read after function creation
data "azurerm_function_app_host_keys" "tissuehk" {
  name                = azurerm_windows_function_app.tissuefa.name
  resource_group_name = azurerm_resource_group.tissuerg.name
  depends_on = [
    azurerm_resource_group.tissuerg, azurerm_windows_function_app.tissuefa, azurerm_function_app_function.tissuefaf
  ]
}

// Then wait for the webpubsub_extension_key update
resource "time_sleep" "tissuehksleep" {
  create_duration = "5m"

  triggers = {
    sleep_extension_key = data.azurerm_function_app_host_keys.tissuehk.webpubsub_extension_key
  }
}

// Data was not updated, the webpubsub_extension_key is null
output "function_host_keys_output" {
  sensitive = true
  value = data.azurerm_function_app_host_keys.tissuehk.webpubsub_extension_key
}

The only way I found for make it works is to wait X minutes after the function is deployed and then create the data source

// Sleep 5m after Function creation
resource "time_sleep" "tissuehksleep" {
  create_duration = "5m"
  depends_on = [ azurerm_function_app_function.tissuefaf ]
}

// Then read the data
data "azurerm_function_app_host_keys" "tissuehk" {
  name                = azurerm_windows_function_app.tissuefa.name
  resource_group_name = azurerm_resource_group.tissuerg.name
  depends_on = [
    time_sleep.tissuehksleep
  ]
}

// Data contains the expected webpubsub_extension_key
output "function_host_keys_output" {
  sensitive = true
  value = data.azurerm_function_app_host_keys.tissuehk.webpubsub_extension_key
}

Questions:

Shouldn’t the data source defer reading the data resource until webpubsub_extension_key is available?
If not, should the trigger force the data store to be queried again in intervals as we are waiting for a trigger that refers to it?
Is there any other way to wait for the webpubsub_extension_key to be created?

hashicorp / terraform-provider-azurerm