Closed favoretti closed 7 months ago
tombuildsstuff
commented
3 years ago Next to it, there's a question why a GA service (https://techcommunity.microsoft.com/t5/azure-database-for-postgresql/azure-database-for-postgresql-hyperscale-citus-now-generally/ba-p/1014865) still using private preview API?
cc @JeffreyRichter for this one
ghost
commented
3 years ago Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @sunilagarwal, @lfittl-msft, @sr-msft, @niklarin.
| Author: | favoretti |
|---|---|
| Assignees: | - |
| Labels: | `PostgreSQL`, `Service Attention`, `customer-reported`, `needs-triage`, `question` |
| Milestone: | - |
ArcturusZhang
commented
3 years ago Hi @favoretti and @tombuildsstuff thanks for this issue!
I will make some internal communication to get an answer/update for this. Thanks
niklarin
commented
3 years ago @favoretti thank you very much for interest in Hyperscale (Citus)! As you rightfully noticed the product is young and is growing very fast.
You're right, REST APIs are not released publicly yet (are in private preview). Due to multiple reasons we had to delay REST APIs release until we can be certain we have a final version. Please bear with us as we will be looking for any opportunity to finally publish them as soon as we can in the next few months.
To release Azure SDKs support for Hyperscale (Citus) we need to have the final version of APIs first. Once SDKs support is released we will be working with our partners at Terraform to add Hyperscale (Citus) support.
If you have any questions about Hyperscale (Citus) or would like to discuss your case with us, the Hyperscale (Citus) product development team, please feel free to email us at AskAzureDBforPostgreSQL@service.microsoft.com.
Kind regards, Nik
Hyperscale (Citus) product
favoretti
commented
3 years ago @niklarin Thank you for your elaborate answer. Since I'm fairly active on azurerm terraform provider development - one could say I can be your terraform partner in implementation. :) We are using Citus currently in our projects and have to band-aid it with ARM templates wrapped in terraform for now, which works, but is sub-optimal.
I appreciate that API needs to be stabilized, however, since portal uses this API currently as well and one is able to use ARM to deploy and manage those resources and, what's IMO more important in this case, the product itself has been GA for over a year now - would it be possible to publish at least a public preview of this SDK as is? I do realize all the risks of breaking changes in the coming months of course.
cmahlert
commented
2 years ago It's pretty frustrating that nothing is happening here.
@niklarin: Can you give an approximate estimate of when the REST APIs will be released? The year is almost over again.
@favoretti: I'd be interested in your "band-aid solution". Is there any way you could share it?
favoretti
commented
2 years ago @cmahlert basically an ARM template wrapped in TF. Happy to share tomorrow my morning, have just my phone with me atm.
favoretti
commented
2 years ago @cmahlert
module that refers to labels is terraform-null-label
resource "azurerm_resource_group" "citus" {
name = module.citus_poc_labels.id_with_suffix.rg
location = module.citus_poc_labels.location
tags = module.citus_poc_labels.tags
}
resource "random_password" "citus_poc" {
length = 32
upper = true
min_upper = 1
lower = true
min_lower = 1
number = true
min_numeric = 1
special = false
min_special = 0
}
resource "azurerm_monitor_diagnostic_setting" "citus" {
name = format("%s-citus-ds", module.citus_poc_labels.id)
target_resource_id = jsondecode(azurerm_resource_group_template_deployment.citus_poc.output_content).serverGroupId.value
log_analytics_workspace_id = local.law.primary.id
dynamic "log" {
for_each = toset(["PostgreSQLLogs"])
content {
category = log.key
enabled = true
retention_policy {
enabled = true
days = 30
}
}
}
dynamic "metric" {
for_each = toset(["AllMetrics"])
content {
category = metric.key
enabled = true
retention_policy {
enabled = true
days = 30
}
}
}
}
resource "azurerm_resource_group_template_deployment" "citus_poc" {
name = "citus-poc"
resource_group_name = azurerm_resource_group.citus.name
deployment_mode = "Complete"
parameters_content = local.citus_poc_parameters
template_content = local.citus_poc_arm
}
locals {
citus_poc_parameters = jsonencode({
administratorLogin = { value = "citus" }
administratorLoginPassword = { value = random_password.citus_poc.result }
location = { value = azurerm_resource_group.citus.location }
serverGroup = { value = module.citus_poc_labels.id }
coordinatorVcores = { value = 16 }
coordinatorStorageSizeMB = { value = 524288 }
workerVcores = { value = 8 }
workerStorageSizeMB = { value = 524288 }
numWorkers = { value = 8 }
serverGroupTags = { value = module.citus_poc_labels.tags }
workerNodeTags = { value = module.citus_poc_labels.tags }
coordinatorNodeTags = { value = module.citus_poc_labels.tags }
vnetData = { value = {} }
standbyCount = { value = 0 }
enableHa = { value = false }
previewFeatures = { value = true }
pgVersion = { value = "13" }
logMinDurationStatement = { value = "1" }
trackActivityQuerySize = { value = "1000000" }
firewallRules = {
value = {
rules = [
{
name = "AllowAllAzureServicesAndResourcesWithinAzureIps_2021-7-6_11-19-4",
endIPAddress = "0.0.0.0",
startIPAddress = "0.0.0.0"
},
]
}
}
})
citus_poc_arm = jsonencode({
"$schema" = "http://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#"
"contentVersion" = "1.0.0.0"
"parameters" = {
"administratorLogin" = {
"type" = "String"
}
"administratorLoginPassword" = {
"type" = "SecureString"
}
"location" = {
"type" = "String"
}
"serverGroup" = {
"type" = "String"
}
"coordinatorVcores" = {
"type" = "Int"
}
"coordinatorStorageSizeMB" = {
"type" = "Int"
}
"workerVcores" = {
"type" = "Int"
}
"workerStorageSizeMB" = {
"type" = "Int"
}
"numWorkers" = {
"type" = "Int"
}
"serverGroupTags" = {
"type" = "Object"
"defaultValue" = {}
}
"workerNodeTags" = {
"type" = "Object"
"defaultValue" = {}
}
"coordinatorNodeTags" = {
"type" = "Object"
"defaultValue" = {}
}
"firewallRules" = {
"type" = "Object"
"defaultValue" = {}
}
"vnetData" = {
"type" = "Object"
"defaultValue" = {}
}
"standbyCount" = {
"type" = "Int"
"defaultValue" = 0
}
"enableHa" = {
"type" = "Bool"
"defaultValue" = false
}
"previewFeatures" = {
"type" = "Bool"
"defaultValue" = false
}
"pgVersion" = {
"type" = "String"
}
"logMinDurationStatement" = {
"type" = "String"
}
"trackActivityQuerySize" = {
"type" = "String"
}
}
"variables" = {
"api" = "2020-10-05-privatepreview"
"serverGroupName" = "[parameters('serverGroup')]"
"firewallRules" = "[parameters('firewallRules').rules]"
"enablePublicIp" = "[if(empty(parameters('vnetData')), bool('true'), bool('false'))]"
"vnetDataSet" = "[if(empty(parameters('vnetData')), json('{ \"subnetArmResourceId\": \"\" }'), parameters('vnetData'))]"
"finalVnetData" = "[json(concat('{ \"SubnetArmResourceId\": \"', variables('vnetDataSet').subnetArmResourceId, '\"}'))]"
}
"resources" = [
{
"apiVersion" = "[variables('api')]"
"location" = "[parameters('location')]"
"name" = "[variables('serverGroupName')]"
"tags" = "[parameters('serverGroupTags')]"
"type" = "Microsoft.DBforPostgreSQL/serverGroupsv2"
"properties" = {
"createMode" = "Default"
"administratorLogin" = "citus"
"administratorLoginPassword" = "[parameters('administratorLoginPassword')]"
"backupRetentionDays" = 35
"DelegatedSubnetArguments" = "[if(empty(parameters('vnetData')), json('null'), variables('finalVnetData'))]"
"enableMx" = false
"enableZfs" = false
"previewFeatures" = "[parameters('previewFeatures')]"
"postgresqlVersion" = "[parameters('pgVersion')]"
"logMinDurationStatement" = "[parameters('logMinDurationStatement')]"
"trackActivityQuerySize" = "[parameters('trackActivityQuerySize')]"
"serverRoleGroups" = [
{
"name" = ""
"role" = "Coordinator"
"serverCount" = 1
"serverEdition" = "GeneralPurpose"
"vCores" = "[parameters('coordinatorVcores')]"
"storageQuotaInMb" = "[parameters('coordinatorStorageSizeMB')]"
"enableHa" = "[parameters('enableHa')]"
"logMinDurationStatement" = "[parameters('logMinDurationStatement')]"
"logStatement" = "All"
"randomPageCost" = 4
},
{
"name" = ""
"role" = "Worker"
"serverCount" = "[parameters('numWorkers')]"
"serverEdition" = "MemoryOptimized"
"vCores" = "[parameters('workerVcores')]"
"storageQuotaInMb" = "[parameters('workerStorageSizeMB')]"
"enableHa" = "[parameters('enableHa')]"
"logMinDurationStatement" = "[parameters('logMinDurationStatement')]"
"logStatement" = "All"
"randomPageCost" = 4
},
]
}
},
{
"condition" = "[greater(length(variables('firewallRules')), 0)]"
"type" = "Microsoft.Resources/deployments"
"apiVersion" = "2019-08-01"
"name" = "[concat(variables('serverGroupName'), concat('-firewallRules-', copyIndex()))]"
"copy" = {
"count" = "[if(greater(length(variables('firewallRules')), 0), length(variables('firewallRules')), 1)]"
"mode" = "Serial"
"name" = "firewallRulesIterator"
}
"dependsOn" = [
"[concat('Microsoft.DBforPostgreSQL/serverGroupsv2/', variables('serverGroupName'))]"
]
"properties" = {
"mode" = "Incremental"
"template" = {
"$schema" = "http://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#"
"contentVersion" = "1.0.0.0"
"resources" = [
{
"type" = "Microsoft.DBforPostgreSQL/serverGroupsv2/firewallRules"
"name" = "[concat(variables('serverGroupName'),'/',variables('firewallRules')[copyIndex()].name)]"
"apiVersion" = "[variables('api')]"
"properties" = {
"startIpAddress" = "[variables('firewallRules')[copyIndex()].startIPAddress]"
"endIpAddress" = "[variables('firewallRules')[copyIndex()].endIPAddress]"
}
}
]
}
}
},
]
"outputs" = {
"serverGroupId" : {
"type" : "String",
"value" : "[resourceId('Microsoft.DBforPostgreSQL/serverGroupsv2/', variables('serverGroupName'))]"
}
}
})
}
resource "azurerm_private_endpoint" "citus" {
name = format("%s-pendp", module.citus_poc_labels.id)
location = module.citus_poc_labels.location
resource_group_name = azurerm_resource_group.this["primary"].name
subnet_id = local.network[module.citus_poc_labels.location].main.subnets.citus.id
private_dns_zone_group {
name = format("%s-pendp", module.citus_poc_labels.id)
private_dns_zone_ids = [azurerm_private_dns_zone.citus.id]
}
private_service_connection {
name = format("%s-psc", module.citus_poc_labels.id)
private_connection_resource_id = jsondecode(azurerm_resource_group_template_deployment.citus_poc.output_content).serverGroupId.value
subresource_names = ["coordinator"]
is_manual_connection = false
}
tags = module.citus_poc_labels.tags
}
resource "azurerm_private_dns_zone" "citus" {
name = format("privatelink.%s.postgres.database.azure.com", module.citus_poc_labels.id)
resource_group_name = azurerm_resource_group.this["primary"].name
tags = module.citus_poc_labels.tags
}
resource "azurerm_private_dns_zone_virtual_network_link" "cituseastus2" {
name = "citus-main"
resource_group_name = azurerm_resource_group.this["primary"].name
private_dns_zone_name = azurerm_private_dns_zone.citus.name
virtual_network_id = local.network[module.eastus2_labels.location].main.id
}
resource "azurerm_key_vault_secret" "citus_login_password" {
name = "citus-login-password"
value = random_password.citus_poc.result
key_vault_id = module.kv_env["dev"].id
}
github-actions[bot]
commented
7 months ago Hi @favoretti, we deeply appreciate your input into this project. Regrettably, this issue has remained inactive for over 2 years, leading us to the decision to close it. We've implemented this policy to maintain the relevance of our issue queue and facilitate easier navigation for new contributors. If you still believe this topic requires attention, please feel free to create a new issue, referencing this one. Thank you for your understanding and ongoing support.
Service Package Request
Could we please add support for PostgreSQL Hyperscale (Citus)? The service has been GA for a while, however REST API specs are still in privatepreview mode (https://github.com/Azure/azure-rest-api-specs/blob/b39d3318a9cba7203da3d56e4cdc69a7e14d3912/specification/postgresqlhsc/resource-manager/Microsoft.DBforPostgreSQL/preview/2020-10-05-privatepreview/postgresqlhsc.json)
Next to it, there's a question why a GA service (https://techcommunity.microsoft.com/t5/azure-database-for-postgresql/azure-database-for-postgresql-hyperscale-citus-now-generally/ba-p/1014865) still using private preview API?
All in all, we would very much like to have this supported by terraform, but without SDK support it makes it hard of course, so would be much appreciated if someone could generate at least a privatepreview SDK.
Thanks!
/cc @ArcturusZhang