search

pulumi / pulumi-azure-native

Azure Native Provider
Apache License 2.0
129 stars 35 forks source link

Creating a database for a PostgreSQL server fails #2916

Open mhauru opened 1 year ago

mhauru commented 1 year ago

What happened?

I have a pulumi config that creates a flexible PostgreSQL server and a database for it. When running pulumi up, the creation of the database fails with

error: resource partially created but read failed autorest/azure: Service returned an error. Status=404
Code="ResourceNotFound" Message="The requested resource of type 'Microsoft.DBforPostgreSQL/flexibleServers/databases'
with name 'dtdb' was not found.": Code="InternalServerError" Message="An unexpected error occured while processing
the request. Tracking ID: 'ec2149d3-f55b-4c5d-8a83-a416f2be417c'"

The PostgreSQL server itself is fine, and I can use e.g. psql to create the dtdb database manually.

Example

The relevant parts of my Python config file are these:

import pulumi_azure_native.dbforpostgresql as postgresql
import pulumi_azure_native.resources as resource
from pulumi import Config, Output, export

CONFIG = Config()
RESOURCE_NAME_PREFIX = CONFIG.get("resource-name-prefix")
SQL_SERVER_USER = "dbadmin"
SQL_DB_NAME = "dtdb"
SQL_SERVER_PASSWORD = CONFIG.require("sql-server-password")

def create_sql_server(resource_group: resource.ResourceGroup) -> postgresql.Server:
    sql_server_name = f"{RESOURCE_NAME_PREFIX}-postgresql"
    sql_server = postgresql.Server(
        sql_server_name,
        server_name=sql_server_name,
        resource_group_name=resource_group.name,
        administrator_login=SQL_SERVER_USER,
        administrator_login_password=SQL_SERVER_PASSWORD,
        create_mode="Default",
        storage=postgresql.StorageArgs(storage_size_gb=32),
        backup=postgresql.BackupArgs(
            backup_retention_days=14, geo_redundant_backup="Disabled"
        ),
        sku=postgresql.SkuArgs(tier="Burstable", name="Standard_B1ms"),
        availability_zone="1",
        version="16",
    )
    return sql_server

def create_pg_database(
    resource_group: resource.ResourceGroup, sql_server: postgresql.Server
) -> postgresql.Database:
    pg_database = postgresql.Database(
        SQL_DB_NAME,
        charset="UTF8",
        collation="English_United States.1252",
        database_name=SQL_DB_NAME,
        resource_group_name=resource_group.name,
        server_name=sql_server.name,
    )
    return pg_database

resource_group = resource.ResourceGroup(f"{RESOURCE_NAME_PREFIX}-rg")
sql_server = create_sql_server(resource_group)
create_pg_database(resource_group, sql_server)

The full config is here: https://github.com/alan-turing-institute/DTBase/blob/update-pulumi-config/infrastructure/__main__.py

Output of pulumi about

CLI
Version      3.94.2
Go Version   go1.21.3
Go Compiler  gc

Plugins
NAME          VERSION
azure-native  2.20.0
python        unknown

Host
OS       darwin
Version  13.6.2
Arch     arm64

This project is written in python: executable='/Users/mhauru/projects/DTBase/.venv/bin/python3' version='3.10.13'

Current Stack: test

TYPE                                       URN
pulumi:pulumi:Stack                        urn:pulumi:test::dtbase-azure::pulumi:pulumi:Stack::dtbase-azure-test
pulumi:providers:azure-native              urn:pulumi:test::dtbase-azure::pulumi:providers:azure-native::default_2_20_0
azure-native:resources:ResourceGroup       urn:pulumi:test::dtbase-azure::azure-native:resources:ResourceGroup::dtbasetest-rg
azure-native:insights:Component            urn:pulumi:test::dtbase-azure::azure-native:insights:Component::dtbasetest-web-ai
azure-native:dbforpostgresql:Server        urn:pulumi:test::dtbase-azure::azure-native:dbforpostgresql:Server::dtbasetest-postgresql
azure-native:web:AppServicePlan            urn:pulumi:test::dtbase-azure::azure-native:web:AppServicePlan::dtbasetest-web-asp
azure-native:storage:StorageAccount        urn:pulumi:test::dtbase-azure::azure-native:storage:StorageAccount::dtbasetestsa
azure-native:web:WebApp                    urn:pulumi:test::dtbase-azure::azure-native:web:WebApp::dtbasetest-backend
azure-native:dbforpostgresql:FirewallRule  urn:pulumi:test::dtbase-azure::azure-native:dbforpostgresql:FirewallRule::dtbasetest-fwrb592
azure-native:dbforpostgresql:FirewallRule  urn:pulumi:test::dtbase-azure::azure-native:dbforpostgresql:FirewallRule::dtbasetest-fwr398a
azure-native:dbforpostgresql:FirewallRule  urn:pulumi:test::dtbase-azure::azure-native:dbforpostgresql:FirewallRule::dtbasetest-fwr3436
azure-native:dbforpostgresql:FirewallRule  urn:pulumi:test::dtbase-azure::azure-native:dbforpostgresql:FirewallRule::dtbasetest-fwr0ea9
azure-native:dbforpostgresql:FirewallRule  urn:pulumi:test::dtbase-azure::azure-native:dbforpostgresql:FirewallRule::dtbasetest-fwr7233
azure-native:dbforpostgresql:FirewallRule  urn:pulumi:test::dtbase-azure::azure-native:dbforpostgresql:FirewallRule::dtbasetest-fwr2795
azure-native:dbforpostgresql:FirewallRule  urn:pulumi:test::dtbase-azure::azure-native:dbforpostgresql:FirewallRule::dtbasetest-fwr01ec
azure-native:web:WebApp                    urn:pulumi:test::dtbase-azure::azure-native:web:WebApp::dtbasetest-frontend
pulumi:providers:azure-native              urn:pulumi:test::dtbase-azure::pulumi:providers:azure-native::default_2_17_0

Found no pending operations associated with test

Backend
Name           403-Q4WFPXDFN0
URL            azblob://dtbasepulumibackend
User           mhauru
Organizations
Token type     personal

Dependencies:
NAME                 VERSION
black                23.11.0
dtbase               0.1.0
lazy-object-proxy    1.9.0
msrest               0.7.1
pip                  23.1.2
pre-commit           3.5.0
pulumi-azure-native  2.20.0
py                   1.11.0
pylint               3.0.2
pytest-cov           4.1.0
ruff                 0.1.5
wrapt                1.15.0

Pulumi locates its logs in /var/folders/wk/zmsrlr9s2cgdpdnqj5d522sw0000gr/T/ by default

Additional context

Very similar to https://github.com/pulumi/pulumi-azure-native/issues/1921, but the workaround used there does not work for me: Changing the SKU doesn't fix the problem.

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

mjeffryes commented 1 year ago

I had to hunt around a bit to find a region where I could get azure to let me provision this at all, but I did manage to get a reproduction in useast. https://github.com/mjeffryes/python-repros/commit/41eb3a93dc379c321c585eac03caa670105094da

I also confirmed that I could create the database from the azure command line tool. Attaching some logs from the failed create. logs.txt

mhauru commented 8 months ago

I tried running the REST API request for creating a database manually using this, but couldn't get it to work.

mhauru commented 8 months ago

@Iain-S just pointed out that removing the collation="English_United States.1252" makes this problem go away.