search

Azure / azure-sdk-for-python

This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/python/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-python.
MIT License
4.53k stars 2.76k forks source link

The latest azure.mgmt.rdbms returns publicNetworkAccess = Enabled whereas latest az cli returns publicNetworkAccess = Disabled #36459

Closed antonkazlouski-orca closed 1 month ago

antonkazlouski-orca commented 2 months ago

Describe the bug list_by_resource_group command of the latest azure.mgmt.rdbms returnspublicNetworkAccess = Enabled whereas latest az cli returns publicNetworkAccess = Disabled.

To Reproduce Steps to reproduce the behavior:

  1. Create a Resource Group: Use the Azure CLI to create a resource group.
  2. Create a MySQL Flexible Server: Use the Azure CLI to create a MySQL Flexible Server within the resource group.
  3. Set Up Azure AD Application and Service Principal:
  4. Create an Azure AD application.
  5. Create a service principal for the application.
  6. Assign the Contributor role to the service principal for the subscription.
  7. 
import logging
from azure.identity import ClientSecretCredential
from azure.mgmt.rdbms.mysql_flexibleservers import MySQLManagementClient

logger = logging.getLogger('azure') logger.setLevel(logging.DEBUG)

ch = logging.StreamHandler() ch.setLevel(logging.DEBUG) logger.addHandler(ch)

subscription_id = "" tenant_id = "" client_id = "" client_secret = ""

resource_group_name = '***' credential = ClientSecretCredential( tenant_id=tenant_id, client_id=client_id, client_secret=client_secret ) client = MySQLManagementClient(credential, subscription_id=subscription_id, logging_enable=True)

servers = client.servers.list_by_resource_group(resource_group_name) server = next(servers)

8. Check the value of `server` and find `publicNetworkAccess` attribute.
9. Compare the results with the output from

az mysql flexible-server list --resource-group --debug --subscription 

10. Compare the results with the output from HTTP API calls.

Obtain token

curl -X POST https://login.microsoftonline.com//oauth2/v2.0/token \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "grant_type=client_credentials&client_id=&client_secret=&scope=https%3A%2F%2Fmanagement.azure.com%2F.default"

Request list MySQL servers with api-version=2021-05-01 curl -X GET https://management.azure.com/subscriptions//resourceGroups//providers/Microsoft.DBforMySQL/flexibleServers?api-version=2021-05-01 \ -H "Authorization: Bearer "

Now do the same for api-version=2024-02-01-preview https://management.azure.com/subscriptions//resourceGroups//providers/Microsoft.DBforMySQL/flexibleServers?api-version=2024-02-01-preview \ -H "Authorization: Bearer " 



**Expected behavior**
I would expect to have the same data no matter which API and client is used. What happens is that Python SDK returns `publicNetworkAccess=Enabled` whereas latest az client and HTTP API calls to the latest version return `publicNetworkAccess=Disabled`.

Attaching response examples.
[az_issue.txt](https://github.com/user-attachments/files/16192913/az_issue.txt)
xiangyan99 commented 2 months ago

Thanks for the feedback, we’ll investigate asap.

msyyc commented 2 months ago

@ChenxiJiang333 Please help on this issue

ChenxiJiang333 commented 1 month ago

got it

ChenxiJiang333 commented 1 month ago

Hi @antonkazlouski-orca First, could you visit the Azure Portal to check and share the actual public access setting of your resource? For we need to confirm which behavior is incorrect.

The issue you mentioned that Python SDK makes different behavior from other interactive method seems impossible since Python SDK is also a tool to send/receive API calls, so it should return values just the same as HTTP API calls returns. And by the log you provided, it seems more likely that you got the different outputs for you have called two different api versions: 2024-02-01-preview & 2021-05-01. Could you try get the server info again through Python SDK with specified api version 2024-02-01-preview by updating the code to:

client = MySQLManagementClient(credential, subscription_id=subscription_id, logging_enable=True, api_version="2024-02-01-preview")

and see whether there would be any change on the publicNetworkAccess's value?

github-actions[bot] commented 1 month ago

Hi @antonkazlouski-orca. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

antonkazlouski-orca commented 1 month ago

Hi @xiangyan99 !

First, could you visit the Azure Portal to check and share the actual public access setting of your resource? For we need to confirm which behavior is incorrect.

I've requested this info as I'm unable to access the UI of the customer.

As for the rest, I've already tried to specify api_version in Python SDK but I don't see any effect of it.

Request URL: 'https://login.microsoftonline.com/***/v2.0/.well-known/openid-configuration'
Request method: 'GET'
Request headers:
    'User-Agent': 'azsdk-python-identity/1.12.0 Python/3.11.4 (Linux-5.10.219-208.866.amzn2.x86_64-x86_64-with-glibc2.36)'
No body was attached to the request
Request URL: 'https://login.microsoftonline.com/***/v2.0/.well-known/openid-configuration'
Request method: 'GET'
Request headers:
    'User-Agent': 'azsdk-python-identity/1.12.0 Python/3.11.4 (Linux-5.10.219-208.866.amzn2.x86_64-x86_64-with-glibc2.36)'
No body was attached to the request
Response status: 200
Response headers:
    'Cache-Control': 'max-age=86400, private'
    'Content-Type': 'application/json; charset=utf-8'
    'Strict-Transport-Security': 'REDACTED'
    'X-Content-Type-Options': 'REDACTED'
    'Access-Control-Allow-Origin': 'REDACTED'
    'Access-Control-Allow-Methods': 'REDACTED'
    'P3P': 'REDACTED'
    'x-ms-request-id': 'f47a437f-d023-4839-a8c5-58441ebe7f00'
    'x-ms-ests-server': 'REDACTED'
    'x-ms-srs': 'REDACTED'
    'X-XSS-Protection': 'REDACTED'
    'Set-Cookie': 'REDACTED'
    'Date': 'Mon, 22 Jul 2024 06:50:37 GMT'
    'Content-Length': '1753'
Response status: 200
Response headers:
    'Cache-Control': 'max-age=86400, private'
    'Content-Type': 'application/json; charset=utf-8'
    'Strict-Transport-Security': 'REDACTED'
    'X-Content-Type-Options': 'REDACTED'
    'Access-Control-Allow-Origin': 'REDACTED'
    'Access-Control-Allow-Methods': 'REDACTED'
    'P3P': 'REDACTED'
    'x-ms-request-id': 'f47a437f-d023-4839-a8c5-58441ebe7f00'
    'x-ms-ests-server': 'REDACTED'
    'x-ms-srs': 'REDACTED'
    'X-XSS-Protection': 'REDACTED'
    'Set-Cookie': 'REDACTED'
    'Date': 'Mon, 22 Jul 2024 06:50:37 GMT'
    'Content-Length': '1753'
Region configured (None) != region detected ('eu-central-1')
Request URL: 'https://login.microsoftonline.com/***/oauth2/v2.0/token'
Request method: 'POST'
Request headers:
    'Accept': 'application/json'
    'x-client-sku': 'REDACTED'
    'x-client-ver': 'REDACTED'
    'x-client-os': 'REDACTED'
    'x-client-cpu': 'REDACTED'
    'x-ms-lib-capability': 'REDACTED'
    'client-request-id': 'REDACTED'
    'x-client-current-telemetry': 'REDACTED'
    'x-client-last-telemetry': 'REDACTED'
    'User-Agent': 'azsdk-python-identity/1.12.0 Python/3.11.4 (Linux-5.10.219-208.866.amzn2.x86_64-x86_64-with-glibc2.36)'
A body is sent with the request
Request URL: 'https://login.microsoftonline.com/***/oauth2/v2.0/token'
Request method: 'POST'
Request headers:
    'Accept': 'application/json'
    'x-client-sku': 'REDACTED'
    'x-client-ver': 'REDACTED'
    'x-client-os': 'REDACTED'
    'x-client-cpu': 'REDACTED'
    'x-ms-lib-capability': 'REDACTED'
    'client-request-id': 'REDACTED'
    'x-client-current-telemetry': 'REDACTED'
    'x-client-last-telemetry': 'REDACTED'
    'User-Agent': 'azsdk-python-identity/1.12.0 Python/3.11.4 (Linux-5.10.219-208.866.amzn2.x86_64-x86_64-with-glibc2.36)'
A body is sent with the request
Response status: 200
Response headers:
    'Cache-Control': 'no-store, no-cache'
    'Pragma': 'no-cache'
    'Content-Type': 'application/json; charset=utf-8'
    'Expires': '-1'
    'Strict-Transport-Security': 'REDACTED'
    'X-Content-Type-Options': 'REDACTED'
    'P3P': 'REDACTED'
    'client-request-id': 'REDACTED'
    'x-ms-request-id': 'f60e9ede-c278-4251-b591-c20661f64b00'
    'x-ms-ests-server': 'REDACTED'
    'x-ms-clitelem': 'REDACTED'
    'x-ms-srs': 'REDACTED'
    'X-XSS-Protection': 'REDACTED'
    'Set-Cookie': 'REDACTED'
    'Date': 'Mon, 22 Jul 2024 06:50:38 GMT'
    'Content-Length': '1488'
Response status: 200
Response headers:
    'Cache-Control': 'no-store, no-cache'
    'Pragma': 'no-cache'
    'Content-Type': 'application/json; charset=utf-8'
    'Expires': '-1'
    'Strict-Transport-Security': 'REDACTED'
    'X-Content-Type-Options': 'REDACTED'
    'P3P': 'REDACTED'
    'client-request-id': 'REDACTED'
    'x-ms-request-id': 'f60e9ede-c278-4251-b591-c20661f64b00'
    'x-ms-ests-server': 'REDACTED'
    'x-ms-clitelem': 'REDACTED'
    'x-ms-srs': 'REDACTED'
    'X-XSS-Protection': 'REDACTED'
    'Set-Cookie': 'REDACTED'
    'Date': 'Mon, 22 Jul 2024 06:50:38 GMT'
    'Content-Length': '1488'
ClientSecretCredential.get_token succeeded
ClientSecretCredential.get_token succeeded
Request URL: 'https://management.azure.com/subscriptions/***/resourceGroups/***/providers/Microsoft.DBForMySql/flexibleServers?api-version=2020-07-01-preview'
Request method: 'GET'
Request headers:
    'Accept': 'application/json'
    'x-ms-client-request-id': 'b4419c20-47f6-11ef-9f08-f6888e3f0086'
    'User-Agent': 'azsdk-python-mgmt-rdbms/unknown Python/3.11.4 (Linux-5.10.219-208.866.amzn2.x86_64-x86_64-with-glibc2.36)'
    'Authorization': 'Bearer ***'
Request body:
This request has no body
Request URL: 'https://management.azure.com/subscriptions/***/resourceGroups/***/providers/Microsoft.DBForMySql/flexibleServers?api-version=2020-07-01-preview'
Request method: 'GET'
Request headers:
    'Accept': 'application/json'
    'x-ms-client-request-id': 'b4419c20-47f6-11ef-9f08-f6888e3f0086'
    'User-Agent': 'azsdk-python-mgmt-rdbms/unknown Python/3.11.4 (Linux-5.10.219-208.866.amzn2.x86_64-x86_64-with-glibc2.36)'
    'Authorization': 'Bearer ***'
Request body:
This request has no body
Request URL: 'https://management.azure.com/subscriptions/***/resourceGroups/***/providers/Microsoft.DBForMySql/flexibleServers?api-version=REDACTED'
Request method: 'GET'
Request headers:
    'Accept': 'application/json'
    'x-ms-client-request-id': 'b4419c20-47f6-11ef-9f08-f6888e3f0086'
    'User-Agent': 'azsdk-python-mgmt-rdbms/unknown Python/3.11.4 (Linux-5.10.219-208.866.amzn2.x86_64-x86_64-with-glibc2.36)'
    'Authorization': 'REDACTED'
No body was attached to the request
Request URL: 'https://management.azure.com/subscriptions/***/resourceGroups/***/providers/Microsoft.DBForMySql/flexibleServers?api-version=REDACTED'
Request method: 'GET'
Request headers:
    'Accept': 'application/json'
    'x-ms-client-request-id': 'b4419c20-47f6-11ef-9f08-f6888e3f0086'
    'User-Agent': 'azsdk-python-mgmt-rdbms/unknown Python/3.11.4 (Linux-5.10.219-208.866.amzn2.x86_64-x86_64-with-glibc2.36)'
    'Authorization': 'REDACTED'
No body was attached to the request
Response status: 200
Response headers:
    'Cache-Control': 'no-cache'
    'Pragma': 'no-cache'
    'Content-Length': '1331'
    'Content-Type': 'application/json; charset=utf-8'
    'Expires': '-1'
    'X-Content-Type-Options': 'REDACTED'
    'x-ms-request-id': '670a55d1-d434-45fb-b9ef-13a4bb4d5110'
    'x-ms-ratelimit-remaining-subscription-reads': 'REDACTED'
    'x-ms-ratelimit-remaining-subscription-global-reads': 'REDACTED'
    'x-ms-correlation-request-id': 'REDACTED'
    'x-ms-routing-request-id': 'REDACTED'
    'Strict-Transport-Security': 'REDACTED'
    'X-Cache': 'REDACTED'
    'X-MSEdge-Ref': 'REDACTED'
    'Date': 'Mon, 22 Jul 2024 06:50:38 GMT'
Response status: 200
Response headers:
    'Cache-Control': 'no-cache'
    'Pragma': 'no-cache'
    'Content-Length': '1331'
    'Content-Type': 'application/json; charset=utf-8'
    'Expires': '-1'
    'X-Content-Type-Options': 'REDACTED'
    'x-ms-request-id': '670a55d1-d434-45fb-b9ef-13a4bb4d5110'
    'x-ms-ratelimit-remaining-subscription-reads': 'REDACTED'
    'x-ms-ratelimit-remaining-subscription-global-reads': 'REDACTED'
    'x-ms-correlation-request-id': 'REDACTED'
    'x-ms-routing-request-id': 'REDACTED'
    'Strict-Transport-Security': 'REDACTED'
    'X-Cache': 'REDACTED'
    'X-MSEdge-Ref': 'REDACTED'
    'Date': 'Mon, 22 Jul 2024 06:50:38 GMT'
Response status: '200'
Response headers:
    'Cache-Control': 'no-cache'
    'Pragma': 'no-cache'
    'Content-Length': '1331'
    'Content-Type': 'application/json; charset=utf-8'
    'Expires': '-1'
    'X-Content-Type-Options': 'nosniff'
    'x-ms-request-id': '670a55d1-d434-45fb-b9ef-13a4bb4d5110'
    'x-ms-ratelimit-remaining-subscription-reads': '249'
    'x-ms-ratelimit-remaining-subscription-global-reads': '3749'
    'x-ms-correlation-request-id': 'd58f5b95-5b85-4dc1-859c-8debf9fe05dd'
    'x-ms-routing-request-id': 'GERMANYWESTCENTRAL:20240722T065039Z:d58f5b95-5b85-4dc1-859c-8debf9fe05dd'
    'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
    'X-Cache': 'CONFIG_NOCACHE'
    'X-MSEdge-Ref': 'Ref A: F93BB325522B48F3B8639626C4EA6435 Ref B: VIEEDGE2409 Ref C: 2024-07-22T06:50:39Z'
    'Date': 'Mon, 22 Jul 2024 06:50:38 GMT'
Response content:
{"value":[{"sku":{"name":"Standard_D2ds_v4","tier":"GeneralPurpose"},"properties":{"administratorLogin":"***","storageProfile":{"storageMB":131072,"storageIops":684,"backupRetentionDays":7,"storageAutogrow":"Enabled","fileStorageSkuName":"Premium_LRS"},"version":"8.0.21","state":"Ready","haState":"NotEnabled","fullyQualifiedDomainName":"***","sourceServerId":"","publicNetworkAccess":"Enabled","sslEnforcement":"Disabled","haEnabled":"Disabled","availabilityZone":"3","standbyAvailabilityZone":"","earliestRestoreDate":"2024-07-15T06:50:39.4785006+00:00","maintenanceWindow":{"customWindow":"Disabled","dayOfWeek":0,"startHour":0,"startMinute":0},"replicationRole":"None","replicaCapacity":10,"byokEnforcement":"Disabled"},"location":"West Europe","tags":{"BusinessOwnerEmail":"***","CostCenter":"***","Criticality":"Low","Department":"***","DevOwnerEmail":"***","MaintenanceWindow":"3rd Sun 10-12am CET","ProjectName":"***","ApplicationUID":"***"},"id":"/subscriptions/***/resourceGroups/***/providers/Microsoft.DBforMySQL/flexibleServers/***","name":"***","type":"Microsoft.DBforMySQL/flexibleServers"}]}
Response status: '200'
Response headers:
    'Cache-Control': 'no-cache'
    'Pragma': 'no-cache'
    'Content-Length': '1331'
    'Content-Type': 'application/json; charset=utf-8'
    'Expires': '-1'
    'X-Content-Type-Options': 'nosniff'
    'x-ms-request-id': '670a55d1-d434-45fb-b9ef-13a4bb4d5110'
    'x-ms-ratelimit-remaining-subscription-reads': '249'
    'x-ms-ratelimit-remaining-subscription-global-reads': '3749'
    'x-ms-correlation-request-id': 'd58f5b95-5b85-4dc1-859c-8debf9fe05dd'
    'x-ms-routing-request-id': 'GERMANYWESTCENTRAL:20240722T065039Z:d58f5b95-5b85-4dc1-859c-8debf9fe05dd'
    'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
    'X-Cache': 'CONFIG_NOCACHE'
    'X-MSEdge-Ref': 'Ref A: F93BB325522B48F3B8639626C4EA6435 Ref B: VIEEDGE2409 Ref C: 2024-07-22T06:50:39Z'
    'Date': 'Mon, 22 Jul 2024 06:50:38 GMT'
Response content:
{"value":[{"sku":{"name":"Standard_D2ds_v4","tier":"GeneralPurpose"},"properties":{"administratorLogin":"***","storageProfile":{"storageMB":131072,"storageIops":684,"backupRetentionDays":7,"storageAutogrow":"Enabled","fileStorageSkuName":"Premium_LRS"},"version":"8.0.21","state":"Ready","haState":"NotEnabled","fullyQualifiedDomainName":"***","sourceServerId":"","publicNetworkAccess":"Enabled","sslEnforcement":"Disabled","haEnabled":"Disabled","availabilityZone":"3","standbyAvailabilityZone":"","earliestRestoreDate":"2024-07-15T06:50:39.4785006+00:00","maintenanceWindow":{"customWindow":"Disabled","dayOfWeek":0,"startHour":0,"startMinute":0},"replicationRole":"None","replicaCapacity":10,"byokEnforcement":"Disabled"},"location":"West Europe","tags":{"BusinessOwnerEmail":"***","CostCenter":"***","Criticality":"Low","Department":"***","DevOwnerEmail":"***","MaintenanceWindow":"3rd Sun 10-12am CET","ProjectName":"***","ApplicationUID":"***"},"id":"/subscriptions/***/resourceGroups/***/providers/Microsoft.DBforMySQL/flexibleServers/***","name":"***","type":"Microsoft.DBforMySQL/flexibleServers"}]}

In [4]:

In the Request URL attribute there is still api-version=2020-07-01-preview. It appears that the api_version is hardcoded in the latest stable release. However, it is not on the master branch.

Screenshot 2024-07-22 at 09 22 28
ChenxiJiang333 commented 1 month ago

Hi @antonkazlouski-orca, please try to use azure-mgmt-rdbms==10.2.0b17 to run the codes instead.

github-actions[bot] commented 1 month ago

Hi @antonkazlouski-orca. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

antonkazlouski-orca commented 1 month ago

@ChenxiJiang333 that worked, but we won't consider using beta package in the production system. Is there any workaround?

msyyc commented 1 month ago

@ChenxiJiang333 that worked, but we won't consider using beta package in the production system. Is there any workaround?

azure-mgmt-rdbms is a complex package which contains several service so it keeps beta as long as not all service is stable. SDK PM @josefree is working on split of the package.

antonkazlouski-orca commented 1 month ago

@msyyc thank you. So, it leaves us with 2 options:

  1. Use beta version
  2. Patch the existing version ourselves and use it until a stable release is ready

Correct?

ChenxiJiang333 commented 1 month ago

@msyyc thank you. So, it leaves us with 2 options:

  1. Use beta version
  2. Patch the existing version ourselves and use it until a stable release is ready

Correct?

Yes, for now, we cannot make a new stable version immediately.