dell / PyU4V

PyU4V is a Python module that simplifies interaction with the Unisphere for PowerMax REST API. It wraps REST calls with simple APIs that abstracts the HTTP request and response handling.
Apache License 2.0
41 stars 37 forks source link

Timeout for get_masking_view_connection() #204

Closed 212850a closed 5 days ago

212850a commented 1 week ago

Hi,

We have couple of dedicated arrays where masking views contains hundreds of lines, as example: `

mv=a.connection.provisioning.get_masking_view_connections('MVNAME') len(mv) 119808 `

As result during collection job we get timeout for get_masking_view_connections() function - it returns:

exception.VolumeBackendAPIException('PyU4V.utils.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error GET symmetrix. The status code received is None and the message is None').

I've tried to increase timeout to 150 seconds with a help of set_requests_timeout(150) function but it didn't really help. Sometimes it works and sometimes not.

Is there any way to make get_masking_view_connections()function faster, may be by using some specific filters?

The reason why we use get_masking_view_connections() is we need to get list of devices with their scsi by host and looks like get_masking_view_connections() is the only one way to get list of volumes with their scsi id's. Or there is another way?

p.s. We use PyU4V 9.2 as we have plenty of PMAX 2k/8k still.

rawstorage commented 1 week ago

Hi Roman, This message is coming from Unisphere API and not PyU4V the only thing you can do in these cases is to do a lookup after the call mate put in a wait and then look to the view to see if it has actually been created. It's worth opening a ticket to look at the Unisphere server to make sure its not overloaded with too many api calls at the time this is happening also.

Sent from Outlook for Androidhttps://aka.ms/AAb9ysg

Internal Use - Confidential


From: Roman @.> Sent: Tuesday, October 15, 2024 9:10:40 AM To: dell/PyU4V @.> Cc: Subscribed @.***> Subject: [dell/PyU4V] Timeout for get_masking_view_connection() (Issue #204)

[EXTERNAL EMAIL]

Hi,

We have couple of dedicated arrays where masking views contains hundreds of lines, as example: `

mv=a.connection.provisioning.get_masking_view_connections('MVNAME') len(mv) 119808 ` As result during collection job we get timeout for get_masking_view_connections() function. it returns exception.VolumeBackendAPIException('PyU4V.utils.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error GET symmetrix. The status code received is None and the message is None').

I've tried to increase timeout to 150 seconds with a help of set_requests_timeout(150) function but it didn't really help. Sometimes it works and sometimes not.

Is there any way to make get_masking_view_connections()function faster, may be by using some specific filters?

The reason why we use get_masking_view_connections() is we need to get list of devices with their scsi by host and looks like get_masking_view_connections() is the only one way to get list of volumes with their scsi id's. Or there is another way?

p.s. We use PyU4V 9.2 as we have plenty of PMAX 2k/8k still.

— Reply to this email directly, view it on GitHub [github.com]https://urldefense.com/v3/__https://github.com/dell/PyU4V/issues/204__;!!LpKI!ip40djyHoRyUZeijOoNMC7rZmRMJFpW5V2JXNJqI2v2VtTShF2ksQsRSIT7nL_OtojqV0hrUMF7ntlAQ0sJydfhK$, or unsubscribe [github.com]https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AF7FWPKXS6B3TMI66KRV3FDZ3TEYBAVCNFSM6AAAAABP6RKPOCVHI2DSMVQWIX3LMV43ASLTON2WKOZSGU4DOOJYHEYDAMA__;!!LpKI!ip40djyHoRyUZeijOoNMC7rZmRMJFpW5V2JXNJqI2v2VtTShF2ksQsRSIT7nL_OtojqV0hrUMF7ntlAQ0uKK4_uR$. You are receiving this because you are subscribed to this thread.Message ID: @.***>

212850a commented 1 week ago

Paul, can you please give me example of code snippet which can wait and do lookup after?

rawstorage commented 1 week ago

job = (conn.provisioning.create_masking_view_existing_components( storage_group_name=SG_NAME, port_group_name=PG_NAME, host_group_name=HG_NAME, _async=True)

conn.common.wait_for_job_to_complete(job) print(conn.common.get_job_by_id(job)

The above code should work, using Async job queue to execute the job and looking up with the inbuilt functions should help.

Paul Martin Senior Principal Engineering Technologist Dell | ISG - PowerMax and VMAX Cell +35387 696 0621 Timezone GMT

Please note that my working days are Tuesday – Friday.

Follow me on Twitter @rawstorage Blog https://rawstorage.wordpress.com/

WhitePapers - https://infohub.delltechnologies.com/t/storage/

Internal Use - Confidential

From: Roman @.> Sent: Tuesday, October 15, 2024 9:29 AM To: dell/PyU4V @.> Cc: Martin, Paul @.>; Comment @.> Subject: Re: [dell/PyU4V] Timeout for get_masking_view_connection() (Issue #204)

[EXTERNAL EMAIL]

Paul, can you please give me example of code snippet which can wait and do lookup after?

— Reply to this email directly, view it on GitHub [github.com]https://urldefense.com/v3/__https:/github.com/dell/PyU4V/issues/204*issuecomment-2413226995__;Iw!!LpKI!ghpTgMbjfC5sRG1EFhDXoOOB_k4FjIDbMZ47y9wydeIqPHePnIpWmRz7H9bgRdmUrfG6alM36X3ZNVRuwMYyJtuq$, or unsubscribe [github.com]https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AF7FWPLSDCCITYJVYWGURO3Z3TG3PAVCNFSM6AAAAABP6RKPOCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJTGIZDMOJZGU__;!!LpKI!ghpTgMbjfC5sRG1EFhDXoOOB_k4FjIDbMZ47y9wydeIqPHePnIpWmRz7H9bgRdmUrfG6alM36X3ZNVRuwNC-pLax$. You are receiving this because you commented.Message ID: @.**@.>>

212850a commented 1 week ago

Thanks for suggestion, but looks like connection.provisioning.get_masking_view_connections can't be run as async job, it does not have _async option. TypeError: get_masking_view_connections() got an unexpected keyword argument '_async'

rawstorage commented 1 week ago

Sorry I misunderstood I thought you wanted to create a masking view. Not review the masking view connections, Presumably you are looking up the host lun address if you are using this call. I know that call can be problematic especially if there are a large number of initiators with multiple paths. I believe there were fixes in later versions of Unisphere for PowerMax to make this more efficient but you would need to check with Dell support.

Paul Martin Senior Principal Engineering Technologist Dell | ISG - PowerMax and VMAX Cell +35387 696 0621 Timezone GMT

Please note that my working days are Tuesday – Friday.

Follow me on Twitter @rawstorage Blog https://rawstorage.wordpress.com/

WhitePapers - https://infohub.delltechnologies.com/t/storage/

Internal Use - Confidential

From: Martin, Paul Sent: Tuesday, October 15, 2024 3:57 PM To: dell/PyU4V @.>; dell/PyU4V @.> Cc: Comment @.***> Subject: RE: [dell/PyU4V] Timeout for get_masking_view_connection() (Issue #204)

job = (conn.provisioning.create_masking_view_existing_components( storage_group_name=SG_NAME, port_group_name=PG_NAME, host_group_name=HG_NAME, _async=True)

conn.common.wait_for_job_to_complete(job) print(conn.common.get_job_by_id(job)

The above code should work, using Async job queue to execute the job and looking up with the inbuilt functions should help.

Paul Martin Senior Principal Engineering Technologist Dell | ISG - PowerMax and VMAX Cell +35387 696 0621 Timezone GMT

Please note that my working days are Tuesday – Friday.

Follow me on Twitter @rawstorage Blog https://rawstorage.wordpress.com/

WhitePapers - https://infohub.delltechnologies.com/t/storage/

From: Roman @.**@.>> Sent: Tuesday, October 15, 2024 9:29 AM To: dell/PyU4V @.**@.>> Cc: Martin, Paul @.**@.>>; Comment @.**@.>> Subject: Re: [dell/PyU4V] Timeout for get_masking_view_connection() (Issue #204)

[EXTERNAL EMAIL]

Paul, can you please give me example of code snippet which can wait and do lookup after?

— Reply to this email directly, view it on GitHub [github.com]https://urldefense.com/v3/__https:/github.com/dell/PyU4V/issues/204*issuecomment-2413226995__;Iw!!LpKI!ghpTgMbjfC5sRG1EFhDXoOOB_k4FjIDbMZ47y9wydeIqPHePnIpWmRz7H9bgRdmUrfG6alM36X3ZNVRuwMYyJtuq$, or unsubscribe [github.com]https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AF7FWPLSDCCITYJVYWGURO3Z3TG3PAVCNFSM6AAAAABP6RKPOCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJTGIZDMOJZGU__;!!LpKI!ghpTgMbjfC5sRG1EFhDXoOOB_k4FjIDbMZ47y9wydeIqPHePnIpWmRz7H9bgRdmUrfG6alM36X3ZNVRuwNC-pLax$. You are receiving this because you commented.Message ID: @.**@.>>

rawstorage commented 5 days ago

Closing as this is not directly an issue for PyU4V and needs to be worked with support