search

PureStorage-OpenConnect / py-pure-client

Pure Storage Unified APIs Python Client
https://code.purestorage.com/py-pure-client/
BSD 2-Clause "Simplified" License
23 stars 29 forks source link

change client scripts to skip module imports #77

Closed tylergmuir closed 10 months ago

tylergmuir commented 1 year ago

This may be a controversial change, but I have seen that the amount of time it takes to import the pypureclient module is fairly long. This gets exacerbated when using Ansible since each task imports the module with no way to persist the python session across multiple tasks.

This removes the import of every API version of each of the modules and instead only imports the version that is required for the array that is being connected to. This does shift the import time to mainly being during the connect instead of during the import, which may be undesired, but the overall change will be a net reduction in time spent.

In my testing, it takes approximately 20 seconds to import the module. This also gets longer each time a new API version is added.

With my changes, it takes approximately 4 seconds to import the module. In the case that you connect to an array running an API version that hasn't been connected to yet, it takes approximately 0.5 additional seconds to connect due to including the import.

tylergmuir commented 1 year ago

@sdodsley I have been using this code for the last few days in Ansible and have seen a huge increase in speed completing tasks due to the reduction in the amount of time it takes for the module to import. It may be worth some testing and input on this PR.

sdodsley commented 1 year ago

@tylergmuir we have raised an internal engineering JIRA to get this addressed. We hope to get this solution into an upcoming release

sdodsley commented 12 months ago

@tylergmuir your request for dynamic import has been accepted and will be in the 1.44 release of py-pure-client, scheduled for some time in December. We missed the cutoff for 1.43.0

tylergmuir commented 11 months ago

@sdodsley I did some testing with the 1.44.0 release and it doesn't appear to be much if any different in terms of the amount of time it takes to run import pypureclient. And running it with python in verbose mode I still see it importing all of the different versions of the REST clients.

sdodsley commented 11 months ago

@tylergmuir Let me investigate what is happening.

Gregory-Bessonov commented 11 months ago

@tylergmuir Thank you for feedback! Dynamic import was rescheduled to 1.45.0 release, it will be available after GA of the FA REST 2.29. cc: @sdodsley

sdodsley commented 11 months ago

My bad @tylergmuir. Reopening until 1.45.0 is released

tylergmuir commented 10 months ago

@sdodsley and @Gregory-Bessonov I just ran some testing against my fleet with the 1.44 and 1.45 versions of the SDK. The 1.44 SDK took 00:29:28 to run an Ansible playbook check for a bunch of configuration. The 1.45 SDK took 00:14:25 to do the same check. Overall a more than 50% decrease in runtime, so I would call it a success.

sdodsley commented 10 months ago

@tylergmuir thanks for the test. We'll close this now.