New unmentioned requirements causing issues (requests, pyOpenSSL, cryptography)

[MallocArray]

Describe the bug

Recent commits upgrade vapi-runtime to 2.37 which bring new required versions of other module, particularly requests==2.27.1 that is not mentioned in the top level requirements.txt file. Previous versions of vapi-runtime was not pinned to a particular version

Not only does this prevent us from being able to run recent/updated versions of requests, but it causes drastic slowdowns in dependency backtracking if requests is listed in a requirements.txt file from other modules.

My primary use case is an Ansible Execution Environment build, where requests is known to be needed by other Ansible modules, so it was added without a specific version, but results in an error when installing vSphere SDK

If requests is listed in the requirement.txt without this specific version, we see the slow dependency issue and ultimate timeout of our build process after 60 minutes. Setting requests==2.27.1 in the requirements.txt allows the build to complete, but now we are locked to an older version and unable to tell when newer versions are supported

Reproduction steps

1. Have a requirements.txt with both the git repo for vsphere SDK as well as a line for just requests along with possibly other python modules, including cryptography and pyopenssl
2. Observe that with current commits in the SDK repo, docker build commands take a very long time with lots of repeating messages such as INFO: pip is looking at multiple versions of urllib3 to determine which version is compatible with other requirements. This could take a while. and eventually may hit a timeout of the build process
3. pinning requests==2.27.1 allows the build process to complete at expected speed
4. Can also change the requirements line to git+https://github.com/vmware/vsphere-automation-sdk-python.git@10857745e296b49899245c25606c67b4a1ef2873 to force an earlier commit when vapi_runtime-2.34.0 did not have this specific version required and builds process as normal May also have some conflicts with pyOpenSSL and cryptography versions as trying to install them with latest versions gave various errors, but having trouble reproducing at the moment. ...

ERROR: Cannot install -r execution_environment/requirements.txt (line 4), requests>=2.28.1 and vSphere-Automation-SDK because these package versions have conflicting dependencies.

The conflict is caused by: The user requested requests>=2.28.1 pyvmomi 7.0.3 depends on requests>=2.3.0 vapi-runtime 2.37.0 depends on requests==2.27.1; python_version > "3.6"

Expected behavior

If requests==2.27.1 is required, it should be listed in this project's main requirements.txt file

It also appears that it depends on cryptography==36.0.0 which is also not mentioned in the requirements files and also limits our ability to upgrade these packages to recent versions

ERROR: Cannot install -r execution_environment/requirements.txt (line 31), cryptography==38.0.1 and vSphere-Automation-SDK because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested cryptography==38.0.1
    pyopenssl 22.1.0 depends on cryptography<39 and >=38.0.0
    vapi-runtime 2.37.0 depends on cryptography==36.0.0; python_version > "3.6"

Additional context

No response

[kunal-pmj]

The issue will be addressed in 8.0 release. The release is scheduled for next week.

[kunal-pmj]

hi @MallocArray

The issue has been addressed. You can find requirements.txt with direct and transient dependencies.

[adpavlov]

The issue will be addressed in 8.0 release. The release is scheduled for next week.

this release is dropping ansible-core<2.13 support and support for older versions of vCenter Server 6.5/6.7 which is not applicable for a lot of use cases.
How older version galaxy collection community.vmware:2.9.1 can be installed then?

[kunal-pmj]

Hi @adpavlov

This is a bug and we are working on aligning the transient dependencies of python SDK. Any workaround at this point would require manually editing SDK's direct dependency's metadata to a lenient dependency constraint. e.g. for vapi_runtime-2.37.0-py2.py3-none-any.whl pip >= 22.3 and use local installation steps as mentioned in readme.

[PurpEE]

The issue will be addressed in 8.0 release. The release is scheduled for next week.

this release is dropping ansible-core<2.13 support and support for older versions of vCenter Server 6.5/6.7 which is not applicable for a lot of use cases. How older version galaxy collection community.vmware:2.9.1 can be installed then?

Pinning the version of communit.vmware didn't work, because it always pulled the latest version of vsphere-automation-sdk-python. The only workaround i found is forking this project and pinning the version of vsphere-automation-sdk-python in requirements.txt yourself and using your forked project instead of upstream temporarily.

[kunal-pmj]

Hi @PurpEE We acknowledge the issue and the additional steps required to make the SDK work. We will have the pinned requirements removed in the next SDK release.