Python 2.0 extension, Build and deployment is successful but issue after pushing the code to the environment

[ssvarian]

Description

During the build of python 2.0 extension , everything was successful but after the deployment it is complaining about the dependency error.

I'm not sure if it is a warning or a error based on below messaging.

2024-07-05 19:29:05.326 UTC [00000455] warning [native] Failed to prepare extension environment: id: f_8873692240273979070 datasource: python:3.10 time elapsed: 2.71s command: "/var/lib/dynatrace/remotepluginmodule/agent/runtime/extensions/python_venvs/custom_dt-gen2-powerbi-utility_0.0.7/bin/python3" -m pip install --no-warn-script-location --no-cache-dir --no-index -f /var/lib/dynatrace/remotepluginmodule/agent/runtime/extensions/cache/custom_dt-gen2-powerbi-utility/0.0.7/lib dt_gen2_powerbi_utility error: WARNING: pyjwt 1.1.0 does not provide the extra 'crypto'

But in the same log it says, requirement satisfied. this is quite confusing

Requirement already satisfied: pyjwt[crypto] in /var/lib/dynatrace/remotepluginmodule/agent/runtime/extensions/python_venvs/custom_dt-gen2-powerbi-utility_0.0.7/lib/python3.10/site-packages (from dt_gen2_powerbi_utility) (1.1.0)

Initially the error started with the Azure Identity library, where build fails for with some incompatibilities, and never clearly mentioned what are the incompatibilities.

In the dynatrace console logs, it doesn't mention any reason on why the extension is not deployed.

Steps to reproduce

During the build of python 2.0 extension , everything was successful but after the deployment it is complaining about the dependency error.

I'm not sure if it is a warning or a error based on below messaging.

2024-07-05 19:29:05.326 UTC [00000455] warning [native] Failed to prepare extension environment: id: f_8873692240273979070 datasource: python:3.10 time elapsed: 2.71s command: "/var/lib/dynatrace/remotepluginmodule/agent/runtime/extensions/python_venvs/custom_dt-gen2-powerbi-utility_0.0.7/bin/python3" -m pip install --no-warn-script-location --no-cache-dir --no-index -f /var/lib/dynatrace/remotepluginmodule/agent/runtime/extensions/cache/custom_dt-gen2-powerbi-utility/0.0.7/lib dt_gen2_powerbi_utility error: WARNING: pyjwt 1.1.0 does not provide the extra 'crypto'

But in the same log it says, requirement satisfied. this is quite confusing

Requirement already satisfied: pyjwt[crypto] in /var/lib/dynatrace/remotepluginmodule/agent/runtime/extensions/python_venvs/custom_dt-gen2-powerbi-utility_0.0.7/lib/python3.10/site-packages (from dt_gen2_powerbi_utility) (1.1.0)

Initially the error started with the Azure Identity library, where build fails for with some incompatibilities, and never clearly mentioned what are the incompatibilities.

In the dynatrace console logs, it doesn't mention any reason on why the extension is not deployed.

Support information

• [ ] Screenshots
• [ ] Extension manifest file
• [ ] YAML snippets
• [X] Support archive (see here)

[radu-stefan-dt]

Hi @ssvarian

This is not an issue with the VS Code extension, but rather with the python extension itself and its modules.

To help you out, I need more information...

1. Why are you using pyjwt version 1.1.0? This was released in 2015. The most recent version is 2.8.0 and it includes the crypto dependency.
2. What does your setup.py file look like?
3. Can you share the entire log?

[ssvarian]

Hi @radu-stefan-dt

I'm not explicitly using the pyjwt package, I think it is coming as part of msal package.

Here is how my setup.py looks like

setup(name="dt_gen2_powerbi_utility", version=find_version(), description="Dt_gen2_powerbi_utility python EF2 extension", author="Dynatrace", packages=find_packages(), python_requires=">=3.10", include_package_data=True, install_requires=["dt-extensions-sdk", "msal", "requests", "python-dateutil"], extras_require={"dev": ["dt-extensions-sdk[cli]"]}, )

Attached the log ruxitagent_extensionsmodule_958.0.log

[radu-stefan-dt]

That might be the case. Try explicitly adding cryptography to install_requires list. See if that gets over the hurdle. From what I read the pyjwt[crypto] is a more recent syntax and might not work for version 1.1.0

[ssvarian]

HI @radu-stefan-dt

I did even try that as well but during the build phase it fails with the below error

{ "error": [ "", "[notice] A new release of pip is available: 24.1.1 -> 24.1.2", "[notice] To update, run: python.exe -m pip install --upgrade pip", "ERROR: Could not find a version that satisfies the requirement cryptography (from dt-gen2-powerbi-utility) (from versions: none)", "ERROR: No matching distribution found for cryptography", "", "[notice] A new release of pip is available: 24.1.1 -> 24.1.2", "[notice] To update, run: python.exe -m pip install --upgrade pip", "Traceback (most recent call last):", " File \"C:\Program Files\Python310\lib\runpy.py\", line 196, in _run_module_as_main", " return _run_code(code, main_globals, None,", " File \"C:\Program Files\Python310\lib\runpy.py\", line 86, in _run_code", " exec(code, run_globals)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\Scripts\dt-sdk.exe\main.py\", line 7, in ", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\typer\main.py\", line 326, in call", " raise e", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\typer\main.py\", line 309, in call", " return get_command(self)(*args, kwargs)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\click\core.py\", line 1157, in call", " return self.main(args, kwargs)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\typer\core.py\", line 723, in main", " return _main(", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\typer\core.py\", line 193, in _main", " rv = self.invoke(ctx)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\click\core.py\", line 1688, in invoke", " return _process_result(sub_ctx.command.invoke(sub_ctx))", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\click\core.py\", line 1434, in invoke", " return ctx.invoke(self.callback, ctx.params)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\click\core.py\", line 783, in invoke", " return __callback(args, kwargs)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\typer\main.py\", line 692, in wrapper", " return callback(**use_params)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\dynatrace_extension\cli\main.py\", line 114, in build", " wheel(extension_dir, extra_platforms, extra_index_url, find_links)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\dynatrace_extension\cli\main.py\", line 227, in wheel", " run_process(command, cwd=extension_dir)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\dynatrace_extension\cli\main.py\", line 410, in run_process", " return subprocess.run(command, cwd=cwd, env=env, check=True) # noqa: S603", " File \"C:\Program Files\Python310\lib\subprocess.py\", line 524, in run", " raise CalledProcessError(retcode, process.args,", "subprocess.CalledProcessError: Command '['C:\\Program Files\\Python310\\python.exe', '-m', 'pip', 'download', '-d', 'extension/lib', '--only-binary=:all:', '--platform', 'linux_x86_64', '.']' returned non-zero exit status 1.", "" ], }

Also If you see the log file it even has the 2.8.0 pyjwt wheel file which is processing

Processing /var/lib/dynatrace/remotepluginmodule/agent/runtime/extensions/cache/custom_dt-gen2-powerbi-utility/0.0.7/lib/PyJWT-2.8.0-py3-none-any.whl (from dt_gen2_powerbi_utility) INFO: pip is looking at multiple versions of msal to determine which version is compatible with other requirements. This could take a while.

I belive there are some compatability issues with msal and azure-identity packages.. Earlier I faced issue with azure-identity package so I had to switch to msal and now I encounter this issues.

[ssvarian]

I've re-wrote the code with just requests package, and removed msal package and was able to make it work. So I'm surer there is some incompatibilities with DT and azure-identity/msal libraries.

Not sure who is responsible for this.

[radu-stefan-dt]

It's not about Dynatrace specifically... all that we do is create a virtual environment, and attempt to install your extension as its own module (hence the setup.py file and install_requires).

From then on it's on pip to find a matching version and wheel on pypi based on all the versions of modules referenced within.

[ssvarian]

Completely Agree, but I don't see a way to resolve the conflicts. but anyways thanks for looking into it even though it is not related to vscode extension..

[radu-stefan-dt]

For what it's worth this may be an issue with the O/S you're trying to run it on. As the locally installed pip will attempt to find a package distribution from pypi matched to the OneAgent or ActiveGate's O/S.

I was able to successfully build and run an extension with the same dependencies as yours ("dt-extensions-sdk", "msal", "requests", "python-dateutil") on a OneAgent on Windows.

It ended up install pyjwt 2.8.0. So it's very strange you got a mention of version 1.1.0 in the log. Cryptography also got installed automatically. Perhaps there's an issue with Linux specifically.

[ssvarian]

I was using the remote install ( active gate ) on ubuntu 22.0.4

[radu-stefan-dt]

In fact, looking at your log again, there is clearly a difference when trying to install to Linux. Where mine ended up installing latest versions of all modules, in your log the winning combo included:

• PyJWT-1.1.0
• certifi-2024.7.4
• charset-normalizer-3.3.2
• idna-3.7
• msal-1.5.1
• python-dateutil-2.9.0.post0
• requests-2.32.3
• six-1.16.0
• urllib3-2.2.2

[radu-stefan-dt]

There is one thing you can try with our VSCode extension. In your workspace, edit the .vscode/settings.json file or create one if it doesn't exist. Add the following setting:

    "dynatraceExtensions.pythonExtraPlatforms": [
        "win_amd_64",
        "many_linux_2_17_x86_64"
    ]

Try building again like this and deploying it.

[ssvarian]

I've tried adding the snipet , but still an issue with the dependencies..

{ "error": [ "", "[notice] A new release of pip is available: 24.1.1 -> 24.1.2", "[notice] To update, run: python.exe -m pip install --upgrade pip", "ERROR: Cannot install dt-gen2-powerbi-utility because these package versions have conflicting dependencies.", "ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts", "", "[notice] A new release of pip is available: 24.1.1 -> 24.1.2", "[notice] To update, run: python.exe -m pip install --upgrade pip", "Traceback (most recent call last):", " File \"C:\Program Files\Python310\lib\runpy.py\", line 196, in _run_module_as_main", " return _run_code(code, main_globals, None,", " File \"C:\Program Files\Python310\lib\runpy.py\", line 86, in _run_code", " exec(code, run_globals)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\Scripts\dt-sdk.exe\main.py\", line 7, in ", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\typer\main.py\", line 326, in call", " raise e", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\typer\main.py\", line 309, in call", " return get_command(self)(*args, kwargs)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\click\core.py\", line 1157, in call", " return self.main(args, kwargs)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\typer\core.py\", line 723, in main", " return _main(", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\typer\core.py\", line 193, in _main", " rv = self.invoke(ctx)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\click\core.py\", line 1688, in invoke", " return _process_result(sub_ctx.command.invoke(sub_ctx))", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\click\core.py\", line 1434, in invoke", " return ctx.invoke(self.callback, ctx.params)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\click\core.py\", line 783, in invoke", " return __callback(args, kwargs)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\typer\main.py\", line 692, in wrapper", " return callback(**use_params)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\dynatrace_extension\cli\main.py\", line 114, in build", " wheel(extension_dir, extra_platforms, extra_index_url, find_links)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\dynatrace_extension\cli\main.py\", line 227, in wheel", " run_process(command, cwd=extension_dir)", " File \"C:\Users\dw777777\AppData\Roaming\Python\Python310\site-packages\dynatrace_extension\cli\main.py\", line 410, in run_process", " return subprocess.run(command, cwd=cwd, env=env, check=True) # noqa: S603", " File \"C:\Program Files\Python310\lib\subprocess.py\", line 524, in run", " raise CalledProcessError(retcode, process.args,", "subprocess.CalledProcessError: Command '['C:\\Program Files\\Python310\\python.exe', '-m', 'pip', 'download', '-d', 'extension/lib', '--only-binary=:all:', '--platform', 'win_amd_64', '.']' returned non-zero exit status 1.", "" ], "detailedOutput": [ "+" ] }

[radu-stefan-dt]

Ok, sorry, that was a typo on my part, it should be win_amd64 instead of win_amd_64. Could you please try again with:

    "dynatraceExtensions.pythonExtraPlatforms": [
        "win_amd64",
        "many_linux_2_17_x86_64"
    ]