Unable to install in windows

[thevishalagarwal]

ran the following command to install TRT-LLM on windows

pip install tensorrt_llm==0.11.0 --extra-index-url https://pypi.nvidia.com

Getting the following error. Followed instructions from here.

Looking in indexes: https://pypi.org/simple, https://pypi.nvidia.com
Collecting tensorrt_llm==0.11.0
  Using cached tensorrt_llm-0.11.0.tar.gz (668 bytes)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [41 lines of output]
      Traceback (most recent call last):
        File "D:\python\envs\trtllm\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "D:\python\envs\trtllm\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\python\envs\trtllm\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 152, in prepare_metadata_for_build_wheel
          whl_basename = backend.build_wheel(metadata_directory, config_settings)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\vrl\AppData\Local\Temp\pip-build-env-vidnd573\overlay\Lib\site-packages\nvidia_stub\buildapi.py", line 29, in build_wheel
          return download_wheel(pathlib.Path(wheel_directory), config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\vrl\AppData\Local\Temp\pip-build-env-vidnd573\overlay\Lib\site-packages\nvidia_stub\wheel.py", line 175, in download_wheel
          report_install_failure(distribution, version, None)
        File "C:\Users\vrl\AppData\Local\Temp\pip-build-env-vidnd573\overlay\Lib\site-packages\nvidia_stub\error.py", line 63, in report_install_failure
          raise InstallFailedError(
      nvidia_stub.error.InstallFailedError:
      *******************************************************************************

      The installation of tensorrt-llm for version 0.11.0 failed.

      This is a special placeholder package which downloads a real wheel package
      from https://pypi.nvidia.com. If https://pypi.nvidia.com is not reachable, we
      cannot download the real wheel file to install.

      You might try installing this package via

  $ pip install --extra-index-url https://pypi.nvidia.com tensorrt-llm
  ```

  Here is some debug information about your platform to include in any bug
  report:

  Python Version: CPython 3.11.9
  Operating System: Windows 10
  CPU Architecture: AMD64
  Driver Version: 560.70
  CUDA Version: 12.6

  *******************************************************************************

  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip. hint: See above for details.

Any solution for this?

[thevishalagarwal]

It was a CUDA version issue. Fixed it by clean installation of 12.4. But running into a new issue

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Program Files\Python310\lib\site-packages\tensorrt_llm\__init__.py", line 92, in <module>
    _init(log_level="error")
  File "C:\Program Files\Python310\lib\site-packages\tensorrt_llm\_common.py", line 57, in _init
    _load_plugin_lib()
  File "C:\Program Files\Python310\lib\site-packages\tensorrt_llm\plugin\plugin.py", line 50, in _load_plugin_lib
    assert handle.initTrtLlmPlugins(None,
OSError: exception: access violation reading 0x0000000000000000

Is there a workaround for this?

[thevishalagarwal]

Looks like the null pointer issue is with conda environment. Built the package locally and installed in a native venv environment. Works fine

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 15 days."

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 15 days."