search

tianocore / edk2-pytool-extensions

Extensions to the edk2 build system allowing for a more robust and plugin based build system and tool execution environment
Other
60 stars 41 forks source link

[Bug]: Getting regex module not found error when trying to build BaseTools with Edk2ToolsBuild.py script #870

Closed jpshivakavi closed 3 months ago

jpshivakavi commented 3 months ago

Contact Details

n.jayaprakash@intel.com

Describe the Bug

I was trying to explore Edk2ToolsBuild.py script and use it instead of the edksetup.bat Rebuild command for building the BaseTools of edk2 repo. I am getting Regex module not found error.

I was able to resolve this by installing the regex module. But it would be good to add Regex as one of the dependencies to edk2-pytool-extensions package so that it gets installed along with the package installation.

Below is the build error log for reference. C:\Users\njayapra\github\src\edk2>python BaseTools\Edk2ToolsBuild.py -t VS2022 SECTION - Init Self Describing Environment SECTION - Loading Plugins ERROR - Failed to import plugin: C:\Users\njayapra\github\src\edk2\BaseTools\Plugin\DebugMacroCheck\BuildPlugin\DebugMacroCheckBuildPlugin.py Traceback (most recent call last): File "C:\Users\njayapra\AppData\Roaming\Python\Python312\site-packages\edk2toolext\environment\plugin_manager.py", line 112, in _load spec.loader.exec_module(module) File "", line 995, in exec_module File "", line 488, in _call_with_frames_removed File "C:\Users\njayapra\github\src\edk2\BaseTools\Plugin\DebugMacroCheck\BuildPlugin\DebugMacroCheckBuildPlugin.py", line 24, in import DebugMacroCheck # noqa: E402 ^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\njayapra\github\src\edk2\BaseTools\Plugin\DebugMacroCheck\DebugMacroCheck.py", line 17, in import regex ModuleNotFoundError: No module named 'regex' CRITICAL - One or more plugins failed to load. Halting build. ERROR - Failed Plugin: Debug Macro Check Plugin Traceback (most recent call last): File "C:\Users\njayapra\github\src\edk2\BaseTools\Edk2ToolsBuild.py", line 188, in main() File "C:\Users\njayapra\github\src\edk2\BaseTools\Edk2ToolsBuild.py", line 184, in main Edk2ToolsBuild().Invoke() File "C:\Users\njayapra\AppData\Roaming\Python\Python312\site-packages\edk2toolext\base_abstract_invocable.py", line 227, in Invoke raise Exception("One or more plugins failed to load.") Exception: One or more plugins failed to load.

What Python version are you using?

Python 3.10, Python 3.11, Python 3.12

Reproduction steps

  1. Cloned edk2 repo
  2. Opened a windows command prompt
  3. Installed the edk2-pytool-extensions package
  4. changed working directory to edk2
  5. Issued the following command python BaseTools\Edk2ToolsBuild.py -t VS2022

Expected behavior

Build command should work normally and create the BaseTools build artifacts in BaseTools\Bin\Win32 folder.

Execution Environment

Windows 11 Python 3.12.4

Pip packages

C:\Users\njayapra\github\src\edk2>python -m pip list Package Version

build 1.0.3 CacheControl 0.14.0 ccbhwapi 1.0.0.12 certifi 2024.7.4 cffi 1.17.0 chardet 5.2.0 charset-normalizer 3.3.2 cleo 2.1.0 code-review-bot-metrics 0.1.0 colorama 0.4.6 contourpy 1.2.1 crashtest 0.4.1 cryptography 43.0.0 cycler 0.12.1 distlib 0.3.8 docutils 0.21.2 dulwich 0.21.7 edk2-pytool-extensions 0.27.11 edk2-pytool-library 0.21.10 et-xmlfile 1.1.0 fasteners 0.19 fastjsonschema 2.19.1 filelock 3.13.1 fonttools 4.53.0 fsspec 2024.6.0 gitdb 4.0.11 GitPython 3.1.43 greenlet 3.0.3 greet 0.42 hello 0.42 hello_cpp 0.42 hellocpp 0.42 idna 3.7 installer 0.7.0 intel-openmp 2021.4.0 jaraco.classes 3.3.1 Jinja2 3.1.4 joblib 1.4.2 keyring 24.3.1 Kivy 2.3.0 kivy-deps.angle 0.4.0 kivy-deps.glew 0.3.1 kivy-deps.sdl2 0.7.0 Kivy-Garden 0.1.5 kiwisolver 1.4.5 markdown-it-py 3.0.0 MarkupSafe 2.1.5 matplotlib 3.9.0 mdurl 0.1.2 mkl 2021.4.0 more-itertools 10.2.0 mpmath 1.3.0 msgpack 1.0.8 networkx 3.3 numpy 1.26.4 opencv-python 4.9.0.80 openpyxl 3.1.5 packaging 23.2 pandas 2.2.2 pefile 2023.2.7 pexpect 4.9.0 pillow 10.3.0 pip 24.0 pkginfo 1.10.0 platformdirs 4.2.0 poetry 1.8.2 poetry-core 1.9.0 poetry-plugin-export 1.7.1 ptyprocess 0.7.0 pyasn1 0.6.0 pyasn1_modules 0.4.0 pycparser 2.22 Pygments 2.18.0 pygount 1.8.0 pyparsing 3.1.2 pypiwin32 223 pyproject_hooks 1.0.0 python-dateutil 2.9.0.post0 pytz 2024.1 pyutil 3.3.6 pywin32 306 pywin32-ctypes 0.2.2 PyYAML 6.0.2 rapidfuzz 3.6.2 requests 2.32.3 requests-toolbelt 1.0.0 rich 13.7.1 semantic-version 2.10.0 setuptools 69.5.1 shellingham 1.5.4 six 1.16.0 smmap 5.0.1 SQLAlchemy 2.0.32 student 0.42 sympy 1.12.1 tbb 2021.13.0 tomlkit 0.12.4 torch 2.3.1 torchvision 0.18.1 trove-classifiers 2024.3.3 typing_extensions 4.12.2 tzdata 2024.1 urllib3 2.2.2 virtualenv 20.25.1 wheel 0.42.0 XlsxWriter 3.2.0

[notice] A new release of pip is available: 24.0 -> 24.2 [notice] To update, run: python.exe -m pip install --upgrade pip

Additional context

The problem solved after installing the Regex module.

Javagedes commented 3 months ago

Hello, the dependency on regex does not come from edk2-pytool-extensions, but by plugins from python source code in edk2. Edk2 has a pip-requirements file that contains all additional python requirements to work with edk2.

See here: https://github.com/tianocore/edk2/blob/master/pip-requirements.txt

Javagedes commented 3 months ago

Here is some additional documentation:

https://github.com/tianocore/tianocore.github.io/wiki/How-to-Build-With-Stuart

jpshivakavi commented 3 months ago

Thank you very much for the details. Followed documentation in edk2 you shared and installed the additional modules by using the pip-requirements.txt input.

This resolved the issue. We can close it.