NLE don't compile in MSYS2 (MinGW) environment on Windows

[cslr]

🐛 Bug

When trying to port Linux library that uses NLE/Python to Windows (MSYS2/MinGW environment) the compilation of NLE fails.

To Reproduce

Steps to reproduce the behavior:

1. Install python, wheel and pip on MSYS2 using pacman.
2. pip install nle

Error messages:

$ pip install nle Collecting nle Using cached nle-0.7.3.tar.gz (6.8 MB) Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing wheel metadata (pyproject.toml) ... done Requirement already satisfied: gym>=0.15 in c:/msys64/mingw64/lib/python3.9/site-packages (from nle) (0.21.0) Requirement already satisfied: numpy>=1.16 in c:/msys64/mingw64/lib/python3.9/site-packages (from nle) (1.21.2) Collecting pybind11>=2.2 Using cached pybind11-2.8.1-py2.py3-none-any.whl (208 kB) Requirement already satisfied: cloudpickle>=1.2.0 in c:/msys64/mingw64/lib/python3.9/site-packages (from gym>=0.15->nle) (2.0.0) Building wheels for collected packages: nle Building wheel for nle (pyproject.toml) ... error ERROR: Command errored out with exit status 1: command: C:/msys64/mingw64/bin/python.exe C:/msys64/mingw64/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel C:/Users/pc/AppData/Local/Temp/tmp9xc7w_29 cwd: C:/Users/pc/AppData/Local/Temp/pip-install-z5gpmg6f/nle_513d6f941f484a8fb8dd408964f75826 Complete output (41 lines): fatal: not a git repository (or any of the parent directories): .git Building wheel nle-0.7.3 running bdist_wheel running build running build_py creating build creating build/lib.mingw_x86_64-3.9 creating build/lib.mingw_x86_64-3.9/nle copying nle/version.py -> build/lib.mingw_x86_64-3.9/nle copying nle/init.py -> build/lib.mingw_x86_64-3.9/nle creating build/lib.mingw_x86_64-3.9/nle/env copying nle/env/base.py -> build/lib.mingw_x86_64-3.9/nle/env copying nle/env/tasks.py -> build/lib.mingw_x86_64-3.9/nle/env copying nle/env/init.py -> build/lib.mingw_x86_64-3.9/nle/env creating build/lib.mingw_x86_64-3.9/nle/nethack copying nle/nethack/actions.py -> build/lib.mingw_x86_64-3.9/nle/nethack copying nle/nethack/nethack.py -> build/lib.mingw_x86_64-3.9/nle/nethack copying nle/nethack/init.py -> build/lib.mingw_x86_64-3.9/nle/nethack creating build/lib.mingw_x86_64-3.9/nle/agent copying nle/agent/agent.py -> build/lib.mingw_x86_64-3.9/nle/agent copying nle/agent/vtrace.py -> build/lib.mingw_x86_64-3.9/nle/agent copying nle/agent/init.py -> build/lib.mingw_x86_64-3.9/nle/agent creating build/lib.mingw_x86_64-3.9/nle/scripts copying nle/scripts/check_nethack_speed.py -> build/lib.mingw_x86_64-3.9/nle/scripts copying nle/scripts/collect_env.py -> build/lib.mingw_x86_64-3.9/nle/scripts copying nle/scripts/play.py -> build/lib.mingw_x86_64-3.9/nle/scripts copying nle/scripts/plot.py -> build/lib.mingw_x86_64-3.9/nle/scripts copying nle/scripts/read_heaplog.py -> build/lib.mingw_x86_64-3.9/nle/scripts copying nle/scripts/read_tty.py -> build/lib.mingw_x86_64-3.9/nle/scripts copying nle/scripts/test_raw_nethack.py -> build/lib.mingw_x86_64-3.9/nle/scripts copying nle/scripts/ttyplay.py -> build/lib.mingw_x86_64-3.9/nle/scripts copying nle/scripts/ttyplay2.py -> build/lib.mingw_x86_64-3.9/nle/scripts copying nle/scripts/ttyrec.py -> build/lib.mingw_x86_64-3.9/nle/scripts copying nle/scripts/init.py -> build/lib.mingw_x86_64-3.9/nle/scripts package init file 'nle/tests/init.py' not found (or not a regular file) creating build/lib.mingw_x86_64-3.9/nle/tests copying nle/tests/test_envs.py -> build/lib.mingw_x86_64-3.9/nle/tests copying nle/tests/test_nethack.py -> build/lib.mingw_x86_64-3.9/nle/tests copying nle/tests/test_profile.py -> build/lib.mingw_x86_64-3.9/nle/tests running build_ext error: [WinError 2] M▒▒ritetty▒ tiedostoa ei l▒ydy

ERROR: Failed building wheel for nle Failed to build nle ERROR: Could not build wheels for nle, which is required to install pyproject.toml-based projects

Expected behavior

NLE should compile and install.

Environment

NLE version: N/A PyTorch version: N/A Is debug build: N/A CUDA used to build PyTorch: N/A

OS: Microsoft Windows 10 Pro GCC version: (Rev2, Built by MSYS2 project) 11.2.0 CMake version: Could not collect

Python version: 3.9 Is CUDA available: N/A CUDA runtime version: Could not collect GPU models and configuration: Could not collect Nvidia driver version: Could not collect cuDNN version: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.4/bin/cudnn_ops_train64_8.dll

Versions of relevant libraries: [pip3] numpy==1.21.2 [conda] Could not collect

Additional context

I installed a package "mingw-w64-x86_64-python-wheel" using pacman in MSYS2 and not installed the wheel package using pip.

[cslr]

Forgot to mention that I use python from the MSYS2 MinGW package: "mingw-w64-x86_64-python" and not the default "python" package of MSYS2.

[heiner]

Hey Tomas,

Thanks for your interest in NLE.

We don't currently support Windows, even though NetHack itself does. If you are looking to port it to Windows as you say, we are happy to accept patches and give you a few pointers on how to do that. I suspect that running it inside a Docker container (e.g., as in the docker/ subdirectory) is easier though.

[cslr]

I managed to get it continue a bit further by installing cmake and pybind11 ("pacman -Syy cmake mingw-w64-x86_64-pybind11"). Now the compilation fails to a non-existing directory (bad handling of "C:" drive letter in path?).

I don't really need nle environment in Windows so I'm unlikely work with this issue more.

[heiner]

Yes so having cmake around is going to be necessary. It's far from sufficient though. We use some POSIX features and porting NLE to Windows will require rewriting our C++ code.