[WINDOWS SUPPORT] ERROR: Could not build wheels for skranger which use PEP 517 and cannot be installed directly

[JamesBolandDing]

I'm trying to install skranger on my Windows machine but it doesn't seem to be working. When I run pip install skranger I get the following error:

Collecting skranger
  Using cached skranger-0.3.0.tar.gz (116 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: numpy in c:\users\james.boland\appdata\local\continuum\miniconda3\envs\clv-models-env\lib\site-packages (from skranger) (1.19.4)
Requirement already satisfied: scikit-learn in c:\users\james.boland\appdata\local\continuum\miniconda3\envs\clv-models-env\lib\site-packages (from skranger) (0.23.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in c:\users\james.boland\appdata\local\continuum\miniconda3\envs\clv-models-env\lib\site-packages (from scikit-learn->skranger) (2.1.0)
Requirement already satisfied: scipy>=0.19.1 in c:\users\james.boland\appdata\local\continuum\miniconda3\envs\clv-models-env\lib\site-packages (from scikit-learn->skranger) (1.5.4)
Requirement already satisfied: numpy in c:\users\james.boland\appdata\local\continuum\miniconda3\envs\clv-models-env\lib\site-packages (from skranger) (1.19.4)
Requirement already satisfied: joblib>=0.11 in c:\users\james.boland\appdata\local\continuum\miniconda3\envs\clv-models-env\lib\site-packages (from scikit-learn->skranger) (0.17.0)
Requirement already satisfied: numpy in c:\users\james.boland\appdata\local\continuum\miniconda3\envs\clv-models-env\lib\site-packages (from skranger) (1.19.4)
Building wheels for collected packages: skranger
  Building wheel for skranger (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: 'C:\Users\James.Boland\AppData\Local\Continuum\miniconda3\envs\clv-models-env\python.exe' 'C:\Users\James.Boland\AppData\Local\Continuum\miniconda3\envs\clv-models-env\lib\site-packages\pip\_vendor\pep517\_in_process.py' build_wheel 'C:\Users\JAMES~1.BOL\AppData\Local\Temp\tmp0bl7r6lz'
       cwd: C:\Users\JAMES~1.BOL\AppData\Local\Temp\pip-install-7bxukra4\skranger_e51b168e77dc40e3a215072a223ea594
  Complete output (25 lines):
  A setup.py file already exists. Using it.
  Traceback (most recent call last):
    File "C:\Users\James.Boland\AppData\Local\Temp\pip-install-7bxukra4\skranger_e51b168e77dc40e3a215072a223ea594\setup.py", line 2, in <module>
      from setuptools import setup
  ModuleNotFoundError: No module named 'setuptools'
  Traceback (most recent call last):
    File "C:\Users\James.Boland\AppData\Local\Continuum\miniconda3\envs\clv-models-env\lib\site-packages\pip\_vendor\pep517\_in_process.py", line 280, in <module>
      main()
    File "C:\Users\James.Boland\AppData\Local\Continuum\miniconda3\envs\clv-models-env\lib\site-packages\pip\_vendor\pep517\_in_process.py", line 263, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "C:\Users\James.Boland\AppData\Local\Continuum\miniconda3\envs\clv-models-env\lib\site-packages\pip\_vendor\pep517\_in_process.py", line 205, in build_wheel
      metadata_directory)
    File "C:\Users\JAMES~1.BOL\AppData\Local\Temp\pip-build-env-_gh4sqt8\overlay\Lib\site-packages\poetry\core\masonry\api.py", line 57, in build_wheel
      return unicode(WheelBuilder.make_in(poetry, Path(wheel_directory)))
    File "C:\Users\JAMES~1.BOL\AppData\Local\Temp\pip-build-env-_gh4sqt8\overlay\Lib\site-packages\poetry\core\masonry\builders\wheel.py", line 59, in make_in
      wb.build()
    File "C:\Users\JAMES~1.BOL\AppData\Local\Temp\pip-build-env-_gh4sqt8\overlay\Lib\site-packages\poetry\core\masonry\builders\wheel.py", line 89, in build
      self._build(zip_file)
    File "C:\Users\JAMES~1.BOL\AppData\Local\Temp\pip-build-env-_gh4sqt8\overlay\Lib\site-packages\poetry\core\masonry\builders\wheel.py", line 121, in _build
      self._run_build_command(setup)
    File "C:\Users\JAMES~1.BOL\AppData\Local\Temp\pip-build-env-_gh4sqt8\overlay\Lib\site-packages\poetry\core\masonry\builders\wheel.py", line 155, in _run_build_command
      str(self._path / "build"),
    File "C:\Users\James.Boland\AppData\Local\Continuum\miniconda3\envs\clv-models-env\lib\subprocess.py", line 363, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['C:/Users/James.Boland/AppData/Local/Continuum/miniconda3/envs/clv-models-env/python.exe', 'C:\\Users\\James.Boland\\AppData\\Local\\Temp\\pip-install-7bxukra4\\skranger_e51b168e77dc40e3a215072a223ea594\\setup.py', 'build', '-b', 'C:\\Users\\James.Boland\\AppData\\Local\\Temp\\pip-install-7bxukra4\\skranger_e51b168e77dc40e3a215072a223ea594\\build']' returned non-zero exit status 1.
  ----------------------------------------
  ERROR: Failed building wheel for skranger
Failed to build skranger
ERROR: Could not build wheels for skranger which use PEP 517 and cannot be installed directly

I'm using Python version 3.7.9 and pip version 20.3. I've also tried installing using the tar.gz and I get the same result.

[crflynn]

I don't have much experience with windows but I've spent many late nights trying to get skranger to build wheels for windows on appveyor. My WIP PR is here https://github.com/crflynn/skranger-wheels/pull/2. The problem is that appveyor doesn't easily support C++11 as far as I can tell, but we might be able to build wheels for windows using mingw following cython's docs here: https://cython.readthedocs.io/en/latest/src/tutorial/appendix.html.

[JamesBolandDing]

I might try to work using some *nix machine instead. It's where it needs to be able to run once in production anyway. Either way, I look forward to the results of your current PR.

I'm new to GitHub issues so: should I close this issue (since I'm happy as it stands) or leave it open (since I'm guessing Windows is still not working)?

[crflynn]

You can leave it open.

[emilhe]

Is there any progress on this issue? I would also like to run on Windows :)

[crflynn]

Not yet, unfortunately. I don't have a windows machine and my testing of building on appveyor has not worked out so far.

[TueVJ]

I managed to get some progress here following the directions in This stackoverflow answer for a different package with a similar issue.

Installing dependencies manually via pip install numpy threadpoolctl scipy joblib scikit-learn cython and then running pip install --no-build-isolation skranger gets me to the point where the package begins building, but it seems my build environment doesn't know what c++11 is: cl : Command line warning D9002 : ignoring unknown option '-std=c++11' Presumably the large number of warnings/errors are due to this.

This could be a workaround provided the user is able to compile the package using their build tools.

skranger_build.log

[crflynn]

I spent some more time on this again with no success. It seems that cython, windows, and c++11 is quite challenging as the current set of windows tooling doesn't directly support c++11, which is required for ranger to build. There is also the question of compatibility between using mingw or msvc which I don't fully understand. I got to the same point where either -std=c++11 is not a known compiler flag or that vcruntime140.dll was not found in PATH by the compiler.

[theovig]

I don't know if this can help, but I found an article on how to implement Ranger without skranger on Python here:https://www.inekto.fr/sigma/articles?id=ranger

[crflynn]

https://rpy2.github.io/ is also an option

[eddiebergman]

Have you considered building them directly on windows github action servers? It seems they have the tooling required. I could help set up a basic workflow that will produce downloadable windows wheels.

[crflynn]

I don't think those were available at the time but I will take a look. Happy to review a PR that solves this.

[eddiebergman]

I'm really not familiar with building skranger but I recently updated one of our libraries to build wheels which was fairly straightforward using cibuildwheel.

I'll leave this here for reference, it's our wheel building for ConfigSpace. I will not commit to this at the moment but if we decide to start using skranger in one of our libraries I can spend time on this.

Hope this info helps :)

[PauBadiaM]

Sorry to ask but has there been any update on this?

[crflynn]

Just set up a draft PR to test out cibuildwheel: #140. Looks fine except on windows and I think I finally see where the build issue is.

Again I'm not very familiar with windows or C++, but the error that I see looks like this: error C2668: 'ranger::make_unique': ambiguous call to overloaded function. This is also apparent in the build log posted earlier.

From what I understand ranger implements its own make_unique function here: https://github.com/imbs-hl/ranger/blob/daa8da58f85e44c90a39ebb98b7f9ad6a6fe65b4/src/utility.h#L539

As I understand it std::make_unique was added in C++14, but ranger uses C++11 so they wrote their own. It seems the compiler on windows is familiar with the std::make_unique, though, which raises this error. I assume the compiler uses C++14 by default, and AFAICT there is no way to explicitly set C++11 on MSCV. Additionally in the logs the compiler expressly ignores the -std:c++11 flag.

Related issue: https://github.com/imbs-hl/ranger/issues/376

[crflynn]

I released 0.8.0rc1 which has windows wheels (3.8, 3.9, 3.10). Please let me know if it works for you.