Open SteveDiamond opened 8 years ago
bodono
commented
8 years ago This is awesome, thanks for writing these recipes. To clarify - where does the mkl come from that should be linked against? Is it a conda distribution or must the user download mkl manually? What are the full set of steps that I should try on a clean windows install to replicate this (python version, conda install, mkl etc.). I have an old windows laptop I can test on.
SteveDiamond
commented
8 years ago Anaconda comes with mkl now on all platforms. The default now is for numpy, scipy, etc to link against mkl. So ideally SCS would only link against mkl as well. This happens on linux. On Mac it links against Accelerate as well, but that's fine.
On a clean windows environment you should install anaconda, run conda install conda-build, clone the cvxgrp/conda-recipes repo, then finally run conda build scs inside the conda-recipes folder.
You can test out the SCS you built by running conda install --use-local scs. Then you can import SCS in Python. The key test is whether SCS works for SDPs. It's easy to get everything working except linking to blas/lapack.
bodono
commented
8 years ago As far as I can tell this is an issue with the numpy conda package. SCS uses numpy.distutils.system_info function get_info to tell it where the blas/lapack libraries it should link against are located. Numpy stores that information in a site.cfg file. But it appears this is broken in the conda numpy package. On a clean windows machine I installed anaconda, then installed numpy via conda install numpy. Then in a python interpreter I run:
>>> from numpy.distutils.system_info import get_info
>>> get_info('blas_opt')
C:\Users\23190281\Miniconda2\lib\site-packages\numpy\distutils\system_info.py:63
9: UserWarning: Specified path C:\Minonda\envs\_build\Library\lib is invalid.
warnings.warn('Specified path %s is invalid.' % d)
C:\Users\23190281\Miniconda2\lib\site-packages\numpy\distutils\system_info.py:63
9: UserWarning: Specified path C:\Minonda\envs\_build\Library\include is invalid
.
warnings.warn('Specified path %s is invalid.' % d)
C:\Users\23190281\Miniconda2\lib\site-packages\numpy\distutils\system_info.py:16
46: UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting
the ATLAS environment variable.
warnings.warn(AtlasNotFoundError.__doc__)
C:\Users\23190281\Miniconda2\lib\site-packages\numpy\distutils\system_info.py:16
55: UserWarning:
Blas (http://www.netlib.org/blas/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [blas]) or by setting
the BLAS environment variable.
warnings.warn(BlasNotFoundError.__doc__)
C:\Users\23190281\Miniconda2\lib\site-packages\numpy\distutils\system_info.py:16
58: UserWarning:
Blas (http://www.netlib.org/blas/) sources not found.
Directories to search for the sources can be specified in the
numpy/distutils/site.cfg file (section [blas_src]) or by setting
the BLAS_SRC environment variable.
warnings.warn(BlasSrcNotFoundError.__doc__)This is before I've even downloaded SCS. It's looking for the libs and includes in C:\Minonda\envs\_build, which doesn't exist on my machine.
After some digging I found this part of the guide in conda about making packages relocatable, which says that the build prefix is replaced in any file that contains it with a placeholder and the file is added to the list in info/has_prefix. If you examine the packages from the conda package repo you can inspect the windows and linux numpy packages manually. In the linux one, it has an info/has_prefix file which has
/opt/anaconda1anaconda2anaconda3 text lib/python2.7/site-packages/numpy/distutils/site.cfgand the site.cfg file has
[mkl]
library_dirs = /opt/anaconda1anaconda2anaconda3/lib
include_dirs = /opt/anaconda1anaconda2anaconda3/include
lapack_libs = mkl_lapack95_lp64
mkl_libs = mkl_intel_lp64, mkl_intel_thread, mkl_core, iomp5In other words the has_prefix file is telling conda to replace the placeholder string /opt/anaconda1anaconda2anaconda3 with the real prefix when it is installing it (since that's when it will be known).
However the latest windows numpy packages (e.g. numpy-1.11.1-py27_1.tar.bz2) do not have a has_prefix file, so it doesn't do the prefix swapping. The site.cfg file simply has
[mkl]
include_dirs = C:\Minonda\envs\_build\Library\include
library_dirs = C:\Minonda\envs\_build\Library\lib
lapack_libs = mkl_lapack95_lp64
mkl_libs = mkl_core_dll, mkl_intel_lp64_dll, mkl_intel_thread_dllBut that directory doesn't exist, and the prefix is not getting replaced, so it ends up with the error above.
However, an older windows version of numpy, numpy-1.11.0-py27_2 does have an info/has_prefix file, that has
"C:\Minonda\envs\_build" text "Lib/site-packages/numpy/distutils/site.cfg"Which is saying to swap out the C:\Minonda\envs\_build for the real prefix in the site.cfg file. So you would think that would replace the path correctly, however it still throws an error for me because the path that it replaces it with is also invalid (doesn't exist, and mixes forwards and backwards slashes in the path).
I think the message here is that the conda numpy packages for windows are untested and broken, and appear to have been broken in different ways for a long time. I guess the right thing to do is flag it with the anaconda team. I never even got to the point of installing SCS on the system!
ghost
commented
8 years ago can you try the same trick using another package that links to lapack/blas (e.g. scipy)? maybe that one is not broken. or really any other one. you can specify " build requirements" in a conda recipe, packages that are installed temporarily when building the package but not when installing it..
Enzo
On Aug 13, 2016, at 06:39, bodono notifications@github.com wrote:
As far as I can tell this is an issue with the numpy conda package. SCS uses numpy.distutils.system_info function get_info to tell it where the blas/lapack libraries it should link against are located. Numpy stores that information in a site.cfg file. But it appears this is broken in the conda numpy package. On a clean windows machine I installed anaconda, then installed numpy via conda install numpy. Then in a python interpreter I run:
from numpy.distutils.system_info import get_info get_info('blas_opt') C:\Users\23190281\Miniconda2\lib\site-packages\numpy\distutils\system_info.py:63 9: UserWarning: Specified path C:\Minonda\envs_build\Library\lib is invalid. warnings.warn('Specified path %s is invalid.' % d) C:\Users\23190281\Miniconda2\lib\site-packages\numpy\distutils\system_info.py:63 9: UserWarning: Specified path C:\Minonda\envs_build\Library\include is invalid . warnings.warn('Specified path %s is invalid.' % d) C:\Users\23190281\Miniconda2\lib\site-packages\numpy\distutils\system_info.py:16 46: UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [atlas]) or by setting the ATLAS environment variable. warnings.warn(AtlasNotFoundError.doc) C:\Users\23190281\Miniconda2\lib\site-packages\numpy\distutils\system_info.py:16 55: UserWarning: Blas (http://www.netlib.org/blas/) libraries not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [blas]) or by setting the BLAS environment variable. warnings.warn(BlasNotFoundError.doc) C:\Users\23190281\Miniconda2\lib\site-packages\numpy\distutils\system_info.py:16 58: UserWarning: Blas (http://www.netlib.org/blas/) sources not found. Directories to search for the sources can be specified in the numpy/distutils/site.cfg file (section [blas_src]) or by setting the BLAS_SRC environment variable. warnings.warn(BlasSrcNotFoundError.doc) This is before I've even downloaded SCS. It's looking for the libs and includes in C:\Minonda\envs_build, which doesn't exist on my machine.
After some digging I found this part of the guide in conda about making packages relocatable, which says that the build prefix is replaced in any file that contains it with a placeholder and the file is added to the list in info/has_prefix. If you examine the packages from the conda package repo you can inspect the windows and linux numpy packages manually. In the linux one, it has an info/has_prefix file which has
/opt/anaconda1anaconda2anaconda3 text lib/python2.7/site-packages/numpy/distutils/site.cfg and the site.cfg file has
[mkl] library_dirs = /opt/anaconda1anaconda2anaconda3/lib include_dirs = /opt/anaconda1anaconda2anaconda3/include lapack_libs = mkl_lapack95_lp64 mkl_libs = mkl_intel_lp64, mkl_intel_thread, mkl_core, iomp5 In other words the has_prefix file is telling conda to replace the placeholder string /opt/anaconda1anaconda2anaconda3 with the real prefix when it is installing it (since that's when it will be known).
However the latest windows numpy packages (e.g. numpy-1.11.1-py27_1.tar.bz2) do not have a has_prefix file, so it doesn't do the prefix swapping. The site.cfg file simply has
[mkl] include_dirs = C:\Minonda\envs_build\Library\include library_dirs = C:\Minonda\envs_build\Library\lib lapack_libs = mkl_lapack95_lp64 mkl_libs = mkl_core_dll, mkl_intel_lp64_dll, mkl_intel_thread_dll But that directory doesn't exist, and the prefix is not getting replaced, so it ends up with the error above.
However, an older windows version of numpy, numpy-1.11.0-py27_2 does have an info/has_prefix file, that has
"C:\Minonda\envs_build" text "Lib/site-packages/numpy/distutils/site.cfg" Which is saying to swap out the C:\Minonda\envs_build for the real prefix in the site.cfg file. So you would think that would replace the path correctly, however it still throws an error for me because the path that it replaces it with is also invalid (doesn't exist, and mixes forwards and backwards slashes in the path).
I think the message here is that the conda numpy packages for windows are untested and broken, and appear to have been broken in different ways for a long time. I guess the right thing to do is flag it with the anaconda team. I never even got to the point of installing SCS on the system!
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
bodono
commented
8 years ago As far as I know other packages don't expose that information like numpy does. And scipy uses the same get_info functionality from numpy to build: https://github.com/scipy/scipy/search?utf8=%E2%9C%93&q=get_info
bodono
commented
8 years ago One potential workaround that would work for all platforms is for the conda recipe for SCS to bypass numpy and get the lapack / blas libs directly. SCS will use the environment variables BLAS_LAPACK_LIB_PATHS and BLAS_LAPACK_LIBS to link with if they are set (before calling get_info). The conda recipe would have to install mkl first, figure out where the libraries are then set the environment variables accordingly and call setup.py. This numpy conda recipe file has all the information you need (since it's what get_info relies on).
bodono
commented
8 years ago To spell it out, the build.sh and bld.bat files would have to be something along the lines of
export BLAS_LAPACK_LIB_PATHS = $PREFIX/lib
export BLAS_LAPACK_LIBS = "mkl_lapack95_lp64:mkl_intel_lp64:mkl_intel_thread:mkl_core:iomp5"
python setup.py install
clagms
commented
7 years ago I was able to install scs on windows anaconda (python 2.7), linking to the lapack/blas libraries. However, there are a few quirks in the setup process, which conspire to make this a nightmare.
I'll detail the steps I followed to successfully install the 32 bits version, and potential problems with installing the 64 bits version.
Steps that I followed for the 32 bits installation:
env_vars['library_dirs'] = [...] to env_vars['library_dirs'] = [env_lib_dirs]. The split causes problems with paths like C:\something, splitting them into "C" and "\something".export BLAS_LAPACK_LIB_PATHS = C:\blas_lapack
export BLAS_LAPACK_LIBS = libblas:liblapack:liblapacke (Note that no extension is added)
python setup.py install[...]Visual C++ for Python\9.0\VC\Bin\link.exe [...] /LIBPATH:C:\blas_lapack [...] libblas.lib liblapack.lib liblapacke.lib [...]
If you get errors at this stage, see the troubleshooting below.scs-1.2[...]py2.7-win32.egg.
Copy the libblas.dll, libblas.lib, liblapack.dll, liblapack.lib, liblapacke.dll, liblapacke.lib, files to that folder.scs-1.2[...]py2.7-win32.egg folder. These ensure that blas and lapack can be loaded when you import scs in python.Issues with the 64 bits installation:
Other issues encountered:
--extraverbose argument does not work. The compiler throws errors like ../src\cones.c(158) : error C2065: '__func__' : undeclared identifier.Troubleshooting:
'__func__' : undeclared identifier. This is caused by using the --extraverbose for the installation.I hope this helps anyone trying to install scs on windows!
bodono
commented
7 years ago Thanks for this very detailed guide! It will really help other people who want to install on windows anaconda. What steps do you think can we take in the SCS codebase to make this easier?
clagms
commented
7 years ago I'm not sure how to make the process completely automated. However, as starting points:
bodono
commented
7 years ago I have made the change as in step 6, and also hopefully fixed the --extraverbose issue.
SteveDiamond
commented
7 years ago @bodono it looks like you got installation working on windows. Could you cut a new SCS version and put it on PyPi?
bodono
commented
7 years ago Ok, I didn't change much but I will cut a new release to PyPi when I get a chance.
bodono
commented
7 years ago Just uploaded version 1.2.7.
Boscop
commented
3 years ago Hi, I tried to install scs via conda on Windows 10, but I get an error:
Collecting scs==2.1.2
Downloading scs-2.1.2.tar.gz (3.5 MB)
Pip subprocess error:
The system cannot find the path specified.
The system cannot find the path specified.
ERROR: Command errored out with exit status 1:
command: 'D:\miniconda3\envs\ml4t\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\me\\AppData\\Local\\Temp\\pip-install-7l7nnm4m\\scs_148cd26a27f44bf684e2636cadf70489\\setup.py'"'"'; __file__='"'"'C:\\Users\\me\\AppData\\Local\\Temp\\pip-install-7l7nnm4m\\scs_148cd26a27f44bf684e2636cadf70489\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\me\AppData\Local\Temp\pip-pip-egg-info-7mktfr4j'
cwd: C:\Users\me\AppData\Local\Temp\pip-install-7l7nnm4m\scs_148cd26a27f44bf684e2636cadf70489\
Complete output (97 lines):
Namespace(blas64=False, extraverbose=False, float32=False, gpu=False, int32=False, scs=False)
running egg_info
creating C:\Users\me\AppData\Local\Temp\pip-pip-egg-info-7mktfr4j\scs.egg-info
writing C:\Users\me\AppData\Local\Temp\pip-pip-egg-info-7mktfr4j\scs.egg-info\PKG-INFO
writing dependency_links to C:\Users\me\AppData\Local\Temp\pip-pip-egg-info-7mktfr4j\scs.egg-info\dependency_links.txt
writing requirements to C:\Users\me\AppData\Local\Temp\pip-pip-egg-info-7mktfr4j\scs.egg-info\requires.txt
writing top-level names to C:\Users\me\AppData\Local\Temp\pip-pip-egg-info-7mktfr4j\scs.egg-info\top_level.txt
writing manifest file 'C:\Users\me\AppData\Local\Temp\pip-pip-egg-info-7mktfr4j\scs.egg-info\SOURCES.txt'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\me\AppData\Local\Temp\pip-install-7l7nnm4m\scs_148cd26a27f44bf684e2636cadf70489\setup.py", line 229, in <module>
run_install()
File "C:\Users\me\AppData\Local\Temp\pip-install-7l7nnm4m\scs_148cd26a27f44bf684e2636cadf70489\setup.py", line 226, in run_install
install_scs()
File "C:\Users\me\AppData\Local\Temp\pip-install-7l7nnm4m\scs_148cd26a27f44bf684e2636cadf70489\setup.py", line 201, in install_scs
setup(
File "D:\miniconda3\envs\ml4t\lib\site-packages\setuptools\__init__.py", line 163, in setup
return distutils.core.setup(**attrs)
File "D:\miniconda3\envs\ml4t\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "D:\miniconda3\envs\ml4t\lib\distutils\dist.py", line 966, in run_commands
self.run_command(cmd)
File "D:\miniconda3\envs\ml4t\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "D:\miniconda3\envs\ml4t\lib\site-packages\setuptools\command\egg_info.py", line 297, in run
self.find_sources()
File "D:\miniconda3\envs\ml4t\lib\site-packages\setuptools\command\egg_info.py", line 304, in find_sources
mm.run()
File "D:\miniconda3\envs\ml4t\lib\site-packages\setuptools\command\egg_info.py", line 535, in run
self.add_defaults()
File "D:\miniconda3\envs\ml4t\lib\site-packages\setuptools\command\egg_info.py", line 571, in add_defaults
sdist.add_defaults(self)
File "D:\miniconda3\envs\ml4t\lib\distutils\command\sdist.py", line 228, in add_defaults
self._add_defaults_ext()
File "D:\miniconda3\envs\ml4t\lib\distutils\command\sdist.py", line 311, in _add_defaults_ext
build_ext = self.get_finalized_command('build_ext')
File "D:\miniconda3\envs\ml4t\lib\distutils\cmd.py", line 299, in get_finalized_command
cmd_obj.ensure_finalized()
File "D:\miniconda3\envs\ml4t\lib\distutils\cmd.py", line 107, in ensure_finalized
self.finalize_options()
File "C:\Users\me\AppData\Local\Temp\pip-install-7l7nnm4m\scs_148cd26a27f44bf684e2636cadf70489\setup.py", line 112, in finalize_options
blas_info, lapack_info = get_infos()
File "C:\Users\me\AppData\Local\Temp\pip-install-7l7nnm4m\scs_148cd26a27f44bf684e2636cadf70489\setup.py", line 82, in get_infos
blas_info = get_info('blas_opt')
File "D:\miniconda3\envs\ml4t\lib\site-packages\numpy\distutils\system_info.py", line 584, in get_info
return cl().get_info(notfound_action)
File "D:\miniconda3\envs\ml4t\lib\site-packages\numpy\distutils\system_info.py", line 844, in get_info
self.calc_info()
File "D:\miniconda3\envs\ml4t\lib\site-packages\numpy\distutils\system_info.py", line 1989, in calc_info
if self._calc_info(blas):
File "D:\miniconda3\envs\ml4t\lib\site-packages\numpy\distutils\system_info.py", line 1981, in _calc_info
return getattr(self, '_calc_info_{}'.format(name))()
File "D:\miniconda3\envs\ml4t\lib\site-packages\numpy\distutils\system_info.py", line 1932, in _calc_info_openblas info = get_info('openblas')
File "D:\miniconda3\envs\ml4t\lib\site-packages\numpy\distutils\system_info.py", line 584, in get_info
return cl().get_info(notfound_action)
File "D:\miniconda3\envs\ml4t\lib\site-packages\numpy\distutils\system_info.py", line 844, in get_info
self.calc_info()
File "D:\miniconda3\envs\ml4t\lib\site-packages\numpy\distutils\system_info.py", line 2195, in calc_info
info = self._calc_info()
File "D:\miniconda3\envs\ml4t\lib\site-packages\numpy\distutils\system_info.py", line 2166, in _calc_info
f = new_fcompiler(c_compiler=c)
File "D:\miniconda3\envs\ml4t\lib\site-packages\numpy\distutils\fcompiler\__init__.py", line 874, in new_fcompiler load_all_fcompiler_classes()
File "D:\miniconda3\envs\ml4t\lib\site-packages\numpy\distutils\fcompiler\__init__.py", line 775, in load_all_fcompiler_classes
__import__ (module_name)
File "D:\miniconda3\envs\ml4t\lib\site-packages\numpy\distutils\fcompiler\compaq.py", line 54, in <module>
class CompaqVisualFCompiler(FCompiler):
File "D:\miniconda3\envs\ml4t\lib\site-packages\numpy\distutils\fcompiler\compaq.py", line 78, in CompaqVisualFCompiler
m.initialize()
File "D:\miniconda3\envs\ml4t\lib\site-packages\numpy\distutils\msvccompiler.py", line 51, in initialize
os.environ['lib'] = _merge(environ_lib, os.environ['lib'])
File "D:\miniconda3\envs\ml4t\lib\os.py", line 675, in __getitem__
raise KeyError(key) from None
KeyError: 'lib'
blas_info:
libraries = ['cblas', 'blas', 'cblas', 'blas', 'cblas', 'blas']
library_dirs = ['D:/miniconda3/envs/ml4t\\Library\\lib']
include_dirs = ['D:/miniconda3/envs/ml4t\\Library\\include']
language = f77
define_macros = [('HAVE_CBLAS', None)]
blas_opt_info:
define_macros = [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]
libraries = ['cblas', 'blas', 'cblas', 'blas', 'cblas', 'blas']
library_dirs = ['D:/miniconda3/envs/ml4t\\Library\\lib']
include_dirs = ['D:/miniconda3/envs/ml4t\\Library\\include']
language = f77
lapack_info:
libraries = ['lapack', 'blas', 'lapack', 'blas']
library_dirs = ['D:/miniconda3/envs/ml4t\\Library\\lib']
language = f77
lapack_opt_info:
libraries = ['lapack', 'blas', 'lapack', 'blas', 'cblas', 'blas', 'cblas', 'blas', 'cblas', 'blas']
library_dirs = ['D:/miniconda3/envs/ml4t\\Library\\lib']
language = f77
define_macros = [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]
include_dirs = ['D:/miniconda3/envs/ml4t\\Library\\include']
----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/1a/72/33be87cce255d4e9dbbfef547e9fd6ec7ee94d0d0910bb2b13badea3fbbe/scs-2.1.2.tar.gz#sha256=667ed6019bb4e2f925bd9291161d2c61cc0077443094437b703ea905333fd585 (from https://pypi.org/simple/scs/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement scs==2.1.2 (from versions: 1.0, 1.0.1, 1.0.2, 1.0.3, 1.0.5, 1.0.6, 1.0.7, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.7, 1.2.1, 1.2.2, 1.2.3, 1.2.5, 1.2.6, 1.2.7, 2.0.1, 2.0.2, 2.1.0, 2.1.1, 2.1.1.post2, 2.1.2, 2.1.3, 2.1.4)
ERROR: No matching distribution found for scs==2.1.2
failed
CondaEnvException: Pip failedI don't know why the script is looking for Visual Studio 2017, I only have 2019.
The folder C:\Program Files (x86)\Microsoft Visual Studio\2017 doesn't exist, only C:\Program Files (x86)\Microsoft Visual Studio\2019:
Apparently the script doesn't find MSVC through registry lookup but I found this registry key:
Looking at this:
https://github.com/python/cpython/blob/e14d5ae5447ae28fc4828a9cee8e9007f9c30700/Lib/distutils/msvccompiler.py#L598
if I understand it correctly, it means that my VC folder in the above screenshot should have a child folder named VC_OBJECTS_PLATFORM_INFO, but it doesn't have one, for some reason.
(Btw, when I search in the whole registry for VC_OBJECTS_PLATFORM_INFO it finds nothing.)
Maybe in MSVC 2019 the key VC_OBJECTS_PLATFORM_INFO does not exist anymore, and the MSVC detection script needs to be adapted?
(Btw, more context here: https://github.com/stefan-jansen/machine-learning-for-trading/issues/177)
Any idea how to make it work? :) I would greatly appreciate it!
h-vetinari
commented
3 years ago When installing with conda, you should need neither the download from PyPI, nor a compiler. Something is definitely off here.
Could you please post the outputs of (within the environment that you want to install scs in) of conda info & conda list, as well as the exact command you're trying to use to install.
Ideally, to keep things readable, please copy the outputs of conda info and conda list in something like the following (in the text-view of your GH comment):
<details>
<summary>Output of <code>conda info</code></summary>
`` ` [remove space between backticks]
[output of conda info]
`` ` [remove space between backticks]
</details>@h-vetinari Thanks for your help, I really appreciate it :) (I'm new to the conda ecosystem, coming from Rust.)
This is the output:
conda infoconda listThe exact command I'm running, that gave the scs error, is:
mamba env create -f installation/windows/ml4t.yml
as described here. This ml4t.yml file can be found here: https://github.com/stefan-jansen/machine-learning-for-trading/blob/2fe1120c9c71658b00a2e620f11836b8c4ecdf97/installation/windows/ml4t.yml#L474
h-vetinari
commented
3 years ago Well.. that environment yaml file contains several things that are not ideal, chief of which is putting scs under the pip-section, when it is easily available under conda-forge. Mixing pip & conda is heavily discouraged -- pip does not understand conda, and will trample over things and can easily bust the environment.
Same goes for osqp, ecos, cvxpy & qdldl. Please move them from the pip-section up into the main section (handled by conda), also removing the .post0.
Finally, this environment is not following best practices that there should be as few channels as possible. conda-forge & anaconda defaults are compatible with each other (at great effort to make sure everything is ABI-compatible), but other channels may for all intents and purposes contain stuff that blows up your setup. I didn't check all packages in that environment, but almost everything should be available from conda-forge, without the need for extra channels.
Aside from trying if things work with:
channels:
- conda-forge
- defaults
- anaconda
- ml4t # assuming this is necessary; if there are custom packages that aren't in the above channels
[and nothing more]I would at the very least recommend ordering conda-forge & defaults first in your channel list, as they represent the highest-quality packaging you can expect for - essentially - any package (actually, conda-forge packaging even assumes conda config --set channel_priority strict for the most part, and little if any effort will be made if other packages break your environment without this setting).
h-vetinari
commented
3 years ago @Boscop, any success?
Boscop
commented
3 years ago Haven't had time to try it yet but I'll try it as soon as I'm back. Thanks :)
@enzobusseti and I have been working on making conda packages for cvxpy and all its dependencies. We are so so close, and would really appreciate help with Windows support. Here's the conda recipes we've made: https://github.com/cvxgrp/conda-recipes We've used these recipes to push conda binaries to https://anaconda.org/cvxgrp They all work except for SCS on windows. The binary doesn't link against mkl for blas/lapack some reason. It does link against mkl on linux.
You should be able to run "conda build scs" inside the conda-recipes folder and have it just work. If anyone can figure out how to get a proper windows build I would be incredibly appreciative. Remember it's not enough for the package to build, it also has to link against mkl.