cython problem

[1rdsk1]

Hi, on my new windows 10 laptop, I installed all packages in requirements.txt and ran the cython build. Following your response to another issue I also installed visualcppbuildtools_full.exe.

(FMNM) C:\Users\x\Documents\FMNM>cd functions/cython (FMNM) C:\Users\x\Documents\FMNM\functions\cython>python setup.py build_ext --inplace running build_ext running build_ext (FMNM) C:\Users\x\Documents\FMNM\functions\cython>

When I run the imports at the start of 1.4 SDE - Heston model in Anaconda Jupyter, I get following error:

---

ModuleNotFoundError Traceback (most recent call last)

in 8 from functools import partial 9 from functions.probabilities import Heston_pdf, Q1, Q2 ---> 10 from functions.cython.cython_Heston import Heston_paths_log, Heston_paths 11 from scipy.optimize import minimize 12 ModuleNotFoundError: No module named 'functions.cython.cython_Heston' but that module is in the functions directory. I am puzzled as I installed FMNM and ran 1.4 without problem on a ubuntu virtualbox even though I had not installed visualcppbuildtools_full.exe on it.

[cantaro86]

Hi @1rdsk1 I am only now aware of this problem. On Linux it gives no errors. Try to have a look here. https://stackoverflow.com/questions/7508803/how-do-i-import-function-from-pyx-file-in-python Let me know if it works.

[1rdsk1]

Thanks. I added a first cell with: import pyximport pyximport.install()

Unfortunately that led to a different error below which makes little sense to me apart from possibly "FutureWarning: Cython directive 'language_level' not set". I have FMNM running in ubuntu fine and so this is more for info in case you can spot the issue?

C:\Users\x\anaconda3\lib\site-packages\Cython\Compiler\Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: C:\Users\x\Documents\FMNM\functions\cython\cython_Heston.pyx tree = Parsing.p_module(s, pxd, full_module_name)

DistutilsExecError Traceback (most recent call last) ~\anaconda3\lib\distutils_msvccompiler.py in compile(self, sources, output_dir, macros, include_dirs, debug, extra_preargs, extra_postargs, depends) 411 try: --> 412 self.spawn(args) 413 except DistutilsExecError as msg:

~\anaconda3\lib\distutils_msvccompiler.py in spawn(self, cmd) 517 os.environ['path'] = self._paths --> 518 return super().spawn(cmd) 519 finally:

~\anaconda3\lib\distutils\ccompiler.py in spawn(self, cmd) 909 def spawn(self, cmd): --> 910 spawn(cmd, dry_run=self.dry_run) 911

~\anaconda3\lib\distutils\spawn.py in spawn(cmd, search_path, verbose, dry_run) 37 elif os.name == 'nt': ---> 38 _spawn_nt(cmd, search_path, dry_run=dry_run) 39 else:

~\anaconda3\lib\distutils\spawn.py in _spawn_nt(cmd, search_path, verbose, dry_run) 79 cmd = executable ---> 80 raise DistutilsExecError( 81 "command %r failed with exit status %d" % (cmd, rc))

DistutilsExecError: command 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe' failed with exit status 2

During handling of the above exception, another exception occurred:

CompileError Traceback (most recent call last) ~\anaconda3\lib\site-packages\pyximport\pyximport.py in load_module(name, pyxfilename, pyxbuild_dir, is_package, build_inplace, language_level, so_path) 213 module_name = name --> 214 so_path = build_module(module_name, pyxfilename, pyxbuild_dir, 215 inplace=build_inplace, language_level=language_level)

~\anaconda3\lib\site-packages\pyximport\pyximport.py in build_module(name, pyxfilename, pyxbuild_dir, inplace, language_level) 185 from . import pyxbuild --> 186 so_path = pyxbuild.pyx_to_dll(pyxfilename, extension_mod, 187 build_in_temp=build_in_temp,

~\anaconda3\lib\site-packages\pyximport\pyxbuild.py in pyx_to_dll(filename, ext, force_rebuild, build_in_temp, pyxbuild_dir, setup_args, reload_support, inplace) 101 obj_build_ext = dist.get_command_obj("build_ext") --> 102 dist.run_commands() 103 so_path = obj_build_ext.get_outputs()[0]

~\anaconda3\lib\distutils\dist.py in run_commands(self) 965 for cmd in self.commands: --> 966 self.run_command(cmd) 967

~\anaconda3\lib\distutils\dist.py in run_command(self, command) 984 cmd_obj.ensure_finalized() --> 985 cmd_obj.run() 986 self.have_run[command] = 1

~\anaconda3\lib\site-packages\Cython\Distutils\old_build_ext.py in run(self) 185 --> 186 _build_ext.build_ext.run(self) 187

~\anaconda3\lib\distutils\command\build_ext.py in run(self) 339 # Now actually compile and link everything. --> 340 self.build_extensions() 341

~\anaconda3\lib\site-packages\Cython\Distutils\old_build_ext.py in build_extensions(self) 194 # Call original build_extensions --> 195 _build_ext.build_ext.build_extensions(self) 196

~\anaconda3\lib\distutils\command\build_ext.py in build_extensions(self) 448 else: --> 449 self._build_extensions_serial() 450

~\anaconda3\lib\distutils\command\build_ext.py in _build_extensions_serial(self) 473 with self._filter_build_errors(ext): --> 474 self.build_extension(ext) 475

~\anaconda3\lib\distutils\command\build_ext.py in build_extension(self, ext) 527 --> 528 objects = self.compiler.compile(sources, 529 output_dir=self.build_temp,

~\anaconda3\lib\distutils_msvccompiler.py in compile(self, sources, output_dir, macros, include_dirs, debug, extra_preargs, extra_postargs, depends) 413 except DistutilsExecError as msg: --> 414 raise CompileError(msg) 415

CompileError: command 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe' failed with exit status 2

During handling of the above exception, another exception occurred:

ImportError Traceback (most recent call last)

in 8 from functools import partial 9 from functions.probabilities import Heston_pdf, Q1, Q2 ---> 10 from functions.cython.cython_Heston import Heston_paths_log, Heston_paths 11 from scipy.optimize import minimize 12 ~\anaconda3\lib\site-packages\pyximport\pyximport.py in load_module(self, fullname) 457 else: 458 #print "MODULE", fullname --> 459 module = load_module(fullname, self.path, 460 self.pyxbuild_dir, 461 build_inplace=self.inplace, ~\anaconda3\lib\site-packages\pyximport\pyximport.py in load_module(name, pyxfilename, pyxbuild_dir, is_package, build_inplace, language_level, so_path) 229 name, traceback.format_exception_only(*sys.exc_info()[:2]))) 230 if sys.version_info[0] >= 3: --> 231 raise exc.with_traceback(tb) 232 else: 233 exec("raise exc, None, tb", {'exc': exc, 'tb': tb}) ~\anaconda3\lib\site-packages\pyximport\pyximport.py in load_module(name, pyxfilename, pyxbuild_dir, is_package, build_inplace, language_level, so_path) 212 else: 213 module_name = name --> 214 so_path = build_module(module_name, pyxfilename, pyxbuild_dir, 215 inplace=build_inplace, language_level=language_level) 216 mod = imp.load_dynamic(name, so_path) ~\anaconda3\lib\site-packages\pyximport\pyximport.py in build_module(name, pyxfilename, pyxbuild_dir, inplace, language_level) 184 185 from . import pyxbuild --> 186 so_path = pyxbuild.pyx_to_dll(pyxfilename, extension_mod, 187 build_in_temp=build_in_temp, 188 pyxbuild_dir=pyxbuild_dir, ~\anaconda3\lib\site-packages\pyximport\pyxbuild.py in pyx_to_dll(filename, ext, force_rebuild, build_in_temp, pyxbuild_dir, setup_args, reload_support, inplace) 100 try: 101 obj_build_ext = dist.get_command_obj("build_ext") --> 102 dist.run_commands() 103 so_path = obj_build_ext.get_outputs()[0] 104 if obj_build_ext.inplace: ~\anaconda3\lib\distutils\dist.py in run_commands(self) 964 """ 965 for cmd in self.commands: --> 966 self.run_command(cmd) 967 968 # -- Methods that operate on its Commands -------------------------- ~\anaconda3\lib\distutils\dist.py in run_command(self, command) 983 cmd_obj = self.get_command_obj(command) 984 cmd_obj.ensure_finalized() --> 985 cmd_obj.run() 986 self.have_run[command] = 1 987 ~\anaconda3\lib\site-packages\Cython\Distutils\old_build_ext.py in run(self) 184 optimization.disable_optimization() 185 --> 186 _build_ext.build_ext.run(self) 187 188 def build_extensions(self): ~\anaconda3\lib\distutils\command\build_ext.py in run(self) 338 339 # Now actually compile and link everything. --> 340 self.build_extensions() 341 342 def check_extensions_list(self, extensions): ~\anaconda3\lib\site-packages\Cython\Distutils\old_build_ext.py in build_extensions(self) 193 ext.sources = self.cython_sources(ext.sources, ext) 194 # Call original build_extensions --> 195 _build_ext.build_ext.build_extensions(self) 196 197 def cython_sources(self, sources, extension): ~\anaconda3\lib\distutils\command\build_ext.py in build_extensions(self) 447 self._build_extensions_parallel() 448 else: --> 449 self._build_extensions_serial() 450 451 def _build_extensions_parallel(self): ~\anaconda3\lib\distutils\command\build_ext.py in _build_extensions_serial(self) 472 for ext in self.extensions: 473 with self._filter_build_errors(ext): --> 474 self.build_extension(ext) 475 476 @contextlib.contextmanager ~\anaconda3\lib\distutils\command\build_ext.py in build_extension(self, ext) 526 macros.append((undef,)) 527 --> 528 objects = self.compiler.compile(sources, 529 output_dir=self.build_temp, 530 macros=macros, ~\anaconda3\lib\distutils\_msvccompiler.py in compile(self, sources, output_dir, macros, include_dirs, debug, extra_preargs, extra_postargs, depends) 412 self.spawn(args) 413 except DistutilsExecError as msg: --> 414 raise CompileError(msg) 415 416 return objects ImportError: Building module functions.cython.cython_Heston failed: ["distutils.errors.CompileError: command 'C:\\\\Program Files (x86)\\\\Microsoft Visual Studio 14.0\\\\VC\\\\BIN\\\\x86_amd64\\\\cl.exe' failed with exit status 2\n"]

[cantaro86]

Thanks @1rdsk1 , I don't have access to Windows so I can't test. Hope your comment can help other users in some way.

[lan-slot]

i'm new to python,and i'm very interested in this programe, i use jupyter notebook on a macbookpro. i confront with the same problem,too. can not 'from functions.cython.cython_functions import SOR', Error: No module named 'functions.cython.cython_functions' it seems it can not import " cython_functions.pyx" . and " cython_Heston.pyx", how to import ".pyx" into jupyter notebook? i tried many ways , including: import pyximport pyximport.install() but it didn't work

[lan-slot]

ModuleNotFoundError: No module named 'functions.cython.cython_Heston'

after trying so many times,i made it work by this "https://cython.readthedocs.io/en/latest/src/quickstart/build.html".

1. i install setuptools, "https://setuptools.readthedocs.io/en/latest/"
2. followed by this "https://cython.readthedocs.io/en/latest/src/quickstart/build.html", i use the original "setup.py" to generate a new "cython_Heston.cpython-37m-darwin.so" and "cython_Heston.c" by the code "!python setup.py build_ext --inplace"in jupyter notebook. i think may be the owner's original "cython_Heston.cpython-37m-x86_64-linux-gnu.so" did not suit our own computer. that's it. in the end,the notebook is wonderful.thanks to cantaro86.@cantaro86.

3.again , after trying so many times,i made it work on Windows. U need to install a "Build Tools for Visual Studio 2015 or 2017 or 20xx (version xxxxx)" from "https://visualstudio.microsoft.com/zh-hant/visual-cpp-build-tools/", so that the Windows can compile the ".pyx" document.

[cantaro86]

Thanks to you @lan-slot for making this solution available.

[xixilllll]

Thanks to @lan-slot ，I solve the problem in windows. You need to run "...\functions\cython\setup.py" in your terminal or cmd, then you will get the "cython_functions.cp38-win_amd64" and other required files. See https://cython.readthedocs.io/en/latest/src/quickstart/build.html

[YuanxiHe00]

Hi @lan-slot, I have the same problem ("'from functions.cython.cython_functions import SOR', Error: No module named 'functions.cython.cython_functions'") with my MacBook Pro, and I have tried so many times, but I'm still having the trouble, could you help me?

[cantaro86]

I decided to create a local package that compiles cython during installation. This should solve many problems, and the user is not required anymore to compile cython by himself.