search

aesara-devs / aesara

Aesara is a Python library for defining, optimizing, and efficiently evaluating mathematical expressions involving multi-dimensional arrays.
https://aesara.readthedocs.io
Other
1.18k stars 155 forks source link

tensor function optimization failure due to: constant_folding #457

Closed fanshi118 closed 3 years ago

fanshi118 commented 3 years ago

Ran into the following issue when trying to apply aesara.tensor functions to an array locally

Please provide a minimal, self-contained, and reproducible example.

import numpy as np

import aesara
import aesara.tensor as at

aesara.config.on_opt_error = "raise"

at.max(at.as_tensor_variable(np.array([[0., 1.], [1., 2.]]))).eval()

Please provide the full traceback of any errors.

Click to expand traceback WARNING (aesara.tensor.blas): Using NumPy C-API based implementation for BLAS functions. ERROR (aesara.graph.opt): Optimization failure due to: constant_folding ERROR (aesara.graph.opt): node: MaxAndArgmax{axis=(0, 1)}(TensorConstant{[[0. 1.] [1. 2.]]}) ERROR (aesara.graph.opt): TRACEBACK: ERROR (aesara.graph.opt): Traceback (most recent call last): File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py", line 2015, in process_node replacements = lopt.transform(fgraph, node) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py", line 1206, in transform return self.fn(*args, **kwargs) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/tensor/basic_opt.py", line 4355, in constant_folding thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[]) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/op.py", line 656, in make_thunk return self.make_c_thunk(node, storage_map, compute_map, no_recycling) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/op.py", line 622, in make_c_thunk outputs = cl.make_thunk( File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/basic.py", line 1204, in make_thunk cthunk, module, in_storage, out_storage, error_storage = self.__compile__( File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/basic.py", line 1139, in __compile__ thunk, module = self.cthunk_factory( File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/basic.py", line 1635, in cthunk_factory module = get_module_cache().module_from_key(key=key, lnk=self) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/cmodule.py", line 1198, in module_from_key module = lnk.compile_cmodule(location) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/basic.py", line 1544, in compile_cmodule module = c_compiler.compile_str( File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/cmodule.py", line 2553, in compile_str raise CompileError( aesara.link.c.exceptions.CompileError: Compilation failed (return status=1): /Users/sf/miniconda3/envs/aesara/bin/clang++ -dynamiclib -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -undefined dynamic_lookup -I/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/numpy/core/include -I/Users/sf/miniconda3/envs/aesara/include/python3.8 -I/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/c_code -L/Users/sf/miniconda3/envs/aesara/lib -fvisibility=hidden -o /Users/sf/.aesara/compiledir_macOS-11.0.1-x86_64-i386-64bit-i386-3.8.10-64/tmpwunj4hgp/m4b2ef76bb0f9461399355330a611097ca43c41e28c2d7fc7506e5efb00d6b0e5.so /Users/sf/.aesara/compiledir_macOS-11.0.1-x86_64-i386-64bit-i386-3.8.10-64/tmpwunj4hgp/mod.cpp In file included from /Users/sf/.aesara/compiledir_macOS-11.0.1-x86_64-i386-64bit-i386-3.8.10-64/tmpwunj4hgp/mod.cpp:1: In file included from /Users/sf/miniconda3/envs/aesara/include/python3.8/Python.h:25: /Users/sf/miniconda3/envs/aesara/bin/../include/c++/v1/stdio.h:107:15: fatal error: 'stdio.h' file not found #include_next ^~~~~~~~~ 1 error generated. ERROR (aesara.graph.opt): SeqOptimizer apply ERROR (aesara.graph.opt): Traceback: ERROR (aesara.graph.opt): Traceback (most recent call last): File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py", line 246, in apply sub_prof = optimizer.optimize(fgraph) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py", line 84, in optimize ret = self.apply(fgraph, *args, **kwargs) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py", line 2563, in apply sub_prof = gopt.apply(fgraph) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py", line 2126, in apply nb += self.process_node(fgraph, node) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py", line 2018, in process_node self.failure_callback( File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py", line 1916, in warn_inplace return NavigatorOptimizer.warn(exc, nav, repl_pairs, local_opt, node) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py", line 1902, in warn raise exc File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py", line 2015, in process_node replacements = lopt.transform(fgraph, node) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py", line 1206, in transform return self.fn(*args, **kwargs) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/tensor/basic_opt.py", line 4355, in constant_folding thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[]) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/op.py", line 656, in make_thunk return self.make_c_thunk(node, storage_map, compute_map, no_recycling) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/op.py", line 622, in make_c_thunk outputs = cl.make_thunk( File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/basic.py", line 1204, in make_thunk cthunk, module, in_storage, out_storage, error_storage = self.__compile__( File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/basic.py", line 1139, in __compile__ thunk, module = self.cthunk_factory( File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/basic.py", line 1635, in cthunk_factory module = get_module_cache().module_from_key(key=key, lnk=self) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/cmodule.py", line 1198, in module_from_key module = lnk.compile_cmodule(location) File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/basic.py", line 1544, in compile_cmodule module = c_compiler.compile_str( File "/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/cmodule.py", line 2553, in compile_str raise CompileError( aesara.link.c.exceptions.CompileError: Compilation failed (return status=1): /Users/sf/miniconda3/envs/aesara/bin/clang++ -dynamiclib -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -undefined dynamic_lookup -I/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/numpy/core/include -I/Users/sf/miniconda3/envs/aesara/include/python3.8 -I/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/c_code -L/Users/sf/miniconda3/envs/aesara/lib -fvisibility=hidden -o /Users/sf/.aesara/compiledir_macOS-11.0.1-x86_64-i386-64bit-i386-3.8.10-64/tmpwunj4hgp/m4b2ef76bb0f9461399355330a611097ca43c41e28c2d7fc7506e5efb00d6b0e5.so /Users/sf/.aesara/compiledir_macOS-11.0.1-x86_64-i386-64bit-i386-3.8.10-64/tmpwunj4hgp/mod.cpp In file included from /Users/sf/.aesara/compiledir_macOS-11.0.1-x86_64-i386-64bit-i386-3.8.10-64/tmpwunj4hgp/mod.cpp:1: In file included from /Users/sf/miniconda3/envs/aesara/include/python3.8/Python.h:25: /Users/sf/miniconda3/envs/aesara/bin/../include/c++/v1/stdio.h:107:15: fatal error: 'stdio.h' file not found #include_next ^~~~~~~~~ 1 error generated. You can find the C code in this temporary file: /var/folders/gl/s96rmcrx5nnc6qy4crb_h2xc0000gp/T/aesara_compilation_error_fz4nfdm9 --------------------------------------------------------------------------- CompileError Traceback (most recent call last) in 7 aesara.config.on_opt_error = "raise" 8 ----> 9 at.max(at.as_tensor_variable(np.array([[0., 1.], [1., 2.]]))).eval() ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/basic.py in eval(self, inputs_to_values) 550 inputs = tuple(sorted(inputs_to_values.keys(), key=id)) 551 if inputs not in self._fn_cache: --> 552 self._fn_cache[inputs] = function(inputs, self) 553 args = [inputs_to_values[param] for param in inputs] 554 ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/compile/function/__init__.py in function(inputs, outputs, mode, updates, givens, no_default_updates, accept_inplace, name, rebuild_strict, allow_input_downcast, profile, on_unused_input) 335 # note: pfunc will also call orig_function -- orig_function is 336 # a choke point that all compilation must pass through --> 337 fn = pfunc( 338 params=inputs, 339 outputs=outputs, ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/compile/function/pfunc.py in pfunc(params, outputs, mode, updates, givens, no_default_updates, accept_inplace, name, rebuild_strict, allow_input_downcast, profile, on_unused_input, output_keys) 522 inputs.append(si) 523 --> 524 return orig_function( 525 inputs, 526 cloned_outputs, ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/compile/function/types.py in orig_function(inputs, outputs, mode, accept_inplace, name, profile, on_unused_input, output_keys) 1970 try: 1971 Maker = getattr(mode, "function_maker", FunctionMaker) -> 1972 m = Maker( 1973 inputs, 1974 outputs, ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/compile/function/types.py in __init__(self, inputs, outputs, mode, accept_inplace, function_builder, profile, on_unused_input, fgraph, output_keys, name) 1615 ) 1616 else: -> 1617 optimizer_profile = optimizer(fgraph) 1618 1619 end_optimizer = time.time() ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py in __call__(self, fgraph) 91 92 """ ---> 93 return self.optimize(fgraph) 94 95 def add_requirements(self, fgraph): ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py in optimize(self, fgraph, *args, **kwargs) 82 """ 83 self.add_requirements(fgraph) ---> 84 ret = self.apply(fgraph, *args, **kwargs) 85 return ret 86 ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py in apply(self, fgraph) 255 except Exception as e: 256 if self.failure_callback: --> 257 self.failure_callback(e, self, optimizer) 258 continue 259 else: ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py in warn(exc, self, optimizer) 185 _logger.error(traceback.format_exc()) 186 if config.on_opt_error == "raise": --> 187 raise exc 188 elif config.on_opt_error == "pdb": 189 pdb.post_mortem(sys.exc_info()[2]) ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py in apply(self, fgraph) 244 nb_nodes_before = len(fgraph.apply_nodes) 245 t0 = time.time() --> 246 sub_prof = optimizer.optimize(fgraph) 247 l.append(float(time.time() - t0)) 248 sub_profs.append(sub_prof) ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py in optimize(self, fgraph, *args, **kwargs) 82 """ 83 self.add_requirements(fgraph) ---> 84 ret = self.apply(fgraph, *args, **kwargs) 85 return ret 86 ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py in apply(self, fgraph, start_from) 2561 nb = change_tracker.nb_imported 2562 t_opt = time.time() -> 2563 sub_prof = gopt.apply(fgraph) 2564 time_opts[gopt] += time.time() - t_opt 2565 sub_profs.append(sub_prof) ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py in apply(self, fgraph, start_from) 2124 continue 2125 current_node = node -> 2126 nb += self.process_node(fgraph, node) 2127 loop_t = time.time() - t0 2128 finally: ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py in process_node(self, fgraph, node, lopt) 2016 except Exception as e: 2017 if self.failure_callback is not None: -> 2018 self.failure_callback( 2019 e, self, [(x, None) for x in node.outputs], lopt, node 2020 ) ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py in warn_inplace(exc, nav, repl_pairs, local_opt, node) 1914 if isinstance(exc, InconsistencyError): 1915 return -> 1916 return NavigatorOptimizer.warn(exc, nav, repl_pairs, local_opt, node) 1917 1918 @staticmethod ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py in warn(exc, nav, repl_pairs, local_opt, node) 1900 # We always crash on AssertionError because something may be 1901 # seriously wrong if such an exception is raised. -> 1902 raise exc 1903 1904 @staticmethod ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py in process_node(self, fgraph, node, lopt) 2013 lopt = lopt or self.local_opt 2014 try: -> 2015 replacements = lopt.transform(fgraph, node) 2016 except Exception as e: 2017 if self.failure_callback is not None: ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/opt.py in transform(self, *args, **kwargs) 1204 1205 def transform(self, *args, **kwargs): -> 1206 return self.fn(*args, **kwargs) 1207 1208 def add_requirements(self, fgraph): ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/tensor/basic_opt.py in constant_folding(fgraph, node) 4353 compute_map[o] = [False] 4354 -> 4355 thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[]) 4356 required = thunk() 4357 # A node whose inputs are all provided should always return successfully ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/op.py in make_thunk(self, node, storage_map, compute_map, no_recycling, impl) 654 ) 655 try: --> 656 return self.make_c_thunk(node, storage_map, compute_map, no_recycling) 657 except (NotImplementedError, MethodNotDefined): 658 # We requested the c code, so don't catch the error. ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/graph/op.py in make_c_thunk(self, node, storage_map, compute_map, no_recycling) 620 print(f"Disabling C code for {self} due to unsupported float16") 621 raise NotImplementedError("float16") --> 622 outputs = cl.make_thunk( 623 input_storage=node_input_storage, output_storage=node_output_storage 624 ) ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/basic.py in make_thunk(self, input_storage, output_storage, storage_map) 1202 """ 1203 init_tasks, tasks = self.get_init_tasks() -> 1204 cthunk, module, in_storage, out_storage, error_storage = self.__compile__( 1205 input_storage, output_storage, storage_map 1206 ) ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/basic.py in __compile__(self, input_storage, output_storage, storage_map) 1137 input_storage = tuple(input_storage) 1138 output_storage = tuple(output_storage) -> 1139 thunk, module = self.cthunk_factory( 1140 error_storage, 1141 input_storage, ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/basic.py in cthunk_factory(self, error_storage, in_storage, out_storage, storage_map) 1633 for node in self.node_order: 1634 node.op.prepare_node(node, storage_map, None, "c") -> 1635 module = get_module_cache().module_from_key(key=key, lnk=self) 1636 1637 vars = self.inputs + self.outputs + self.orphans ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/cmodule.py in module_from_key(self, key, lnk) 1196 try: 1197 location = dlimport_workdir(self.dirname) -> 1198 module = lnk.compile_cmodule(location) 1199 name = module.__file__ 1200 assert name.startswith(location) ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/basic.py in compile_cmodule(self, location) 1542 try: 1543 _logger.debug(f"LOCATION {location}") -> 1544 module = c_compiler.compile_str( 1545 module_name=mod.code_hash, 1546 src_code=src_code, ~/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/cmodule.py in compile_str(module_name, src_code, location, include_dirs, lib_dirs, libs, preargs, py_module, hide_symbols) 2551 # difficult to read. 2552 # compile_stderr = compile_stderr.replace("\n", ". ") -> 2553 raise CompileError( 2554 f"Compilation failed (return status={status}):\n{' '.join(cmd)}\n{compile_stderr}" 2555 ) CompileError: Compilation failed (return status=1): /Users/sf/miniconda3/envs/aesara/bin/clang++ -dynamiclib -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -undefined dynamic_lookup -I/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/numpy/core/include -I/Users/sf/miniconda3/envs/aesara/include/python3.8 -I/Users/sf/miniconda3/envs/aesara/lib/python3.8/site-packages/aesara/link/c/c_code -L/Users/sf/miniconda3/envs/aesara/lib -fvisibility=hidden -o /Users/sf/.aesara/compiledir_macOS-11.0.1-x86_64-i386-64bit-i386-3.8.10-64/tmpwunj4hgp/m4b2ef76bb0f9461399355330a611097ca43c41e28c2d7fc7506e5efb00d6b0e5.so /Users/sf/.aesara/compiledir_macOS-11.0.1-x86_64-i386-64bit-i386-3.8.10-64/tmpwunj4hgp/mod.cpp In file included from /Users/sf/.aesara/compiledir_macOS-11.0.1-x86_64-i386-64bit-i386-3.8.10-64/tmpwunj4hgp/mod.cpp:1: In file included from /Users/sf/miniconda3/envs/aesara/include/python3.8/Python.h:25: /Users/sf/miniconda3/envs/aesara/bin/../include/c++/v1/stdio.h:107:15: fatal error: 'stdio.h' file not found #include_next ^~~~~~~~~ 1 error generated.

Please provide any additional information below.

I installed mkl-service in addition to aesara. Could I be missing something else?

Versions and main components

brandonwillard commented 3 years ago

I installed mkl-service in addition to aesara. Could I be missing something else?

I doubt it; mkl and other BLAS/LAPACK-related things don't appear to be involved—at least not directly.

fanshi118 commented 3 years ago

Issue resolved when I installed the master branch of aesara in a new virtual environment. There must have been some version issues with my old one.