Unknown compiler: x86_64-linux-gnu-gcc

[salvadormrf]

I was trying to run the basic sum example, but seems the compiler does not get detected.

(test)sf@sf-MacBookPro:~/Projects/test/hope_test$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=13.10
DISTRIB_CODENAME=saucy
DISTRIB_DESCRIPTION="Ubuntu 13.10"

#!/usr/bin/env python

from hope import jit

@jit
def sum(x, y):
    return x + y

(test)sf@sf-MacBookPro:~/Projects/test/hope_test$ python test.py 
Traceback (most recent call last):
  File "test.py", line 3, in <module>
    from hope import jit
  File "/home/sf/.virtualenvs/test/local/lib/python2.7/site-packages/hope/__init__.py", line 11, in <module>
    from hope import config
  File "/home/sf/.virtualenvs/test/local/lib/python2.7/site-packages/hope/config.py", line 9, in <module>
    cxxflags = get_cxxflags()
  File "/home/sf/.virtualenvs/test/local/lib/python2.7/site-packages/hope/options.py", line 72, in get_cxxflags
    raise UnsupportedCompilerException("Unknown compiler: {0}".format(compiler_name))
hope.exceptions.UnsupportedCompilerException: Unknown compiler: x86_64-linux-gnu-gcc

(test)sf@sf-MacBookPro:~/Projects/test/hope_test$ python -c "from distutils.ccompiler import new_compiler; print new_compiler().compiler[0].split('/')[-1]"
cc

[iankronquist]

I fired up an ubuntu VM and can confirm this is an issue.

[fermat618]

the same problem in Debian Jessie

[iankronquist]

@fermat618 The issue is that hope doesn't really support linux right now. When it queries a python library for your C compiler it gets a compiler it doesn't know what to do with. I have a PR open to add CXXFLAGS for a generic x86_64 linux target. If you'd like to try it out, check out my fork: https://github.com/iankronquist/hope/tree/linux-compiler-support-3

[cosmo-ethz]

So far, we have been using the package mostly on OS X, ScientificLinux and Debian boxes where the current compiler detection seems to do its job. However, we have seen this error on Ubuntu VM’s before and it seemed that the compiler that is found heavily depends on how the system is set up. We definitely need to figure out how to deal with this situation.

However, this problem can be overcome by exporting the CC and CXX environment variables (e.g. export CC=<path-to-your-gcc> )

[cosmo-ethz]

@salvadormrf , @iankronquist with my latest changes HOPE should support x86_64-linux-gnu-gcc.

I check the fix on a Ubuntu box. @fermat618 would be great if you could check this fix on your Debian machine as on mine the old compiler detection seemed to work.

[fermat618]

@cosmo-ethz this bug seems to be fixed, but I still cannot use hope.

In [1]: from hope import jit

In [2]: add = jit(lambda x, y: x + y)

In [3]: add(1,2
   ...:
KeyboardInterrupt

In [3]: add(1, 2)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-3-a1cbb1df9896> in <module>()
----> 1 add(1, 2)

/usr/local/lib/python3.4/dist-packages/hope-0.3.0-py3.4.egg/hope/_wrapper.py in _hope_callback(*args)
     69
     70         def _hope_callback(*args):
---> 71             return self(*args) if self.cache is None else self.cache(*args)
     72         self.callback = _hope_callback
     73         setattr(cache, str(id(self.callback)), fkt)

/usr/local/lib/python3.4/dist-packages/hope-0.3.0-py3.4.egg/hope/_wrapper.py in __call__(self, *args)
     75
     76     def __call__(self, *args):
---> 77         ASTTransformer(self.modtoken).module_visit(self.fkt, args)
     78
     79         if config.optimize:

/usr/local/lib/python3.4/dist-packages/hope-0.3.0-py3.4.egg/hope/_transformer.py in module_visit(self, fkt, args, argid)
    621         self.fkt, self.args, self.merged, self.variables, self.token = fkt, args, {}, {}, None
    622
--> 623         fktast = get_fkt_ast(fkt)
    624
    625         if len(fktast.args.defaults) or fktast.args.vararg is not None or fktast.args.kwarg is not None:

/usr/local/lib/python3.4/dist-packages/hope-0.3.0-py3.4.egg/hope/_transformer.py in get_fkt_ast(fkt)
    708             outdent = line.find("def")
    709             break
--> 710     source = "\n".join([line[outdent:] for line in lines])
    711     modast = ast.parse(source)
    712

/usr/local/lib/python3.4/dist-packages/hope-0.3.0-py3.4.egg/hope/_transformer.py in <listcomp>(.0)
    708             outdent = line.find("def")
    709             break
--> 710     source = "\n".join([line[outdent:] for line in lines])
    711     modast = ast.parse(source)
    712

NameError: free variable 'outdent' referenced before assignment in enclosing scope

[cosmo-ethz]

@fermat618 thanks for checking! Regarding the error: HOPE does not support lambda expressions.

If you do something like this instead it should work:

In [1]: import hope

In [2]: import numpy as np

In [3]: @hope.jit
  ...: def add(x, y):
  ...:     return x + y
  ...: 

In [4]: add(4, 5)
Out[4]: 9

In [5]: a,b = np.arange(10), np.arange(10)

In [6]: add(a, b)
Out[6]: array([ 0,  2,  4,  6,  8, 10, 12, 14, 16, 18])

[fermat618]

In [1]: import hope
ho
In [2]: @hope.jit
   ...: def add(x, y)
  File "<ipython-input-2-3545c88303de>", line 2
    def add(x, y)
                 ^
SyntaxError: invalid syntax

In [3]: @hope.jit
def add(x, y):
   ...:     return x + y
   ...:

In [4]: add(2, 3)
gcc-4.8.real: error: unrecognized command line option ‘-fstack-protector-strong’
running build_ext
building 'add_08753fa4e11a762cd75a1868ab2ab62db34ada91d60c4a27abc9e9e8_0' extension
C compiler: x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC

compile options: '-I/usr/lib/python3/dist-packages/numpy/core/include -I/usr/include/python3.4m -c'
extra options: '-Wall -Wno-unused-variable -std=c++11'
x86_64-linux-gnu-gcc: /tmp/hope73secekp/add_08753fa4e11a762cd75a1868ab2ab62db34ada91d60c4a27abc9e9e8_0.cpp
gcc-4.8.real: error: unrecognized command line option ‘-fstack-protector-strong’
error: Command "x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I/usr/lib/python3/dist-packages/numpy/core/include -I/usr/include/python3.4m -c /tmp/hope73secekp/add_08753fa4e11a762cd75a1868ab2ab62db34ada91d60c4a27abc9e9e8_0.cpp -o /tmp/hope73secekp/add_08753fa4e11a762cd75a1868ab2ab62db34ada91d60c4a27abc9e9e8_0.o -Wall -Wno-unused-variable -std=c++11" failed with exit status 1479

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-4-c88cae139153> in <module>()
----> 1 add(2, 3)

/usr/local/lib/python3.4/dist-packages/hope-0.3.0-py3.4.egg/hope/_wrapper.py in _hope_callback(*args)
     69
     70         def _hope_callback(*args):
---> 71             return self(*args) if self.cache is None else self.cache(*args)
     72         self.callback = _hope_callback
     73         setattr(cache, str(id(self.callback)), fkt)

/usr/local/lib/python3.4/dist-packages/hope-0.3.0-py3.4.egg/hope/_wrapper.py in __call__(self, *args)
    102                 fp.write(code)
    103
--> 104             so_filename = _compile(tempfolder, localfilename, self.fkt.__name__)
    105
    106             self._store_state(tempfolder, localfilename)

/usr/local/lib/python3.4/dist-packages/hope-0.3.0-py3.4.egg/hope/_wrapper.py in _compile(target, localfilename, fkt_name)
    265     if not os.path.isfile(os.path.join(target, so_filename)) or out.find("error:") > -1:
    266         print(out)
--> 267         raise Exception("Error compiling function {0} (compiled to {1})".format(fkt_name, target))
    268
    269     #TODO: add test case

Exception: Error compiling function add (compiled to /tmp/hope73secekp)

[cosmo-ethz]

We don’t set the ‘-fstack-protector-strong’ argument for the compilation. From what I have seen while debugging, distutils.sysconfig reads out the Makefile that was used to compile Python and enhances the argument list with environment variables (OPT, CFLAGS). Could it be that this arg is in your env?