sarojrout commented 8 years ago

Error: must have python development packages for 2.6 or 2.7. Could not find Python.h. Please install python2.6-devel or python2.7-devel". Stop.

sarojrout commented 8 years ago

The python version in my mac is 2.7.10

sarojrout commented 8 years ago

The issue got resolved after upgrading to El Capitan osx and running the below command. export LDFLAGS=-L/usr/local/opt/openssl/lib && export CPPFLAGS=-I/usr/local/opt/openssl/include The reason was: /usr/include was not getting created and python2.6/2.7 was not getting installed inside /include

scheung38 commented 8 years ago

Also in El Capitan:

In my home directory:

$ brew install hadoop
$ brew install openssl
$ export LDFLAGS=-L/usr/local/opt/openssl/lib && export CPPFLAGS=-I/usr/local/opt/openssl/include
at cd into hue directory

and cd into hue:

$ make apps
/Users/seb/hue/Makefile.vars:42: *** “Error: must have python development packages for 2.6 or 2.7. Could not find Python.h. Please install python2.6-devel or python2.7-devel”. Stop.

Why is that thanks

romainr commented 8 years ago

Do you have a python like /usr/include/python2.7/Python.h or /usr/include/python2.6/Python.h installed?

spaztic1215 commented 8 years ago

Sometimes the Python dev packages are instead installed under /usr/local/Cellar/python/ and typically you can just symlink the /usr/include/python2.7 link to where it's actually installed. But I think that El Capitan has actually disabled the entire /usr/include tree so you might want to try just skipping the check by setting export SKIP_PYTHONDEV_CHECK=1 before you run make apps.

scheung38 commented 8 years ago

My default Python is installed via anaconda:

$ which python /Users/seb/anaconda/bin/python

$ python Python 2.7.10 |Anaconda 2.2.0 (x86_64)| (default, May 28 2015, 17:04:42) [GCC 4.2.1 (Apple Inc. build 5577)] on darwin Type "help", "copyright", "credits" or "license" for more information. Anaconda is brought to you by Continuum Analytics. Please check out: http://continuum.io/thanks and https://binstar.org

Hi spaztic1215 how to symlink, from which source file to which target file?

ok after

$ ssh localhost

then added the line

export SKIP_PYTHONDEV_CHECK=1

and then cd into hue directory and

$ make apps

and now it seems to build, but

building 'Crypto.PublicKey._fastmath' extension
gcc -fno-strict-aliasing -I/Users/seb/anaconda/include -arch x86_64 -fwrapv -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/ -I/Users/seb/anaconda/include/python2.7 -c src/_fastmath.c -o build/temp.macosx-10.5-x86_64-2.7/src/_fastmath.o
src/_fastmath.c:36:11: fatal error: 'gmp.h' file not found
# include <gmp.h>
          ^
1 error generated.
error: command 'gcc' failed with exit status 1
make[2]: *** [/Users/seb/hue/desktop/core/build/pycrypto-2.6.1/egg.stamp] Error 1
make[1]: *** [.recursive-env-install/core] Error 2
make: *** [desktop] Error 2

even though I already had

brew install gmp

???

The confusion for me is do you install brew install hadoop, etc in Mac OSX or after ssh localhost?

spaztic1215 commented 8 years ago

All of the brew commands, as well as the make apps command, should be done within the terminal in Mac OSX. Is there a particular reason why you aren't using the Python 2.7 that is shipped with El Capitan? Assuming that gmp installed correctly, then the extra step export the openssl flags:

export LDFLAGS=-L/usr/local/opt/openssl/lib && export CPPFLAGS=-I/usr/local/opt/openssl/include

should allow the build to find those missing extensions. Does that path /usr/local/opt/openssl exist on your machine?

scheung38 commented 8 years ago

Yes already did these two commands:

$ export LDFLAGS=-L/usr/local/opt/openssl/lib
$ export CPPFLAGS=-I/usr/local/opt/openssl/include

But still the same gmp not found, is it because I am using this anaconda python distribution? Since I noticed this src/_fastmath.c line 36 gmp.h not found:

building 'Crypto.PublicKey._fastmath' extension gcc -fno-strict-aliasing -I/Users/seb/anaconda/include -arch x86_64 -fwrapv -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/ -I/Users/seb/anaconda/include/python2.7 -c src/_fastmath.c -o build/temp.macosx-10.5-x86_64-2.7/src/_fastmath.o src/_fastmath.c:36:11: fatal error: 'gmp.h' file not found -> include gmp.h

If thats the case not sure how to switch anaconda python back to default Apple Python, if that's the cause?

Yes path

/usr/local/opt/openssl/

exists

enricoberti commented 8 years ago

You did brew install gmp right?

scheung38 commented 8 years ago

@enricoberti :

$ brew install gmp
Warning: gmp-6.1.0 already installed

so looks correct to me

or do I need to define?

export HADOOP_PREFIX={your hadoop distribution directory}

seems like HADOOP_PREFIX is redundant?

Detry322 commented 8 years ago

was this solved? having this issue right now :/

PedroMDuarte commented 8 years ago

I was having a similar issue when running make apps. The first time I run it, my PATH environment variable was set up so anaconda was being used. I went ahead and changed my PATH so that it used the system's python located at /usr/bin/python. That was not enough to solve the problem, as it seems that the hue path was polluted with some content related to the original failure. I tried running make clean, but that did not work either. I deleted the repo and cloned it again and that solved the problem!

After cloning the repo, here are the commands I used to get make apps to work:

[09:20]  @im-MBPR13-18 : my-sandbox$ which brew 
/usr/local/bin/brew
[09:20]  @im-MBPR13-18 : my-sandbox$ echo $PATH
/Users/pedroduarte/anaconda/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
[09:20]  @im-MBPR13-18 : my-sandbox$ export PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
[09:21]  @im-MBPR13-18 : my-sandbox$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
[09:21]  @im-MBPR13-18 : my-sandbox$ which brew
/usr/local/bin/brew
[09:21]  @im-MBPR13-18 : my-sandbox$ brew install maven
Warning: maven-3.3.9 already installed
[09:21]  @im-MBPR13-18 : my-sandbox$ brew install mysql
Warning: mysql-5.7.12 already installed
[09:21]  @im-MBPR13-18 : my-sandbox$ brew install gmp
Warning: gmp-6.1.0 already installed
[09:21]  @im-MBPR13-18 : my-sandbox$ export LDFLAGS=-L/usr/local/opt/openssl/lib && export CPPFLAGS=-I/usr/local/opt/openssl/include
[09:21]  @im-MBPR13-18 : my-sandbox$ echo $LDFLAGS
-L/usr/local/opt/openssl/lib
[09:21]  @im-MBPR13-18 : my-sandbox$ echo $CPPFLAGS
-I/usr/local/opt/openssl/include
[09:21]  @im-MBPR13-18 : my-sandbox$ which python
/usr/bin/python
[09:21]  @im-MBPR13-18 : my-sandbox$ cd hue
[09:22]  @im-MBPR13-18 : hue$ make apps
--- Creating virtual environment at /Users/pedroduarte/my-sandbox/hue/build/env
python2.7 /Users/pedroduarte/my-sandbox/hue/tools/virtual-bootstrap/virtual-bootstrap.py \
        -qq --system-site-packages /Users/pedroduarte/my-sandbox/hue/build/env
--- /Users/pedroduarte/my-sandbox/hue/build/env ready
cd /Users/pedroduarte/my-sandbox/hue/maven && mvn install 
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Hue Maven Parent POM 3.10.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------

riching commented 8 years ago

ln -s /usr/local/opt/openssl/include/openssl /usr/local/include/openssl

romainr commented 8 years ago

http://gethue.com/start-developing-hue-on-a-mac-in-a-few-minutes/

darshanmodh commented 7 years ago

if you installed/updated your xcode recently, use following command to create necessary files in /usr

xcode-select --install

richtrf commented 7 years ago

Is this solved? I'm still seeing the following error by doing "make apps"

I already do brew install gmp but still see similar error as @scheung38 's one

src/_fastmath.c:36:11: fatal error: 'gmp.h' file not found
# include <gmp.h>
          ^
324 warnings and 1 error generated.
error: command 'clang' failed with exit status 1
make[2]: *** [/Users/ychang/Documents/Projects-Python/hue/desktop/core/build/pycrypto-2.6.1/egg.stamp] Error 1
make[1]: *** [.recursive-env-install/core] Error 2
make: *** [desktop] Error 2

romainr commented 7 years ago

You are missing the gmp package, etc https://github.com/cloudera/hue#development-prerequisites

richtrf commented 7 years ago

@romainr I did use brew install gmp to have the package. Any other suggestion to install it? Thanks!

My gmp version: gmp-6.1.2 already installed

sayan1886 commented 7 years ago

steps to fix

symlink gmp.h to /usr/inlude/
add LDFLGS="-L/usr/local/opt/openssl/lib -L/path/to/gmp/lib"

Hope this will work.

jonathanchc commented 7 years ago

I was experimenting the same issue on MAC Sierra

You Should try with this commands.

gem install ffi

make clean

make app

Good luck!!

FrommyMind commented 5 years ago

building '_cffi_backend' extension /usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -I/usr/include/ffi -I/usr/include/libffi -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c c/_cffi_backend.c -o build/temp.macosx-10.6-intel-2.7/c/_cffi_backend.o c/_cffi_backend.c:15:10: fatal error: 'ffi.h' file not found

include

Then I do https://stackoverflow.com/questions/22875270/error-installing-bcrypt-with-pip-on-os-x-cant-find-ffi-h-libffi-is-installed/25854749#25854749 brew install pkg-config libffi

make apps got below error

No local packages or download links found for cffi>=1.4.1

Then I copy the folder cffi-1.11.5.dist-info to PATH/hue/build/env/lib/python2.7/site-packages/

arturogonzalezm commented 5 years ago

In file Makefile.vars replace line 39:

PYTHON_H ?= $(shell ls /usr/include/python2.7/Python.h 2>/dev/null || ls /usr/local/python27/include/python2.7/Python.h 2>/dev/null || ls /opt/rh/python27/root/usr/include/python2.7/Python.h 2>/dev/null)

to where your Python.h is

romainr commented 5 years ago

FYI @wing2fly, maybe the PYTHON_H could be provided by the user as default value

enricoberti commented 5 years ago

It can be that in MacOS Mojave, especially after upgrading, you are missing the MacOS SDK headers (including Python's). This helps:

$ open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

codeStryke commented 4 years ago

I'm running into the same problem with OSX 10.15.3 (Catalina). Any suggestions?


"PYTHON_VER is python2.7"
/hue/Makefile.vars:62: *** "Error: must have python development packages for python2.7. Could not find Python.h. Please install python2.7-devel".  Stop.```

romainr commented 4 years ago

https://docs.gethue.com/administrator/installation/dependencies/#macos

codeStryke commented 4 years ago

I followed the steps and need to make a change to Makefile.vars (I can submit a PR if I get everything working first) but I am now stuck with another error

"PYTHON_VER is python2.7"
cd /Users/foo/development/learning/hue/maven && mvn install
[INFO] Scanning for projects...
[INFO]
[INFO] --------------------< com.cloudera.hue:hue-parent >---------------------
[INFO] Building Hue Maven Parent POM 4.6.0-SNAPSHOT
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:1.0:enforce (default) @ hue-parent ---
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ hue-parent ---
[INFO] Installing /Users/foo/development/learning/hue/maven/pom.xml to /Users/foo/.m2/repository/com/cloudera/hue/hue-parent/4.6.0-SNAPSHOT/hue-parent-4.6.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.612 s
[INFO] Finished at: 2020-02-20T12:57:49-08:00
[INFO] ------------------------------------------------------------------------
"PYTHON_VER is python2.7"
"----------------> ENV_INSTALL_TARGETS: .recursive-env-install/core .recursive-env-install/libs/aws .recursive-env-install/libs/azure .recursive-env-install/libs/hadoop .recursive-env-install/libs/indexer .recursive-env-install/libs/libanalyze .recursive-env-install/libs/liboauth .recursive-env-install/libs/liboozie .recursive-env-install/libs/librdbms .recursive-env-install/libs/libsaml .recursive-env-install/libs/libsentry .recursive-env-install/libs/libsolr .recursive-env-install/libs/libzookeeper .recursive-env-install/libs/metadata .recursive-env-install/libs/notebook .recursive-env-install/libs/dashboard .recursive-env-install/libs/kafka"
/Library/Developer/CommandLineTools/usr/bin/make -C core env-install
"PYTHON_VER is python2.7"
--- Building egg for MySQL-python-1.2.5
running bdist_egg
running egg_info
writing MySQL_python.egg-info/PKG-INFO
writing top-level names to MySQL_python.egg-info/top_level.txt
writing dependency_links to MySQL_python.egg-info/dependency_links.txt
reading manifest file 'MySQL_python.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'MySQL_python.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.15-x86_64/egg
running install_lib
running build_py
copying MySQLdb/release.py -> build/lib.macosx-10.15-x86_64-2.7/MySQLdb
running build_ext
building '_mysql' extension
cc -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -iwithsysroot /usr/local/libressl/include -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -I/usr/local/opt/openssl/include -arch x86_64 -pipe -Dversion_info=(1,2,5,'final',1) -D__version__=1.2.5 -I/usr/local/Cellar/mysql/8.0.19/include/mysql -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _mysql.c -o build/temp.macosx-10.15-x86_64-2.7/_mysql.o
_mysql.c:44:10: fatal error: 'my_config.h' file not found
#include "my_config.h"
         ^~~~~~~~~~~~~
1 error generated.
error: command 'cc' failed with exit status 1
make[2]: *** [/Users/foo/development/learning/hue/desktop/core/build/MySQL-python-1.2.5/egg.stamp] Error 1
make[1]: *** [.recursive-env-install/core] Error 2
make: *** [desktop] Error 2

Also thank you for your quick response. Very helpful.

romainr commented 4 years ago

It looks like it is a similar problem, do you have the mysql dev package installed and on the build path?

codeStryke commented 4 years ago

I think the problem is brew installs the latest version of mysql which no longer as the my_config.h file. As per the release notes https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-2.html, it says Client programs should only need to #include the <mysql.h> header file. In particular, <my_config.h> should not be needed, and is no longer installed

https://dev.mysql.com/doc/dev/mysql-server/8.0.11/my__config_8h.html exists but https://dev.mysql.com/doc/dev/mysql-server/8.0.19/dir_d44c64559bbebec7f509842c48db8b23.html does not

I downloaded my_config.h from https://dev.mysql.com/doc/dev/mysql-server/8.0.11/my__config_8h.html, placed the file in /usr/local/include, and the installation was able to continue. But stopped at missing sasl.

For that, running export CFLAGS="-I$(xcrun --show-sdk-path)/usr/include/sasl” and then running make apps fixed the issue.

I then ran into npm issues. I did an npm update and got past that.

Finally I'm stuck at

=== Installing app at zookeeper
Updating registry with zookeeper v.4.6.0
Running 'make -C /Users/foo/development/learning/hue/apps/zookeeper egg-info ext-eggs' with {'ROOT': '/Users/foo/development/learning/hue'}
"PYTHON_VER is python2.7"
make[2]: Nothing to be done for `egg-info'.
make[2]: Nothing to be done for `ext-eggs'.
=== Saved registry at /Users/foo/development/learning/hue/app.reg
=== Saved /Users/foo/development/learning/hue/build/env/lib/python2.7/site-packages/hue.pth
Running '/Users/foo/development/learning/hue/build/env/bin/hue makemigrations --noinput' with None
Running '/Users/foo/development/learning/hue/build/env/bin/hue migrate --fake-initial' with None
Running '/Users/foo/development/learning/hue/build/env/bin/hue collectstatic --noinput' with None
"PYTHON_VER is python2.7"
./build/env/bin/hue collectstatic --noinput
make[1]: *** [create-static] Abort trap: 6
make: *** [apps] Error 2

romainr commented 4 years ago

Good point, we should indeed see for replacing it by https://pypi.org/project/mysql-connector-python/ or just removing it.

https://stackoverflow.com/questions/25865270/how-to-install-python-mysqldb-module-using-pip

It is currently bundled here https://github.com/cloudera/hue/tree/master/desktop/core/ext-py/MySQL-python-1.2.5

codeStryke commented 4 years ago

Any idea if it's possible to get past the error I'm seeing? I'm not at all familiar with python development, I might be missing something obvious.

romainr commented 4 years ago

Are you planning to use MySql or another DB?

If not planning to use MySql, how about deleting MySQL-python directory?

codeStryke commented 4 years ago

I'm primarily interested in following along this https://gethue.com/build-your-own-autocompleter/

After deleting MySQL-python i still face this error

/Users/foo/development/learning/hue/build/env/bin/python2.7 /Users/foo/development/learning/hue/tools/app_reg/app_reg.py --install about beeswax filebrowser hbase help hive impala jobbrowser jobsub metastore oozie pig proxy rdbms search security spark sqoop useradmin zookeeper --relative-paths
=== Installing app at about
=== about v.4.6.0 is already installed
=== Installing app at beeswax
=== beeswax v.4.6.0 is already installed
=== Installing app at filebrowser
=== filebrowser v.4.6.0 is already installed
=== Installing app at hbase
=== hbase v.4.6.0 is already installed
=== Installing app at help
=== help v.4.6.0 is already installed
=== Installing app at hive
=== hive v.4.6.0 is already installed
=== Installing app at impala
=== impala v.4.6.0 is already installed
=== Installing app at jobbrowser
=== jobbrowser v.4.6.0 is already installed
=== Installing app at jobsub
=== jobsub v.4.6.0 is already installed
=== Installing app at metastore
=== metastore v.4.6.0 is already installed
=== Installing app at oozie
=== oozie v.4.6.0 is already installed
=== Installing app at pig
=== pig v.4.6.0 is already installed
=== Installing app at proxy
=== proxy v.4.6.0 is already installed
=== Installing app at rdbms
=== rdbms v.4.6.0 is already installed
=== Installing app at search
=== search v.4.6.0 is already installed
=== Installing app at security
=== security v.4.6.0 is already installed
=== Installing app at spark
=== spark v.4.6.0 is already installed
=== Installing app at sqoop
=== sqoop v.4.6.0 is already installed
=== Installing app at useradmin
=== useradmin v.4.6.0 is already installed
=== Installing app at zookeeper
=== zookeeper v.4.6.0 is already installed
=== Saved registry at /Users/foo/development/learning/hue/app.reg
=== Saved /Users/foo/development/learning/hue/build/env/lib/python2.7/site-packages/hue.pth
Running '/Users/foo/development/learning/hue/build/env/bin/hue makemigrations --noinput' with None
Running '/Users/foo/development/learning/hue/build/env/bin/hue migrate --fake-initial' with None
Running '/Users/foo/development/learning/hue/build/env/bin/hue collectstatic --noinput' with None
"PYTHON_VER is python2.7"
./build/env/bin/hue collectstatic --noinput
make[1]: *** [create-static] Abort trap: 6
make: *** [apps] Error 2

romainr commented 4 years ago

If you run only

./build/env/bin/hue collectstatic --noinput

Does it give more info?

romainr commented 4 years ago

Also good to do a 'make clean', then 'make apps' after deleting the MySQL-python lib.

codeStryke commented 4 years ago

make clean and then make apps didn't help :( also running only $ ./build/env/bin/hue collectstatic --noinput return Abort trap: 6

[:~/development/learning/hue] master(+3459/-12722) 2 ± ./build/env/bin/hue collectstatic --noinput
Abort trap: 6

Magic-Jay commented 4 years ago

Is this a bug or something? And what is exactly causing this? Because it's already 2020, more than 4 years after this issue has been opened but I see people are still struggling with this issue, from mac El Captain to mac Catalina. Is there a universal fix to this or not?

romainr commented 4 years ago

If you read carefully you will see that the errors are different each time (main one is about 'gmp') and is fixable.

Magic-Jay commented 4 years ago

If you read carefully you will see that the errors are different each time (main one is about 'gmp') and is fixable.

not really.... what I see is people are attempting and recommending random solutions while not knowing exactly what the cause of is. The only reasonable explanation was back in 2016 stating that sometimes "python dev packages are instead installed under /usr/local/Cellar/python/ ... ..."

And bluntly flowing all these instructions has not fixed this issue for me yet.

romainr commented 4 years ago

So please open a new issue linked to these one with the specific problem explained (it is easier to manage one detailed problem at the time)

cloudera / hue

Facing issue while executing make apps in my Mac OSX 10.10.2 version #251

include