jiansoung commented 6 years ago

pyenv install 3.6.6 - zipimport.ZipImportError: can't decompress data; zlib not available

macOS + Homebrew + Pyenv

Issue

$ pyenv install 3.6.6
python-build: use openssl from homebrew
python-build: use readline from homebrew
Installing Python-3.6.6...
python-build: use readline from homebrew

BUILD FAILED (OS X 10.14 using python-build 20180424)

Inspect or clean up the working tree at /var/folders/9s/4dz5cwfj0dv6j_1_k8nlnd5m0000gn/T/python-build.20181028140503.9082
Results logged to /var/folders/9s/4dz5cwfj0dv6j_1_k8nlnd5m0000gn/T/python-build.20181028140503.9082.log

Last 10 log lines:
  File "/private/var/folders/9s/4dz5cwfj0dv6j_1_k8nlnd5m0000gn/T/python-build.20181028140503.9082/Python-3.6.6/Lib/ensurepip/__main__.py", line 5, in <module>
    sys.exit(ensurepip._main())
  File "/private/var/folders/9s/4dz5cwfj0dv6j_1_k8nlnd5m0000gn/T/python-build.20181028140503.9082/Python-3.6.6/Lib/ensurepip/__init__.py", line 204, in _main
    default_pip=args.default_pip,
  File "/private/var/folders/9s/4dz5cwfj0dv6j_1_k8nlnd5m0000gn/T/python-build.20181028140503.9082/Python-3.6.6/Lib/ensurepip/__init__.py", line 117, in _bootstrap
    return _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File "/private/var/folders/9s/4dz5cwfj0dv6j_1_k8nlnd5m0000gn/T/python-build.20181028140503.9082/Python-3.6.6/Lib/ensurepip/__init__.py", line 27, in _run_pip
    import pip._internal
zipimport.ZipImportError: can't decompress data; zlib not available
make: *** [install] Error 1

Fix

Install zlib

brew install zlib

Add the following to your `~/.zshrc` (if you use zsh)

the following content partly from the output of brew install zlib

DO NOT FORGET TO ADD ${LDFLAGS}, ${CPPFLAGS}, ${PKG_CONFIG_PATH} !

# For compilers to find zlib you may need to set:
export LDFLAGS="${LDFLAGS} -L/usr/local/opt/zlib/lib"
export CPPFLAGS="${CPPFLAGS} -I/usr/local/opt/zlib/include"

# For pkg-config to find zlib you may need to set:
export PKG_CONFIG_PATH="${PKG_CONFIG_PATH} /usr/local/opt/zlib/lib/pkgconfig"

tuanluu-agilityio commented 5 years ago

It saves my time. Thanks

Daletxt commented 5 years ago

Thank you! And other libs are also needed, such as SQLite3 and we also need to configure ~/.zshrc according to homebrew's information. And:!!! the .zshrc file such as:

For compilers to find zlib you may need to set:

export LDFLAGS="${LDFLAGS} -L/usr/local/opt/zlib/lib" export CPPFLAGS="${CPPFLAGS} -I/usr/local/opt/zlib/include" export LDFLAGS="${LDFLAGS} -L/usr/local/opt/sqlite/lib" export CPPFLAGS="${CPPFLAGS} -I/usr/local/opt/sqlite/include"

For pkg-config to find zlib you may need to set:

export PKG_CONFIG_PATH="${PKG_CONFIG_PATH} /usr/local/opt/zlib/lib/pkgconfig" export PKG_CONFIG_PATH="${PKG_CONFIG_PATH} /usr/local/opt/sqlite/lib/pkgconfig" are OK. But

For compilers to find zlib you may need to set:

export LDFLAGS="${LDFLAGS} -L/usr/local/opt/zlib/lib" export CPPFLAGS="${CPPFLAGS} -I/usr/local/opt/zlib/include"

For pkg-config to find zlib you may need to set:

export PKG_CONFIG_PATH="${PKG_CONFIG_PATH} /usr/local/opt/zlib/lib/pkgconfig"

export LDFLAGS="-L/usr/local/opt/sqlite/lib" export CPPFLAGS="-I/usr/local/opt/sqlite/include" export PKG_CONFIG_PATH="/usr/local/opt/sqlite/lib/pkgconfig" are not OK! I think maybe we need "${LDFLAGS} 、${CPPFLAGS} 、${PKG_CONFIG_PATH} ". And I need to source ~/.zshrc every time I used pyenv install xxx.

antonagestam commented 5 years ago

You don't need to add anything permanently to your dotfiles, the flags just need to be set during compile time. So just set the environment variables in your shell session. This worked for me:

$ brew install zlib
$ brew install sqlite
$ export LDFLAGS="${LDFLAGS} -L/usr/local/opt/zlib/lib"
$ export CPPFLAGS="${CPPFLAGS} -I/usr/local/opt/zlib/include"
$ export LDFLAGS="${LDFLAGS} -L/usr/local/opt/sqlite/lib"
$ export CPPFLAGS="${CPPFLAGS} -I/usr/local/opt/sqlite/include"
$ export PKG_CONFIG_PATH="${PKG_CONFIG_PATH} /usr/local/opt/zlib/lib/pkgconfig"
$ export PKG_CONFIG_PATH="${PKG_CONFIG_PATH} /usr/local/opt/sqlite/lib/pkgconfig"
$ pyenv install 3.6.8

jocampo commented 5 years ago

Thanks @antonagestam , worked like a charm!

slhck commented 5 years ago

Thanks @antonagestam for the instructions (I just wish you removed the $ so this could be copied straight into a terminal).

I can't believe this is the only resource on the web for this problem. I haven't had this issue so far, using Homebrew and pyenv. What caused this change?

antonagestam commented 5 years ago

@slhck It is not the only resource for this on the web, it is a reported issue on the pyenv project: https://github.com/pyenv/pyenv/issues/1219

The dollar signs are in there to prevent people from copy-pasting code without knowing what they're doing.

slhck commented 5 years ago

Thanks for the link. I was a little imprecise in my statement, but this was the most prominent search result I got for this particular error, with a simple fix presented. The others being https://github.com/pyenv/pyenv/issues/454 (which suggests xcode-select, which I did), and this Q&A post, which is about Unix. And then you get to some other Stack Overflow posts that finally, after following another redirect, point you to the Pyenv issue.

janjur commented 5 years ago

cat <<EOT >> ~/.zshrc
# For compilers to find zlib you may need to set:
export LDFLAGS="${LDFLAGS} -L/usr/local/opt/zlib/lib"
export CPPFLAGS="${CPPFLAGS} -I/usr/local/opt/zlib/include"

# For pkg-config to find zlib you may need to set:
export PKG_CONFIG_PATH="${PKG_CONFIG_PATH} /usr/local/opt/zlib/lib/pkgconfig"
EOT

Edit: above is permanent, thus (as @antonagestam said) wrong. His answer for the lazy:

brew install zlib
brew install sqlite
export LDFLAGS="${LDFLAGS} -L/usr/local/opt/zlib/lib"
export CPPFLAGS="${CPPFLAGS} -I/usr/local/opt/zlib/include"
export LDFLAGS="${LDFLAGS} -L/usr/local/opt/sqlite/lib"
export CPPFLAGS="${CPPFLAGS} -I/usr/local/opt/sqlite/include"
export PKG_CONFIG_PATH="${PKG_CONFIG_PATH} /usr/local/opt/zlib/lib/pkgconfig"
export PKG_CONFIG_PATH="${PKG_CONFIG_PATH} /usr/local/opt/sqlite/lib/pkgconfig"
pyenv install 3.6.8

frankie-ug commented 5 years ago

This saved me a lot of time!

empeje commented 5 years ago

For someone using fish shell can check this out https://gist.github.com/empeje/fd7b654b2e57e33f74b833fedcd5f51e

Rabia23 commented 5 years ago

You don't need to add anything permanently to your dotfiles, the flags just need to be set during compile time. So just set the environment variables in your shell session. This worked for me:

$ brew install zlib
$ brew install sqlite
$ export LDFLAGS="${LDFLAGS} -L/usr/local/opt/zlib/lib"
$ export CPPFLAGS="${CPPFLAGS} -I/usr/local/opt/zlib/include"
$ export LDFLAGS="${LDFLAGS} -L/usr/local/opt/sqlite/lib"
$ export CPPFLAGS="${CPPFLAGS} -I/usr/local/opt/sqlite/include"
$ export PKG_CONFIG_PATH="${PKG_CONFIG_PATH} /usr/local/opt/zlib/lib/pkgconfig"
$ export PKG_CONFIG_PATH="${PKG_CONFIG_PATH} /usr/local/opt/sqlite/lib/pkgconfig"
$ pyenv install 3.6.8

Thanks, @antonagestam for instructions. It really helped a lot.

dews commented 5 years ago

For fish shell

brew install zlib
brew install sqlite
set -g -x LDFLAGS "$LDFLAGS  -L/usr/local/opt/zlib/lib"
set -g -x CPPFLAGS "$CPPFLAGS  -I/usr/local/opt/zlib/include"
set -g -x LDFLAGS "$LDFLAGS  -L/usr/local/opt/sqlite/lib"
set -g -x CPPFLAGS "$CPPFLAGS  -I/usr/local/opt/sqlite/include"
set -g -x PKG_CONFIG_PATH "$PKG_CONFIG_PATH  /usr/local/opt/zlib/lib/pkgconfig"
set -g -x PKG_CONFIG_PATH "$PKG_CONFIG_PATH  /usr/local/opt/sqlite/lib/pkgconfig"
pyenv install 3.6.8

kylefoley76 commented 5 years ago

Ok, I loved the problem on a Mac using Mojave OS. I first removed pyenv

rm -rf "$HOME/.pyenv"

Then installed it using homebrew

brew install pyenv

That did it. The website where I got the info was here: https://github.com/pyenv/pyenv/issues/1281

RulerOf commented 5 years ago

Thanks @antonagestam .

To cover future installs, I used an alias to turn the concept you're using that sets those values every time I invoke pyenv by adding this to my ~/.bash_profile:

alias pyenv='LDFLAGS="${LDFLAGS} -L/usr/local/opt/zlib/lib -L/usr/local/opt/sqlite/lib" CPPFLAGS="${CPPFLAGS} -I/usr/local/opt/zlib/include -I/usr/local/opt/sqlite/include" PKG_CONFIG_PATH="${PKG_CONFIG_PATH} /usr/local/opt/zlib/lib/pkgconfig /usr/local/opt/sqlite/lib/pkgconfig" pyenv'

ghost commented 5 years ago

@RulerOf

Thanks @antonagestam .

To cover future installs, I turned used an alias to turn the concept you're using that sets those values every time I invoke pyenv by adding this to my ~/.bash_profile:
alias pyenv='LDFLAGS="${LDFLAGS} -L/usr/local/opt/zlib/lib -L/usr/local/opt/sqlite/lib" CPPFLAGS="${CPPFLAGS} -I/usr/local/opt/zlib/include -I/usr/local/opt/sqlite/include" PKG_CONFIG_PATH="${PKG_CONFIG_PATH} /usr/local/opt/zlib/lib/pkgconfig /usr/local/opt/sqlite/lib/pkgconfig" pyenv'

The alias worked very well for me!!! 👌🏻

idcrook commented 5 years ago

ran into similar problem on Linux Mint (Ubuntu) found this issue during searching. did the following:

sudo apt install zlib1g-dev

did more googlings/search pyenv issues, then found this page that might cover additional build issues (assorted Linux and macOS listed), and it lists many other potential build dependencies

https://github.com/pyenv/pyenv/wiki/Common-build-problems

Sjors commented 5 years ago

The pyenv alias breaks pyenv init in .bash_profile for me.

Another approach is to use /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk as explained here

The Apple notes suggest that this is a bug in PyEnv:

The command line tools will search the SDK for system headers by default. However, some software may fail to build correctly against the SDK and require macOS headers to be installed in the base system under /usr/include. If you are the maintainer of such software, we encourage you to update your project to work with the SDK or file a bug report for issues that are preventing you from doing so

ishandutta2007 commented 5 years ago

sudo apt install zlib1g-dev

For me it didn't work, what worked is

sudo yum/apt-get install zlib-devel

zoedaemon commented 5 years ago

ran into similar problem on Linux Mint (Ubuntu) found this issue during searching. did the following:
sudo apt install zlib1g-dev
did more googlings/search pyenv issues, then found this page that might cover additional build issues (assorted Linux and macOS listed), and it lists many other potential build dependencies

https://github.com/pyenv/pyenv/wiki/Common-build-problems

thanks it's worked on debian 9 GCP

eliethesaiyan commented 5 years ago

ran into similar problem on Linux Mint (Ubuntu) found this issue during searching. did the following:
sudo apt install zlib1g-dev
did more googlings/search pyenv issues, then found this page that might cover additional build issues (assorted Linux and macOS listed), and it lists many other potential build dependencies

https://github.com/pyenv/pyenv/wiki/Common-build-problems

I can confirm that this fixed my problems

coolaj86 commented 3 years ago

Looks like this is safe to close out, no?

Also, my solution for Linux: https://stackoverflow.com/questions/56999217/cant-fix-zipimport-zipimporterror-cant-decompress-data-zlib-not-available/65925481#65925481

If you don't have zlib1g-dev you're probably also missing libssl-dev too.

ktavabi commented 2 years ago

I am still hitting a wall with installing versions of python < 3.10.0.

pyenv versions
  system
* 3.10.0 (set by /Users/me/.pyenv/version)
  3.8-dev-debug

I've managed to narrow it down to setting LD_LIBRARY_PATH during the build process...

LD_LIBRARY_PATH=: clang -DZLIB -DZLIB_SHARED -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DVPAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DOPENSSLDIR="/Users/me/.pyenv/versions/3.6.8/openssl/ssl" -DENGINESDIR="/Users/me/.pyenv/versions/3.6.8/openssl/lib/engines-1.1" -O3 -D_REENTRANT -arch i386 -DL_ENDIAN -fomit-frame-pointer -fPIC -arch i386 -dynamiclib -current_version 1.1 -compatibility_version 1.1 -install_name /Users/me/.pyenv/versions/3.6.8/openssl/lib/libcrypto.1.1.dylib -o libcrypto.1.1.dylib -all_load libcrypto.a -Wl,-search_paths_first

This results in the following, and its head against the wall for me. So any help is appreciated.

ld: warning: The i386 architecture is deprecated for macOS (remove from the Xcode build setting: ARCHS)

I've installed my dependencies via mac ports and neither lazy or permanent shell variable settings worked out for me. I suspect there is a solution here but I can't make it out.

jiansoung / issues-list