pbrady / fastcache

C implementation of Python 3 lru_cache
MIT License
154 stars 19 forks source link

Problems installing on windows 8.1 x64 - Python 2.7.8 #32

Closed canesin closed 9 years ago

canesin commented 9 years ago

I have tried all pip, conda and git options but looks like some header or library is missing:

_lrucache.obj : error LNK2019: unresolved external symbol _PyGC_generation0 referenced in function make_key
pbrady commented 9 years ago

@canesin I don't have access to a windows machine to test this out. Can you post the steps/results for trying to install via conda (starting with a clean build)?

canesin commented 9 years ago

@pbrady C:\Users\Fabio\Anaconda>conda-build conda-recipes/fastcache BUILD START: fastcache-1.0.1-py27_0 Fetching package metadata: ... Solving package specifications: . The following NEW packages will be INSTALLED:

python:     2.7.8-0
setuptools: 7.0-py27_0

Linking packages ... [ COMPLETE ] |#################################################| 100% Removing old work directory Source cache directory is: C:\Users\Fabio\Anaconda\conda-bld\src_cache Found source in cache: fastcache-1.0.1.tar.gz Extracting download Package: fastcache-1.0.1-py27_0 source tree in: C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1 Warning: Couldn't find Visual Studio: 'C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcvarsall.bat'

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set TMP=C:\Users\Fabio\AppData\Local\Temp

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set COMPUTERNAME=ESCRITORIO

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set SP_DIR=C:\Users\Fabio\Anaconda\envs_build\Lib\site-packages

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set VS110COMNTOOLS=C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common 7\Tools\

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set INTEL_LICENSE_FILE=C:\Program Files (x86)\Common Files\Intel\Licenses

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PYTHON=C:\Users\Fabio\Anaconda\envs_build\python.exe

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PERL_VER=5.18.2

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PY3K=0

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set INTEL_DEV_REDIST=C:\Program Files (x86)\Common Files\Intel\Shared Librari es\

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PSMODULEPATH=C:\Windows\system32\WindowsPowerShell\v1.0\Modules\;C:\Progr am Files\Intel\

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set COMMONPROGRAMFILES=C:\Program Files\Common Files

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 69 Stepping 1, GenuineIntel

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set CPATH=C:\Users\Fabio\Anaconda\include;C:\Users\Fabio\Anaconda\libs

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PROGRAMFILES=C:\Program Files

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PROCESSOR_REVISION=4501

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set SYSTEMROOT=C:\Windows

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PY_VER=2.7

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PATH=C:\Users\Fabio\Anaconda\envs_build\Library\bin;C:\Users\Fabio\Anaco nda\envs_build;C:\Users\Fabio\Anaconda\envs_build\Scripts;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel6 4\mpirt;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\ia32\mpirt;C:\Program Files (x86)\Common Files\Intel\Share d Libraries\redist\intel64\compiler;C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\ia32\compiler;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\Sys tem32\WindowsPowerShell\v1.0\;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Manag ement Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Inte l(R) Management Engine Components\IPT;C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\;C:\Program Files\Microsof t SQL Server\110\Tools\Binn\;C:\Program Files (x86)\Microsoft SDKs\TypeScript\1.0\;C:\Program Files (x86)\Git\cmd;C:\TDM-GCC-64\bin; C:\Program Files (x86)\CMake\bin;C:\Users\Fabio\Anaconda;C:\Users\Fabio\Anaconda\Scripts;

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set ARCH=64

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PROGRAMFILES(X86)=C:\Program Files (x86)

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PKG_BUILDNUM=0

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set ADVISOR_XE_2015_DIR=C:\Program Files (x86)\Intel\Advisor XE 2015\

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set COMMONPROGRAMFILES(X86)=C:\Program Files (x86)\Common Files

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set USERDOMAIN=ESCRITORIO

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PROCESSOR_ARCHITECTURE=AMD64

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set STDLIB_DIR=C:\Users\Fabio\Anaconda\envs_build\Lib

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set MIC_LD_LIBRARY_PATH=C:\Program Files (x86)\Common Files\Intel\Shared Libr aries\compiler\lib\mic

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set ALLUSERSPROFILE=C:\ProgramData

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set LIBRARY_LIB=C:\Users\Fabio\Anaconda\envs_build\Library\lib

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set LOCALAPPDATA=C:\Users\Fabio\AppData\Local

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set HOMEPATH=\Users\Fabio

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set USERDOMAIN_ROAMINGPROFILE=ESCRITORIO

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set VS120COMNTOOLS=C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common 7\Tools\

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set SYS_PYTHON=C:\Users\Fabio\Anaconda\python.exe

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PROGRAMW6432=C:\Program Files

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set USERNAME=Fabio

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set LOGONSERVER=\MicrosoftAccount

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PROMPT=$P$G

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set COMSPEC=C:\Windows\system32\cmd.exe

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PROGRAMDATA=C:\ProgramData

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set LIBRARY_BIN=C:\Users\Fabio\Anaconda\envs_build\Library\bin

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set CPU_COUNT=4

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set RECIPE_DIR=C:\Users\Fabio\Anaconda\conda-recipes\fastcache

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set VTUNE_AMPLIFIER_XE_2015_DIR=C:\Program Files (x86)\Intel\VTune Amplifier XE 2015\

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PKG_VERSION=1.0.1

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set NPY_VER=1.9

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PKG_NAME=fastcache

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set ICPP_COMPILER15=C:\Program Files (x86)\Intel\Composer XE 2015\

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set SESSIONNAME=Console

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set INSPECTOR_XE_2015_DIR=C:\Program Files (x86)\Intel\Inspector XE 2015\

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set FP_NO_HOST_CHECK=NO

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set WINDIR=C:\Windows

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set TEMP=C:\Users\Fabio\AppData\Local\Temp

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set HOMEDRIVE=C:

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set CONDA_BUILD=1

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set SYSTEMDRIVE=C:

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set LIBRARY_INC=C:\Users\Fabio\Anaconda\envs_build\Library\include

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PREFIX=C:\Users\Fabio\Anaconda\envs_build

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set NUMBER_OF_PROCESSORS=4

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set APPDATA=C:\Users\Fabio\AppData\Roaming

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PROCESSOR_LEVEL=6

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set LIBRARY_PREFIX=C:\Users\Fabio\Anaconda\envs_build\Library

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set SCRIPTS=C:\Users\Fabio\Anaconda\envs_build\Scripts

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set SYS_PREFIX=C:\Users\Fabio\Anaconda

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set SRC_DIR=C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set COMMONPROGRAMW6432=C:\Program Files\Common Files

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set OS=Windows_NT

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set PUBLIC=C:\Users\Public

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>set USERPROFILE=C:\Users\Fabio

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>REM ===== end generated header =====

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>"C:\Users\Fabio\Anaconda\envs_build\python.exe" setup.py install running install running build running build_py creating build creating build\lib.win-amd64-2.7 creating build\lib.win-amd64-2.7\fastcache copying fastcache\benchmark.py -> build\lib.win-amd64-2.7\fastcache copying fastcacheinit.py -> build\lib.win-amd64-2.7\fastcache creating build\lib.win-amd64-2.7\fastcache\tests copying fastcache\tests\test_clrucache.py -> build\lib.win-amd64-2.7\fastcache\tests copying fastcache\tests\test_functools.py -> build\lib.win-amd64-2.7\fastcache\tests copying fastcache\tests\test_lrucache.py -> build\lib.win-amd64-2.7\fastcache\tests copying fastcache\tests\test_thread.py -> build\lib.win-amd64-2.7\fastcache\tests copying fastcache\testsinit.py -> build\lib.win-amd64-2.7\fastcache\tests running build_ext building 'fastcache._lrucache' extension creating build\temp.win-amd64-2.7 creating build\temp.win-amd64-2.7\Release creating build\temp.win-amd64-2.7\Release\src C:\Users\Fabio\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DN DEBUG -IC:\Users\Fabio\Anaconda\envs_build\include -IC:\Users\Fabio\Anaconda\envs_build\PC /Tcsrc/_lrucache.c /Fobuild\temp.win-am d64-2.7\Release\src/_lrucache.obj _lrucache.c C:\Users\Fabio\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:C:\Users\Fabio\Anaconda\envs_build\libs /LIBPATH:C:\Users\Fabio\Anaconda\envs_build\PCbuild\amd64 /EXPORT:init_lrucache b uild\temp.win-amd64-2.7\Release\src/_lrucache.obj /OUT:build\lib.win-amd64-2.7\fastcache_lrucache.pyd /IMPLIB:build\temp.win-amd64- 2.7\Release\src_lrucache.lib /MANIFESTFILE:build\temp.win-amd64-2.7\Release\src_lrucache.pyd.manifest _lrucache.obj : warning LNK4197: export 'init_lrucache' specified multiple times; using first specification Creating library build\temp.win-amd64-2.7\Release\src_lrucache.lib and object build\temp.win-amd64-2.7\Release\src_lrucache.exp

_lrucache.obj : error LNK2019: unresolved external symbol _PyGC_generation0 referenced in function make_key build\lib.win-amd64-2.7\fastcache_lrucache.pyd : fatal error LNK1120: 1 unresolved externals error: command 'C:\Users\Fabio\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\link.exe' failed with exit status 1120

C:\Users\Fabio\Anaconda\conda-bld\work\fastcache-1.0.1>if errorlevel 1 exit 1 Command failed: C:\Windows\system32\cmd.exe /c bld.bat

pbrady commented 9 years ago

@asmeurer, have you seen this error before? _PyGC_generation0 is part of the garbage collection infrastructure so I'm not how it could be missing.

@canesin, can you try to build the v0.4.3 release? I added support for threadsafety in 1.0 and I'm wondering if that's the culprit here.

canesin commented 9 years ago

@pbrady , with 0.4.3 I'm getting the same error. Has someone used this extension on windows before ?? I learned about it by reading sympy-0.7.6rc2 release notes, they are thinking of integrating it. Looking in binstar @asmeurer has as package for OSX only.

pbrady commented 9 years ago

Thanks for testing that out. Yours is the first feedback on installing this on windows but I would think that with conda everything would work fine. This may be as simple as needing to supply a custom build.bat file or some such thing. I'll ask around on the anaconda mailing list and see if there's a solution

asmeurer commented 9 years ago

You do need a blt.bat for your recipe to work on Windows. It will just be python setup.py install

asmeurer commented 9 years ago

I can reproduce this in both Python 2.7 and 3.4.

asmeurer commented 9 years ago

I believe the symbol is coming from _PyObject_GC_TRACK(hs);. According to https://hg.python.org/cpython/file/0f663e0ce1d3/Include/objimpl.h#l256, that symbol is not defined with Py_LIMITED_API. I don't know if that is the issue. I assume that because the macro starts with an underscore, it is private and shouldn't be used.

I have Windows VMs that I can test things on if you need me to.

pbrady commented 9 years ago

Now that you bring that up, I seem to recall something about *nix linkers exposing everything as the default while the windows linker has to be explicitly told to make things public. The _PyObject_GC_TRACK was copy/pasted from the list initialization code but if the symbols aren't exported I'll have to find another way around this.

I'll need to read the docs some more but I believe this is used for potentially self-referential objects. Even though the keys are lists, they can't be self-referential (since they are not exposed to the user) so I may be able to delete the offending line(s).

pbrady commented 9 years ago

This makes me think that @canesin is the first windows user!

canesin commented 9 years ago

@pbrady I'm trying to =)

pbrady commented 9 years ago

@asmeurer @canesin I've made a fix in #33. It actually says right in the docs not to use the macro version _PyObject_GC_TRACK in extensions so I switched to the function version. If you guys could test it out that would be great!

canesin commented 9 years ago

Nice =), it pass the test:

========================= 29 passed in 66.94 seconds ========================== True

pbrady commented 9 years ago

@canesin thanks for sticking with this and testing the fix out. I'll do an updated release soon.

asmeurer commented 9 years ago

As soon as you upload a release we will put this in the Continuum conda repos, so that people can get it to use with SymPy. We should perhaps make it a dependency of SymPy in the conda package, so that people using SymPy will always get it. Thoughts?

canesin commented 9 years ago

First, sorry of using this issue to telling this.. but since @asmeurer is talking about conda, I'm for it - I searched this to do exactly that, use fastcache in sympy, also - Spyder IDE is broken in conda, because it is incompatible with pygments >=2.0, there is a patch to it in the repository that conda should apply until they didn't release a new version https://code.google.com/p/spyderlib/source/detail?r=fc906f49c9c1

pbrady commented 9 years ago

Version 1.0.2 is now on pypi!

I think it would be really cool to make the binaries available for all platforms via conda. And once it's precompiled I don't see any problem with making it a dependency for the Sympy conda package since it's guaranteed to install.

pbrady commented 9 years ago

Ping me when it's in the Continuum repos and I'll change the readme

asmeurer commented 9 years ago

@canesin can you open an issue at https://github.com/continuumio/anaconda-issues/issues about that