Import error [VS2012 vs Python27?]

GoogleCodeExporter commented 9 years ago

Hello to everybody, I'm trying to make a precompiled installer for the CGAL 
bindings for Python 2.7 for win32 and I have some problem in importing the 
modules.

What I did.
-----------
I did the following macro-steps:
1) I compiled CGAL-4.3 using Visual Studio - win32;
2) I compiled CGAL-bindings using Visual Studio - win32;
Then I've put the resulting files in a single folder "CGAL" (by joining the 
folders "build-python\CGAL" and "build-python\CGAL\Release". Here is a summary 
of the files I can found directly under my "CGAL" folder:
- "__init__.py"
- "CGAL_[module].py"
- "_CGAL_[module].exp"
- "_CGAL_[module].lib"
- "_CGAL_[module].pyd"
( for every [module] in the bindings )

The problem.
------------
I'm trying to load a module, to see if everything is working properly. For 
example, let's take "CGAL_Kernel". I open a Python console inside the "CGAL" 
folder and here is my error:

    >>> import sys
    >>> sys.path.append(CGAL_folder)
    >>> import CGAL_Kernel
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "CGAL_Kernel.py", line 28, in <module>
        _CGAL_Kernel = swig_import_helper()
      File "CGAL_Kernel.py", line 24, in swig_import_helper
        _mod = imp.load_module('_CGAL_Kernel', fp, pathname, description)
    ImportError: DLL load failed

Of course I get the same error if I try "import _CGAL_Kernel.pyd".

System informations:
--------------------
I'm using Windows 7 x64, Python 2.7 win32. The CGAL were compiled using Visual 
Studio 2012 Professional, in "Release" mode, for win32 systems (and the python 
bindings too).

I wonder what I'm doing wrong, maybe I have to compile everything using Visual 
Studio 2008 (or compiling Python with VS2012)?

Best regards, and thank you for your work!

P.S. If I manage to make it work I can make a "setup.py" and upload some 
precompiled-binary-installer.

Original issue reported on code.google.com by marco.ce...@gmail.com on 8 Nov 2013 at 1:55

GoogleCodeExporter commented 9 years ago

what is "CGAL_folder" ? An absolute or a relative path?

Original comment by sloriot...@gmail.com on 18 Nov 2013 at 8:44

GoogleCodeExporter commented 9 years ago

CGAL_folder is an absolute path... I can't understand if the DLL is not found 
or something fails during loading (due to dependencies, for example). I 
recently compiled everything with Visual Studio 2008 and added some DLL to 
CGAL_folder, such as gmp, boost_system, boost_thread and msvc dlls. I also 
added "CGAL_folder" to my PATH environment variable and to PYTHONPATH, but no 
success.

I can make a ZIP with the content of CGAL_folder if you want to test directly...

Original comment by marco.ce...@gmail.com on 18 Nov 2013 at 4:22

GoogleCodeExporter commented 9 years ago

Is it working when you use the cmake setup provided, or it is only a problem 
with the process to create the installer?

Original comment by sloriot...@gmail.com on 19 Nov 2013 at 7:55

GoogleCodeExporter commented 9 years ago

The problem is that I can't make it work from Python: import always fails.

It looks like the swig-generated python script which loads the ".pyd" files 
fails. I don't have the files right here, but if I remember correctly it has 
two steps:
- try: swig_find _CGAL_module
- except: import _CGAL_module
It always fall in the second case and then it fails, and even if I put every 
file in the current folder and set environment variables and python path. What 
I am trying to understand now is what is behind this "ImportError: DLL load 
failed".

This vague error message can arise from multiple causes, for example it can be 
related to some DLL dependencies. I've traced the dependencies of the "*.pyd" 
files and I've put everything together, but I'm still not able to load anything 
from python.

If you want to test yourself from a Windows machine I can zip the compiled 
files and post a link.

Original comment by marco.ce...@gmail.com on 19 Nov 2013 at 9:52

GoogleCodeExporter commented 9 years ago

Note: some similar or related problems on stackoverflow,
http://stackoverflow.com/questions/5667556/
http://stackoverflow.com/questions/4684697/

Original comment by marco.ce...@gmail.com on 19 Nov 2013 at 10:02

GoogleCodeExporter commented 9 years ago

I've managed to make it work. The problem was in the DLL dependencies. I've 
used the Dependency Walker (www.dependencywalker.com) to track them down and I 
tried to find them in my system (or in MS redistributable packages). I've 
created precompiled installer for Python 2.7.x on win32. Here is the link for 
any (unfortunate) Python windows user:
http://goo.gl/FL91S9

Original comment by marco.ce...@gmail.com on 19 Nov 2013 at 9:00

Attachments:

cgal-bindings-20131119.py27win32.exe

GoogleCodeExporter commented 9 years ago

Could you describe the procedure to build this installer? I could try to make 
it automatic each time I update the bindings.

Original comment by sloriot...@gmail.com on 19 Nov 2013 at 9:11

GoogleCodeExporter commented 9 years ago

Sure! this is a full step by step tutorial from the beginning.

Step 1: compile the CGAL
------------------------
I assume this is no problem for you, but for everyone reading after a Google 
search...

Useful links:
- Visual Studio 2008 Express: http://go.microsoft.com/?linkid=7729279
- Boost precompiled binaries: http://boost.teeks99.com/
- Zlib: http://www.zlib.net/
- CMake for Windows: http://www.cmake.org/cmake/resources/software.html 
- CGAL setup installer: https://gforge.inria.fr/frs/?group_id=52
- CGAL guide: 
http://doc.cgal.org/latest/Manual/installation.html#sseccgalwindows

Note:
Have patience and set manually Boost_INCLUDE_DIR, Boost_LIBRARY_DIR, 
Boost_SYSTEM_LIBRARY_DEBUG,
Boost_SYSTEM_LIRARY_RELEASE, Boost_THREAD_LIBRARY_DEBUG, 
Boost_THREAD_LIBRARY_RELEASE,
ZLIB_INCLUDE_DIR, ZLIB_LIBRARY. Uncheck WITH_CGAL_Qt3. Configure multiple times 
until everything
works. Then open the solution using Visual Studio and compile. At the end you 
will have the
binaries under CGAL-4.3-Setup-win32\bin and the libraries under 
CGAL-4.3-Setup-win32\lib. Keep
the content of the folder CGAL-4.3-Setup-win32 as is, you'll need it later.

Step 2: compile the CGAL-bindings
---------------------------------
This is even less problematic I think :-)

Useful links:
- CGAL-bindings source:
  http://cgal-bindings.googlecode.com/archive/756580059385720451bfed45491cee8f86f6551b.zip

Note:
Uncheck BUILD_JAVA, check BUILD_PYTHON, set CGAL_DIR to your 
CGAL-4.3-Setup-win32 folder.
The result of this operation is contained in the folder 
cgal-bindings\build-python.

Step 3: track down the DLL dependencies
---------------------------------------
Before making an installer you need to put in a folder a Python module which 
works even
if you move that folder on another computer. This means that you have to put 
everything
it needs in one folder, and track the dependencies of the DLL file. I've put 
the binaries
of the CGAL and the bindings into a single folder "binaries".

Useful links:
- Dependency Walker: http://www.dependencywalker.com/
- Microsoft Visual C++ 2008 Redist: 
http://www.microsoft.com/en-us/download/details.aspx?id=29
- Visual Studio 2008 SDK 1.1: 
http://www.microsoft.com/en-us/download/details.aspx?id=21827
- In desperate cases: http://www.opendll.com/

The structure of the package will look like this:

MyLocalFolder\CGAL\binaries: 
    insert in this folder the dll, pyd and every dependency you found

MyLocalFolder\CGAL\
    put here the swig generated modules "CGAL_[module].py" and the "__init__.py"

MyLocalFolder\
    create in this folder a file named "setup.py"

Here is my final folder structure, in case the above instruction are not clear.
There are probably many useless files here... but just to be extrasafe...
    MyLocalFolder\
    |    + CGAL\
    |    |    + binaries\
    |    |    |    api-ms-win-appmodel-runtime-l1-1-0.dll
    |    |    |    api-ms-win-core-debug-l1-1-1.dll
    |    |    |    api-ms-win-core-delayload-l1-1-1.dll
    |    |    |    api-ms-win-core-errorhandling-l1-1-1.dll
    |    |    |    api-ms-win-core-heap-l1-2-0.dll
    |    |    |    api-ms-win-core-interlocked-l1-2-0.dll
    |    |    |    api-ms-win-core-libraryloader-l1-1-1.dll
    |    |    |    api-ms-win-core-memory-l1-1-1.dll
    |    |    |    api-ms-win-core-processthreads-l1-1-1.dll
    |    |    |    api-ms-win-core-registry-l1-1-0.dll
    |    |    |    api-ms-win-core-rtlsupport-l1-2-0.dll
    |    |    |    api-ms-win-core-synch-l1-2-0.dll
    |    |    |    api-ms-win-core-sysinfo-l1-2-0.dll
    |    |    |    api-ms-win-core-threadpool-l1-2-0.dll
    |    |    |    api-ms-win-core-winrt-error-l1-1-0.dll
    |    |    |    api-ms-win-core-winrt-l1-1-0.dll
    |    |    |    api-ms-win-core-winrt-robuffer-l1-1-0.dll
    |    |    |    api-ms-win-core-winrt-string-l1-1-0.dll
    |    |    |    api-ms-win-eventing-provider-l1-1-0.dll
    |    |    |    api-ms-win-shcore-scaling-l1-1-1.dll
    |    |    |    api-ms-win-shcore-scaling-l1-1.dll
    |    |    |    boost_chrono-vc90-mt-1_54.dll
    |    |    |    boost_system-vc90-mt-1_54.dll
    |    |    |    boost_thread-vc90-mt-1_54.dll
    |    |    |    CGAL-vc90-mt-4.3.dll
    |    |    |    CGAL_AABB_tree_cpp.dll
    |    |    |    CGAL_Alpha_shape_2_cpp.dll
    |    |    |    CGAL_Core-vc90-mt-4.3.dll
    |    |    |    CGAL_Core-vc90-mt-gd-4.3.dll
    |    |    |    CGAL_ImageIO-vc90-mt-4.3.dll
    |    |    |    CGAL_ImageIO-vc90-mt-gd-4.3.dll
    |    |    |    CGAL_Kernel_cpp.dll
    |    |    |    CGAL_Mesh_3_cpp.dll
    |    |    |    CGAL_Surface_mesher_cpp.dll
    |    |    |    CGAL_Triangulation_2_cpp.dll
    |    |    |    CGAL_Triangulation_3_cpp.dll
    |    |    |    dcomp.dll
    |    |    |    gpsvc.dll
    |    |    |    ieshims.dll
    |    |    |    kernel32.dll
    |    |    |    libgmp-10.dll
    |    |    |    libmpfr-4.dll
    |    |    |    listfiles.py
    |    |    |    msvcm90.dll
    |    |    |    msvcp90.dll
    |    |    |    msvcr90.dll
    |    |    |    msvcrt.dll
    |    |    |    python27.dll
    |    |    |    sysntfy.dll
    |    |    |    _CGAL_AABB_tree.exp
    |    |    |    _CGAL_AABB_tree.lib
    |    |    |    _CGAL_AABB_tree.pyd
    |    |    |    _CGAL_Alpha_shape_2.exp
    |    |    |    _CGAL_Alpha_shape_2.lib
    |    |    |    _CGAL_Alpha_shape_2.pyd
    |    |    |    _CGAL_Convex_hull_2.exp
    |    |    |    _CGAL_Convex_hull_2.lib
    |    |    |    _CGAL_Convex_hull_2.pyd
    |    |    |    _CGAL_HalfedgeDS.exp
    |    |    |    _CGAL_HalfedgeDS.lib
    |    |    |    _CGAL_HalfedgeDS.pyd
    |    |    |    _CGAL_Interpolation.exp
    |    |    |    _CGAL_Interpolation.lib
    |    |    |    _CGAL_Interpolation.pyd
    |    |    |    _CGAL_Kernel.exp
    |    |    |    _CGAL_Kernel.lib
    |    |    |    _CGAL_Kernel.pyd
    |    |    |    _CGAL_Mesh_2.exp
    |    |    |    _CGAL_Mesh_2.lib
    |    |    |    _CGAL_Mesh_2.pyd
    |    |    |    _CGAL_Mesh_3.exp
    |    |    |    _CGAL_Mesh_3.lib
    |    |    |    _CGAL_Mesh_3.pyd
    |    |    |    _CGAL_Polyhedron_3.exp
    |    |    |    _CGAL_Polyhedron_3.lib
    |    |    |    _CGAL_Polyhedron_3.pyd
    |    |    |    _CGAL_Spatial_searching.exp
    |    |    |    _CGAL_Spatial_searching.lib
    |    |    |    _CGAL_Spatial_searching.pyd
    |    |    |    _CGAL_Surface_mesher.exp
    |    |    |    _CGAL_Surface_mesher.lib
    |    |    |    _CGAL_Surface_mesher.pyd
    |    |    |    _CGAL_Triangulation_2.exp
    |    |    |    _CGAL_Triangulation_2.lib
    |    |    |    _CGAL_Triangulation_2.pyd
    |    |    |    _CGAL_Triangulation_3.exp
    |    |    |    _CGAL_Triangulation_3.lib
    |    |    |    _CGAL_Triangulation_3.pyd
    |    |    |    _CGAL_Voronoi_diagram_2.exp
    |    |    |    _CGAL_Voronoi_diagram_2.lib
    |    |    |    _CGAL_Voronoi_diagram_2.pyd
    |    |     __init__.py
    |    |     CGAL_AABB_tree.py
    |    |     CGAL_Alpha_shape_2.py
    |    |     CGAL_Convex_hull_2.py
    |    |     CGAL_HalfedgeDS.py
    |    |     CGAL_Interpolation.py
    |    |     CGAL_Kernel.py
    |    |     CGAL_Mesh_2.py
    |    |     CGAL_Mesh_3.py
    |    |     CGAL_Polyhedron_3.py
    |    |     CGAL_Spatial_searching.py
    |    |     CGAL_Surface_mesher.py
    |    |     CGAL_Triangulation_2.py
    |    |     CGAL_Triangulation_3.py
    |    |     CGAL_Voronoi_diagram_2.py
    |    setup.py

Step 4: add the binaries folder to the path in the __init__.py
--------------------------------------------------------------
Swig have its own way of searching modules...
...but just to be extrasafe, here is my "__init__.py" file

import os, sys

# Make sure python finds what is in this file's folder
_folder = os.path.join(os.path.dirname(os.path.realpath(__file__)))
os.environ['LD_LIBRARY_PATH'] = _folder
os.putenv('LD_LIBRARY_PATH', _folder)
sys.path.insert(0, _folder)

# Make sure python finds what is in the subfolder binaries
_binfolder = os.path.join(_folder, "binaries")
os.environ["LD_LIBRARY_PATH"] = _binfolder
os.putenv("LD_LIBRARY_PATH", _binfolder)
sys.path.insert(0, _binfolder)

import CGAL_AABB_tree
import CGAL_Alpha_shape_2
import CGAL_Convex_hull_2
import CGAL_HalfedgeDS
import CGAL_Interpolation
import CGAL_Kernel
import CGAL_Mesh_2
import CGAL_Mesh_3
import CGAL_Polyhedron_3
import CGAL_Spatial_searching
import CGAL_Surface_mesher
import CGAL_Triangulation_2
import CGAL_Triangulation_3
import CGAL_Voronoi_diagram_2

Step 5: test the module on another computer
-------------------------------------------
Before packaging everything copy the folder in another computer, open the 
terminal,
navigate to "MyLocalFolder" and test if everything is working:

> cd C:\Users\MyName\Path\To\MyLocalFolder
> python
    Python 2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit (Intel)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import CGAL
    >>> # OK!

Step 6: create the setup.py file
--------------------------------
Is a simple python script which uses distutils for defining what goes into the 
package.
Here is my setup.py file, as an example:

import sys
from distutils.core import setup

setup(
    name = "cgal-bindings",
    version = "20131119",
    description = "Precompiled binaries for CGAL-bindings",
    author = "Marco Centin (of this installer!)",
    author_email = "marco [dot] centin [at] gmail [dot] com",
    url = "http://code.google.com/p/cgal-bindings/",
    packages = ['CGAL'],
    package_dir = { 'CGAL': 'CGAL' },
    package_data = { 'CGAL': [
        'binaries/*.dll',
        'binaries/*.lib', 
        'binaries/*.pyd', 
        'binaries/*.exp' ] },
    long_description = \
    """ Precompiled binaries for CGAL-bindings (unofficial)

    This installers provides precompiled binaries for the CGAL-bindings
    under Windows. The official project web page can be found here:
    * http://code.google.com/p/cgal-bindings/source/browse/

    WARNING: this is an experimental package, use it at your risk!

    System requirements:
    - Microsoft Windows (tested on Windows 7 64bit only);
    - Python 2.7.x for Windows 32bit;
    - Microsoft Windows SDK for Windows 7 and .NET Framework 4
    - Microsoft Visual C++ 2008 Redistributable Package (x86)

    Installation notes:
    - Make sure you have Python 2.7 for win32;
    - Run this installer as administrator (rightclick->run as admin)!

    """
)

Step 7: create the installer!
-----------------------------
Go into the folder MyLocalFolder, where the "setup.py" file is, and run the 
command:

>[path\to\python\]python.exe setup.py bdist --format=wininst

The final installer will be saved in the "dist" subfolder.

I hope that helps,
Marco

Original comment by marco.ce...@gmail.com on 20 Nov 2013 at 6:24

GoogleCodeExporter commented 9 years ago

Hello,

Thank you for this tutorial. 
Anybody as tried the .exe file proposed by marco? 
Do you still planned to make it automatic for futur version?

Regards,
Lorenzo

Original comment by lorenzo....@gmail.com on 18 Feb 2014 at 5:13

GoogleCodeExporter commented 9 years ago

It's on my TODO list but I don't have time for now.

Original comment by sloriot...@gmail.com on 18 Feb 2014 at 6:56

chrisidefix / cgal-bindings

Import error [VS2012 vs Python27?] #39