pyodbc won't import within an embedded python interpreter

[GoogleCodeExporter]

I have run into a problem I can't figure out how to work around:

I can import pyodbc without error if I do it within a command line python 
interpreter.  However, I need to import and use it from an embedded python 
interpreter within my C++ program, and this fails.
(See attached screenshot.)

Attached is a simple zipped up VS2010 C++ program that embeds a python 
interpreter into some C++ code and demonstrates the problem.  (Included is a 
class that abstracts the nasty details of the embedding the interpreter, to 
provide a relatively clean interface for running python and extracting results 
from C++ code.)

It executes two simple python commands, one to import the pyexpat module, (also 
a  module implemented with a .pyd), which fails, and one to import pyodbc, 
which fails, claiming that the module could not be found.  (The pyexpat import 
is included merely to show that the embedded interpreter is working; if 
removed, the pyodbc import still fails.)

Python version: 2.7.2
pyodbc version: 2.1.8.win32-py2.7
Embedding language: (VS2010) C++
OS: Windows Vista or 7 (fails the same way on both)

Original issue reported on code.google.com by wsad...@gmail.com on 30 Jun 2011 at 8:04

Attachments:

• pyodbcBugScreenshot.jpg
• PyOdbcTest.zip

[GoogleCodeExporter]

I just noticed that I had a typo in my description; the import of pyexpat 
*succeeds*; it's just pyodbc that fails to import.  

I am eagerly awaiting some investigation of this, as I can't proceed without a 
fix for this problem.  (priority=high?)

Thanks,
/Will Sadkin

Original comment by wsad...@gmail.com on 8 Jul 2011 at 3:26

[GoogleCodeExporter]

I am almost certain the issue is because you are using Visual Studio 2010.  
Python 2.7 was built with 2008 and Microsoft put some boneheaded code into the 
2005 & 2008 CRTs making it very difficult to mix and match DLLs.  They have 
corrected their (completely inexcusable) error in 2010, BTW.

I created a simple Win32 command line project with only the following code:
int _tmain(int argc, TCHAR* argv[], TCHAR* envp[])
{
    Py_Initialize();
    if (!Py_IsInitialized())
    {
        printf("init fail\n");
        return 1;
    }
    if (PyRun_SimpleString("import pyodbc"))
    {
        PyErr_Print();
        return 1;
    }
    return 0;
}

This works fine with 2008 but fails with the same error with 2010.

(Also note that I built these in release mode since building a debug version of 
Python is so much trouble on Windows.)

I'm going to close this for now, but please reopen if you 2008 tests do not 
work.

Original comment by mkleehammer on 8 Jul 2011 at 9:00

• Changed state: NoFix

[GoogleCodeExporter]

<jaw on floor>

Just to be clear:  Is the incompatibilty because pyodbc.pyd was generated with 
VS2008, or because python2.7 (which I've linked into my VS2010 code with no 
reported problems, by the way) is compiled with VS2008, or both?

Is this "python is only compiled for compatibility with VS2008" restriction 
documented anywhere?

And, if your recommendation therefore to use the older compiler, for how long 
should I expect this to persist?
(related) Any idea when will Python 2.7 be compiled with the latest MS 
compiler, and will pyodbc then follow suit? 

<jaw on floor/>

Sorry, but I'm speechless.  I spent a long time porting my code to the latest 
MS compiler (because that's what you *do* in the microsoft world.)  And of 
course, I also upgraded python at the same time, because its supposed to be the 
stable release... @#$=&%^!

Respectfully,
/Will Sadkin

Original comment by wsad...@gmail.com on 8 Jul 2011 at 10:38

[GoogleCodeExporter]

The issue is that Python 2.5 - 2.7 are all built with VS2008.  That means 
Microsoft's XML manifests are embedded in the executable and the msvcrt DLL 
will check it and fail to load it if doesn't match exactly.  I believe what is 
happening is that the pyodbc manifest requires the 2008 CRT.

That means it might be possible to make something work by building without a 
manifest (I think would require a little distutils hacking?), modifying or 
removing it.  When I get some free time I'll try that, but you might look at it 
too.

There are no plans to ever change 2.7 compilers since it would break existing 
extensions.  pyodbc will always have to be built with the version that Python 
is built with.  I know 2010 is being looked at for some 3.x versions, but I 
don't remember the details.

Caveat: I haven't embedded Python in years and am mostly familiar with building 
extensions for python.exe.  I could be wrong, but I do know my tests worked w/ 
2008 and did not work w/ 2010.  I also know I have to update the manifests from 
SP1 down to the original python version of some other extensions I build for 
work.

Original comment by mkleehammer on 8 Jul 2011 at 10:50

• Changed state: Investigating

[GoogleCodeExporter]

Another question then:  If python *were* compiled with 2010, and if, as you 
say, "They have corrected their (completely inexcusable) error in 2010,", then 
wouldn't such a version of python be able to load older (ie. 2008-compiled) 
extensions?

Original comment by wsad...@gmail.com on 9 Jul 2011 at 2:18

[GoogleCodeExporter]

http://bugs.python.org/issue7833#msg98805

Original comment by mkleehammer on 9 Jul 2011 at 1:28

[GoogleCodeExporter]

I have a similar problem using pyodbc from an embedded interpreter compiled 
with VS2005 and python 2.7. While pyodbc is imported correctly from the 
standard interpreter, the embedded one returns a dll loading error.

Original comment by alex.r...@gmail.com on 29 Aug 2011 at 4:49

[GoogleCodeExporter]

I believe you will have a harder time fixing yours than I did mine.  I had to 
explicitly add a manifest dependency on the VS2008 CRT, using an additional 
manifest dependency of 
"type='Win32' name='Microsoft.VC90.CRT' version='9.0.21022.8' 
processorArchitecture='X86' publicKeyToken='1fc8b3b9a1e18e3b' " 

in the linker settings for the executable in question.  Once I did this, the 
interpreter would load properly.  However, that was compiling with VS2010; I 
don't know how "forward-compatible" VS2005 will be in that regard...

Original comment by wsad...@gmail.com on 29 Aug 2011 at 4:58

[GoogleCodeExporter]

Thank you very much. The strange thing is that using python 2.6 and the 2.1.7 
version of pyodbc all was working.

Original comment by alex.r...@gmail.com on 29 Aug 2011 at 5:03

[GoogleCodeExporter]

I temporarily solved the problem by recompiling pyodbc with VS2005, now the 
pyodbc.pyd module is successfully loaded from the embedded interpreter.

Original comment by alex.r...@gmail.com on 30 Aug 2011 at 8:20

[GoogleCodeExporter]

Please see issue 214.

Original comment by alfr...@gmail.com on 6 Oct 2011 at 3:30

[GoogleCodeExporter]

I am running into the same issue where I cannot import pyodbc into an embedded 
Python interpretor, but this is on Linux (CentOS 5.5). The following error is 
given:

ImportError: /usr/local/lib/python2.6/site-packages/pyodbc.so: undefined 
symbol: _Py_ZeroStruct

Does anyone know what what could be causing this?

Original comment by rich.ma...@gmail.com on 8 Nov 2011 at 12:44