search

jpype-project / jpype

JPype is cross language bridge to allow Python programs full access to Java class libraries.
http://www.jpype.org
Apache License 2.0
1.12k stars 183 forks source link

Python 3.12 support with the xlclang compiler #1177

Closed pfgdrupalas closed 8 months ago

pfgdrupalas commented 8 months ago

Hello,

We are needing to migrate to Python 3.12, but in trying to install JayDeBeapi, it looks like we are taking an error that is due to JPYPE not being supported for v3r12.

The install for JayDeBE looks like: `Using pip 23.2.1 from /MVST/products/python/v3r12/pyz/lib/python3.12/site-packages/pip (python 3.12) Collecting JayDeBeApi Obtaining dependency information for JayDeBeApi from https://files.pythonhosted.org/packages/ff/1f/6a627c9bd7dea13235b65fce0fff987507269d41f957c578031796f70319/JayDeBeApi-1.2.3-py3-none-any.whl.metadata Downloading JayDeBeApi-1.2.3-py3-none-any.whl.metadata (10 kB) Collecting JPype1 (from JayDeBeApi) Using cached JPype1-1.5.0.tar.gz (819 kB) Running command pip subprocess to install build dependencies Collecting setuptools>=40.8.0 Obtaining dependency information for setuptools>=40.8.0 from https://files.pythonhosted.org/packages/92/e1/1c8bb3420105e70bdf357d57dd5567202b4ef8d27f810e98bb962d950834/setuptools-69.2.0-py3-none-any.whl.metadata Downloading setuptools-69.2.0-py3-none-any.whl.metadata (6.3 kB) Collecting wheel Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl.metadata Using cached wheel-0.43.0-py3-none-any.whl.metadata (2.2 kB) Downloading setuptools-69.2.0-py3-none-any.whl (821 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 821.5/821.5 kB 4.7 MB/s eta 0:00:00 Using cached wheel-0.43.0-py3-none-any.whl (65 kB) Installing collected packages: wheel, setuptools Successfully installed setuptools-69.2.0 wheel-0.43.0

[notice] A new release of pip is available: 23.2.1 -> 24.0 [notice] To update, run: python3.12 -m pip install --upgrade pip Installing build dependencies ... done Running command Getting requirements to build wheel Falling back to provided JNI headers, since your provided JAVA_HOME "/usr/lpp/java/J17/J17.0_64" does not provide jni.h running egg_info writing JPype1.egg-info/PKG-INFO writing dependency_links to JPype1.egg-info/dependency_links.txt writing entry points to JPype1.egg-info/entry_points.txt writing requirements to JPype1.egg-info/requires.txt writing top-level names to JPype1.egg-info/top_level.txt reading manifest file 'JPype1.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching '.class' under directory 'native' no previously-included directories found matching 'test/classes/' adding license file 'LICENSE' adding license file 'NOTICE' adding license file 'AUTHORS.rst' writing manifest file 'JPype1.egg-info/SOURCES.txt' Getting requirements to build wheel ... done Running command Preparing metadata (pyproject.toml) Falling back to provided JNI headers, since your provided JAVA_HOME "/usr/lpp/java/J17/J17.0_64" does not provide jni.h running dist_info creating /MVST/products/python/tmp/pip-modern-metadata-fz0_doqh/JPype1.egg-info writing /MVST/products/python/tmp/pip-modern-metadata-fz0_doqh/JPype1.egg-info/PKG-INFO writing dependency_links to /MVST/products/python/tmp/pip-modern-metadata-fz0_doqh/JPype1.egg-info/dependency_links.txt writing entry points to /MVST/products/python/tmp/pip-modern-metadata-fz0_doqh/JPype1.egg-info/entry_points.txt writing requirements to /MVST/products/python/tmp/pip-modern-metadata-fz0_doqh/JPype1.egg-info/requires.txt writing top-level names to /MVST/products/python/tmp/pip-modern-metadata-fz0_doqh/JPype1.egg-info/top_level.txt writing manifest file '/MVST/products/python/tmp/pip-modern-metadata-fz0_doqh/JPype1.egg-info/SOURCES.txt' reading manifest file '/MVST/products/python/tmp/pip-modern-metadata-fz0_doqh/JPype1.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching '.class' under directory 'native' no previously-included directories found matching 'test/classes/' adding license file 'LICENSE' adding license file 'NOTICE' adding license file 'AUTHORS.rst' writing manifest file '/MVST/products/python/tmp/pip-modern-metadata-fz0_doqh/JPype1.egg-info/SOURCES.txt' creating '/MVST/products/python/tmp/pip-modern-metadata-fz0_doqh/JPype1-1.5.0.dist-info' Preparing metadata (pyproject.toml) ... done Collecting packaging (from JPype1->JayDeBeApi) Obtaining dependency information for packaging from https://files.pythonhosted.org/packages/49/df/1fceb2f8900f8639e278b056416d49134fb8d84c5942ffaa01ad34782422/packaging-24.0-py3-none-any.whl.metadata Downloading packaging-24.0-py3-none-any.whl.metadata (3.2 kB) Using cached JayDeBeApi-1.2.3-py3-none-any.whl (26 kB) Downloading packaging-24.0-py3-none-any.whl (53 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.5/53.5 kB 1.1 MB/s eta 0:00:00 Building wheels for collected packages: JPype1 Running command Building wheel for JPype1 (pyproject.toml) Falling back to provided JNI headers, since your provided JAVA_HOME "/usr/lpp/java/J17/J17.0_64" does not provide jni.h running bdist_wheel running build running build_py creating build creating build/lib.os390-28.00-3931-cpython-312 creating build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/init.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_classpath.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_core.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_gui.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_jarray.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_jclass.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_jcollection.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_jcustomizer.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_jexception.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_jinit.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_jio.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_jmethod.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_jobject.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_jpackage.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_jproxy.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_jstring.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_jthread.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_jvmfinder.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_pykeywords.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/beans.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/config.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/dbapi2.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/imports.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/nio.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/pickle.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/protocol.py -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/types.py -> build/lib.os390-28.00-3931-cpython-312/jpype creating build/lib.os390-28.00-3931-cpython-312/jpype/_pyinstaller copying jpype/_pyinstaller/entry_points.py -> build/lib.os390-28.00-3931-cpython-312/jpype/_pyinstaller copying jpype/_pyinstaller/example.py -> build/lib.os390-28.00-3931-cpython-312/jpype/_pyinstaller copying jpype/_pyinstaller/hook-jpype.py -> build/lib.os390-28.00-3931-cpython-312/jpype/_pyinstaller copying jpype/_pyinstaller/test_jpype_pyinstaller.py -> build/lib.os390-28.00-3931-cpython-312/jpype/_pyinstaller copying jpype/_jcollection.pyi -> build/lib.os390-28.00-3931-cpython-312/jpype copying jpype/_jio.pyi -> build/lib.os390-28.00-3931-cpython-312/jpype running build_ext Call build extensions Using Jar cache copying native/jars/org.jpype.jar -> build/lib.os390-28.00-3931-cpython-312 Call build ext building '_jpype' extension creating build/temp.os390-28.00-3931-cpython-312 creating build/temp.os390-28.00-3931-cpython-312/native creating build/temp.os390-28.00-3931-cpython-312/native/common creating build/temp.os390-28.00-3931-cpython-312/native/python /usr/lpp/cbclib/xlclang/bin/xlclang -DNDEBUG -D_XOPEN_SOURCE_EXTENDED -D_UNIX03_THREADS -D_POSIX_THREADS -D_OPEN_SYS_FILE_EXT -Inative/common/include -Inative/python/include -Inative/embedded/include -I/usr/lpp/java/J17/J17.0_64/include -Inative/jni_include -I/MVST/products/python/v3r12/pyz/include/python3.12 -c native/common/jp_array.cpp -o build/temp.os390-28.00-3931-cpython-312/native/common/jp_array.o In file included from native/common/jp_array.cpp:16: In file included from native/common/include/jpype.h:184: In file included from native/python/include/jp_pythontypes.h:18: In file included from /MVST/products/python/v3r12/pyz/include/python3.12/Python.h:38: In file included from /MVST/products/python/v3r12/pyz/include/python3.12/pyport.h:202: In file included from /usr/include/c++/math.h:301: /usr/include/le/math.h:5327:9: error: use of undeclared identifier 'static_assert' __static_assert((sizeof(T) < 0 ), ^ /usr/include/le/math.h:5361:9: error: use of undeclared identifier 'static_assert' static_assert((sizeof(T) < 0 ), ^ /usr/include/le/math.h:5393:9: error: use of undeclared identifier '__static_assert' static_assert((sizeof(T) < 0 ), ^ /usr/include/le/math.h:5426:9: error: use of undeclared identifier 'static_assert' __static_assert((sizeof(T) < 0 ), ^ /usr/include/le/math.h:5459:9: error: use of undeclared identifier 'static_assert' static_assert((sizeof(T) < 0 ), ^ /usr/include/le/math.h:5492:9: error: use of undeclared identifier '__static_assert' static_assert((sizeof(T) < 0 ), ^ /usr/include/le/math.h:5529:9: error: use of undeclared identifier 'static_assert' __static_assert((sizeof(T) < 0 ), ^ /usr/include/le/math.h:5562:9: error: use of undeclared identifier 'static_assert' static_assert((sizeof(T) < 0 ), ^ /usr/include/le/math.h:5595:9: error: use of undeclared identifier '__static_assert' static_assert((sizeof(T) < 0 ), ^ /usr/include/le/math.h:5631:9: error: use of undeclared identifier 'static_assert' __static_assert((sizeof(T) < 0 ), ^ /usr/include/le/math.h:5664:9: error: use of undeclared identifier 'static_assert' static_assert((sizeof(T) < 0 ), ^ /usr/include/le/math.h:5697:9: error: use of undeclared identifier '__static_assert' static_assert((sizeof(T) < 0 ), ^ /usr/include/le/math.h:5732:9: error: use of undeclared identifier '__static_assert' __static_assert((sizeof(T) < 0 ), ^ 13 errors generated. Error while processing native/common/jp_array.cpp. CCN0793(I) Compilation failed for file native/common/jp_array.cpp. Object file not created. error: command '/usr/lpp/cbclib/xlclang/bin/xlclang' failed with exit code 12 error: subprocess-exited-with-error

× Building wheel for JPype1 (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip. full command: /MVST/products/python/v3r12/pyz/bin/python3.12 /MVST/products/python/v3r12/pyz/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /products/python/tmp/tmpnbsasrnh cwd: /MVST/products/python/tmp/pip-install-xh_2_l8i/jpype1_bf4998e581a14076b5a3a7c7e066b47b Building wheel for JPype1 (pyproject.toml) ... error ERROR: Failed building wheel for JPype1 Failed to build JPype1 ERROR: Could not build wheels for JPype1, which is required to install pyproject.toml-based projects `

Thanks!

Thrameos commented 8 months ago

1147

The error message says something is wrong with math.h which is not part of jpype.

pfgdrupalas commented 8 months ago

The end error: Building wheel for JPype1 (pyproject.toml) ... error ERROR: Failed building wheel for JPype1 Failed to build JPype1 ERROR: Could not build wheels for JPype1, which is required to install pyproject.toml-based projects

Indicates this was part of the JPYPE1 wheel build.

You can also see the program call error path: In file included from native/common/jp_array.cpp:16: In file included from native/common/include/jpype.h:184: In file included from native/python/include/jp_pythontypes.h:18: In file included from /MVST/products/python/v3r12/pyz/include/python3.12/Python.h:38: In file included from /MVST/products/python/v3r12/pyz/include/python3.12/pyport.h:202: In file included from /usr/include/c++/math.h:301:

Thanks!

Thrameos commented 8 months ago

Yes... If JPype can't import the system include file by importing "Python.h" then it can't be built. But this has little to do with JPype and more to do with a broken system. Calling #include "Python.h" and having it fail is not a coding bug with the module, but rather an issue with the system.

I see a few possibilities: 1) The flags required for your system are different that others. You would need to figure out what those flags need to be and modify "setupext/platform.py" to reflect the changes and submit a PR.
2) Your system is just busted with a C++ compiler and headers that are incompatible. Meaning any C++ code that includes Python.h will bust. In such a case, the error has nothing to do with JPype.

If I had to guess the headers for C++ and the compiler came from different packages and that is causing the issue. But I can't rule out the flags. Just doing a google search, see page 13 of https://www.ibm.com/support/pages/sites/default/files/inline-files/$FILE/SC31580100_0.pdf

pfgdrupalas commented 8 months ago

All the pip3 install commands worked on the same system with v3.9. We are just trying to migrate to v3.12 and took this type of error on 3 of the 13 install items.

Thanks!

Classification: Internal Use From: Karl Nelson @.> Sent: Friday, March 15, 2024 3:51 PM To: jpype-project/jpype @.> Cc: Drew, AJ @.>; Author @.> Subject: [External] Re: [jpype-project/jpype] When will JPYPE support v3.12? (Issue #1177)

CAUTION: This email originated from a source outside the Principal network. Do not click links or open attachments unless you recognize the sender and know the content is safe. The original sender of this email is @.**@.>.

Yes... If JPype can't import the system include file by importing "Python.h" then it can't be built. But this has little to do with JPype and more to do with a broken system. Calling #include "Python.h" and having it fail is not a coding bug with the module, but rather an issue with the system.

I see a few possibilities:

  1. The flags required for your system are different that others. You would need to figure out what those flags need to be and modify "setupext/platform.py" to reflect the changes and submit a PR.
  2. Your system is just busted with a C++ compiler and headers that are incompatible. Meaning any C++ code that includes Python.h will bust. In such a case, the error has nothing to do with JPype.

If I had to guess the headers for C++ and the compiler came from different packages and that is causing the issue. But I can't rule out the flags. Just doing a google search, see page 13 of https://www.ibm.com/support/pages/sites/default/files/inline-files/$FILE/SC31580100_0.pdfhttps://urldefense.com/v3/__https:/www.ibm.com/support/pages/sites/default/files/inline-files/$FILE/SC31580100_0.pdf__;!!Hd8rfk_wYxoe!OVkbPKI3mexm_xq6T_Cd1eIyaYxMtUYDi196AfPXbskSTfgaRXfvKmyt8zaKMgOX6DWDQ6pLQneS78tbzB2f_4ZtTg$

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/jpype-project/jpype/issues/1177*issuecomment-2000423111__;Iw!!Hd8rfk_wYxoe!OVkbPKI3mexm_xq6T_Cd1eIyaYxMtUYDi196AfPXbskSTfgaRXfvKmyt8zaKMgOX6DWDQ6pLQneS78tbzB01F8JF4A$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ADHJPDBIJR6PIV77M6O2G2DYYNNLHAVCNFSM6AAAAABEWU67DSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBQGQZDGMJRGE__;!!Hd8rfk_wYxoe!OVkbPKI3mexm_xq6T_Cd1eIyaYxMtUYDi196AfPXbskSTfgaRXfvKmyt8zaKMgOX6DWDQ6pLQneS78tbzB0Ri1J8pA$. You are receiving this because you authored the thread.Message ID: @.**@.>>

-----Message Disclaimer----- This email is intended for the recipient specified in this message. It may contain information that is privileged, confidential and exempt from disclosure under applicable laws. If you are not the intended recipient, sharing or copying this communication is prohibited. If received in error, please send a note to @.*** and delete the original message and any attachments. Principal Financial Group or any of its member companies may retain emails as required by law or regulation. This email is not an electronic signature, unless a specific statement to the contrary is included in this message.

To unsubscribe from marketing communications, go to www.principal.com/do-not-contact-form.

Thrameos commented 8 months ago

@pelson could there be a change in the flags with the installion upgrade? I don't see how it would work in 3.9 and not 3.12. As the error is in a system library, the only way I could see a difference is if Python 3.12 shipped with flags that were incompatible with c++. Unfortunately I don't know xlclang, but it does seem odd that the c++11 flag isn't there, so I am hoping you may have an idea.

pelson commented 8 months ago

The first thing to say - JPype does support Python 3.12, and even ships pre-built binaries for the manylinux platforms. The JPype pipelines validate Python 3.12 support systematically. I think we should update the issue's title to be more helpful - "Python 3.12 support with the xlclang compiler". Unfortunately, I don't have the permission to do that (perhaps that is something we can fix @Thrameos / @marscher?).

It seems unlikely to me that this has changed with the move to pyproject.toml in !1133, for a start, this report is based on a build coming from a source tarball release from December :wink:.

As for being able to reproduce this - I don't have access to an machine with xlclang either.

We are just trying to migrate to v3.12 and took this type of error on 3 of the 13 install items.

Is this to say that this isn't JPype specific, and you are seeing it with other project too? In such a scenario it is clear that the problem is that the system is not in a good state, and that is not something we can fix/workaround on the JPype level.

pfgdrupalas commented 8 months ago

Hello,

The 3 modules that failed xlclang are JayDeBe (install log provided), ansible-tower, and zoautil_py,

The rest did not try to use/need xlclang.

Other than the versions of the modules trying to be installed nothing has change on our system. We are running on an IBM z/OS mainframe.

So, yes something might be "off" with our system, but prior versions of these modules installed just fine with no errors, so I am at a loss as to how to try and figure out what exactly is wrong.

Thanks!

pelson commented 8 months ago

Are you able to re-build JPype on Python 3.9? At least then we can see if this is a indeed something that has changed on our side, or if this is a system change for which you will need to speak to your sys admins about.

pfgdrupalas commented 8 months ago

Hello,

Yes, I was able to uninstall and then reinstall JPYPE1 in our Python 3.9 environment:

pip3 install --trusted-host pypi.org --trusted-host files.pythonhosted.org JPype1 Collecting JPype1 Using cached JPype1-1.5.0.tar.gz (819 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: packaging in /MVST/products/python/v3r9/pyz/lib/python3.9/site-packages (from JPype1) (23.1) Building wheels for collected packages: JPype1 Building wheel for JPype1 (pyproject.toml) ... done Created wheel for JPype1: filename=JPype1-1.5.0-cp39-cp39-os390_28_00_3931.whl size=1559027 sha256=5ed06b0fa8f957b1a8e0f9a2f3ef6d26fa8578e0120a530c9fc6aaa22ab13cf3 Stored in directory: /u/d020216/.cache/pip/wheels/25/db/5e/17e495c39f5d36df1078c226f875a890ccbf1ead1b243524d8 Successfully built JPype1 Installing collected packages: JPype1 Successfully installed JPype1-1.5.0

pip3 list Package Version

... Jinja2 3.1.2 JPype1 1.5.0 ... wheel 0.40.0 ...

The version of JPype1 was 1.4.1, but pulling in new got us to 1.5.0

Thanks!

Thrameos commented 8 months ago

I think we need to find some way of seeing what flags were used to compile on 3.9 and 3.12. I suspect that the issue is that Python changed their compile flags. I recommend calling “python setup.py build_ext” on both versions and comparing the first build line for jp_array.o. Given that both use math.h and only one compiles the broken flag has to show up there.

Ie on my system it shows…

x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -Wall -fstack-protector-strong -Wformat -Werror=format-security -fwrapv -fPIC -Inative/common/include -Inative/python/include -Inative/embedded/include -I/usr/lib/jvm/java-17-openjdk-amd64/include -I/usr/lib/jvm/java-17-openjdk-amd64/include/linux -I/usr/include/python3.10 -I/usr/include/python3.10 -c native/common/jp_array.cpp -o build/temp.linux-x86_64-cpython-310/native/common/jp_array.o -g0 -std=c++11 -O2

(The important bit for my compiler is -std=c++11)

pfgdrupalas commented 8 months ago

Hello,

I tried to execute the command but received the message “No such file or directory”.

I did a find to see where the file might reside and found: ./pyz/lib/python3.9/test/libregrtest/setup.py ./pyz/lib/python3.9/site-packages/numpy/testing/setup.py ./pyz/lib/python3.9/site-packages/numpy/random/_examples/cython/setup.py ./pyz/lib/python3.9/site-packages/numpy/random/setup.py ./pyz/lib/python3.9/site-packages/numpy/polynomial/setup.py ./pyz/lib/python3.9/site-packages/numpy/matrixlib/setup.py ./pyz/lib/python3.9/site-packages/numpy/ma/setup.py ./pyz/lib/python3.9/site-packages/numpy/linalg/setup.py ./pyz/lib/python3.9/site-packages/numpy/lib/setup.py ./pyz/lib/python3.9/site-packages/numpy/fft/setup.py ./pyz/lib/python3.9/site-packages/numpy/f2py/setup.py ./pyz/lib/python3.9/site-packages/numpy/distutils/setup.py ./pyz/lib/python3.9/site-packages/numpy/core/setup.py ./pyz/lib/python3.9/site-packages/numpy/compat/setup.py ./pyz/lib/python3.9/site-packages/numpy/setup.py

Where the only places.

We are running on z/OS, so we do the base python install by using an SMPE package for the core. Then we install the needed modules into it using pip.

So, do you think any of these would work?

I did try the numpy/core/setup.py one, but took and error indicating that there was no numpy module of build_ext.

Thanks!

Classification: Internal Use From: Karl Nelson @.> Sent: Monday, March 18, 2024 10:41 AM To: jpype-project/jpype @.> Cc: Drew, AJ @.>; Author @.> Subject: [External] Re: [jpype-project/jpype] Python 3.12 support with the xlclang compiler (Issue #1177)

CAUTION: This email originated from a source outside the Principal network. Do not click links or open attachments unless you recognize the sender and know the content is safe. The original sender of this email is @.**@.>. I think we need to find some way of seeing what flags were used to compile on 3.9 and 3.12. I suspect that the issue is that Python changed their compile flags. I recommend calling “python setup.py build_ext” on both versions and comparing the first build line for jp_array.o. Given that both use math.h and only one compiles the broken flag has to show up there.

Ie on my system it shows…

x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -Wall -fstack-protector-strong -Wformat -Werror=format-security -fwrapv -fPIC -Inative/common/include -Inative/python/include -Inative/embedded/include -I/usr/lib/jvm/java-17-openjdk-amd64/include -I/usr/lib/jvm/java-17-openjdk-amd64/include/linux -I/usr/include/python3.10 -I/usr/include/python3.10 -c native/common/jp_array.cpp -o build/temp.linux-x86_64-cpython-310/native/common/jp_array.o -g0 -std=c++11 -O2

(The important bit for my compiler is -std=c++11)

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/jpype-project/jpype/issues/1177*issuecomment-2004257555__;Iw!!Hd8rfk_wYxoe!KQmLv4CAaWXsaEBWGuwfk8yd0jyvblvrC0jVLtXBIYa3johNork6K_0K_fhnfpbFp4s8TsTB6F0csK4TTJW2eQeXkg$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ADHJPDE4NUXN2EZFSFQTQYDYY4DJHAVCNFSM6AAAAABEWU67DSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBUGI2TONJVGU__;!!Hd8rfk_wYxoe!KQmLv4CAaWXsaEBWGuwfk8yd0jyvblvrC0jVLtXBIYa3johNork6K_0K_fhnfpbFp4s8TsTB6F0csK4TTJXsTT1M_g$. You are receiving this because you authored the thread.Message ID: @.**@.>>

-----Message Disclaimer----- This email is intended for the recipient specified in this message. It may contain information that is privileged, confidential and exempt from disclosure under applicable laws. If you are not the intended recipient, sharing or copying this communication is prohibited. If received in error, please send a note to @.*** and delete the original message and any attachments. Principal Financial Group or any of its member companies may retain emails as required by law or regulation. This email is not an electronic signature, unless a specific statement to the contrary is included in this message.

To unsubscribe from marketing communications, go to www.principal.com/do-not-contact-form.

marscher commented 8 months ago

you need to clone this repository (or dowload the code as archive and unpack) to run the specified command.

Thrameos commented 8 months ago

You will need to download the JPype1 package, unpack it and then execute the command in there.

pfgdrupalas commented 8 months ago

For Python 3.9, that line is: /usr/lpp/cbclib/xlclang/bin/xlclang++ -fno-strict-aliasing -DNDEBUG -qarch=10 -q64 -Wc,DLL -D_XOPEN_SOURCE_EXTENDED -D_UNIX03_THREADS -D_POSIX_THREADS -D_OPEN_SYS_FILE_EXT -qexportall -qascii -qstrict -qnocsect -Wa,asa,goff -Wa,xplink -qgonumber -qenum=int -Inative/common/include -Inative/python/include -Inative/embedded/include -I/usr/lpp/java/J11/J11.0_64/include -I/usr/lpp/java/J11/J11.0_64/include/zos -I/MVST/products/python/v3r9/pyz/include/python3.9 -c native/common/jp_array.cpp -o build/temp.os390-28.00-3931-3.9/native/common/jp_array.o

And for Python 3.12 the line is: /usr/lpp/cbclib/xlclang/bin/xlclang -DNDEBUG -D_XOPEN_SOURCE_EXTENDED -D_UNIX03_THREADS -D_POSIX_THREADS -D_OPEN_SYS_FILE_EXT -Inative/common/include -Inative/python/include -Inative/embedded/include -I/usr/lpp/java/J11/J11.0_64/include -I/usr/lpp/java/J11/J11.0_64/include/zos -I/MVST/products/python/v3r12/pyz/include/python3.12 -c native/common/jp_array.cpp -o build/temp.os390-28.00-3931-cpython-312/native/common/jp_array.o

Thanks!

Thrameos commented 8 months ago

This is definitely a problem with your Python site installation. It is calling the C compiler to build a C++ project on 3.12. That would definitely cause the error that you reported. Assuming that our setup scripts are set to call the C++ compiler I am not sure what we can do in this case as it appears to be a site issue. Do you know the source of this Python 3.12 build? If it is a published source then everyone with that system is likely to get it.

@.***

pfgdrupalas commented 8 months ago

The source for both our Python 3.9 and 3.12 is from IBM.

So, I don’t normally deal with any C code and am not sure what exactly I should report to IBM in the ticket, I am guessing I need to open.

Thoughts?

Thanks!

Classification: Internal Use From: Karl Nelson @.> Sent: Monday, March 18, 2024 12:00 PM To: jpype-project/jpype @.> Cc: Drew, AJ @.>; Author @.> Subject: [External] Re: [jpype-project/jpype] Python 3.12 support with the xlclang compiler (Issue #1177)

CAUTION: This email originated from a source outside the Principal network. Do not click links or open attachments unless you recognize the sender and know the content is safe. The original sender of this email is @.**@.>. This is definitely a problem with your Python site installation. It is calling the C compiler to build a C++ project on 3.12. That would definitely cause the error that you reported. Assuming that our setup scripts are set to call the C++ compiler I am not sure what we can do in this case as it appears to be a site issue. Do you know the source of this Python 3.12 build? If it is a published source then everyone with that system is likely to get it.

@.<mailto:@.>

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/jpype-project/jpype/issues/1177*issuecomment-2004459990__;Iw!!Hd8rfk_wYxoe!O_djj3n6kTxwpS9pxo6vvPj5t69wrTYbw7z_w0KxSvzW2mQXuEYtT91rT0hEzRhbVfYxKFz92G6aWPwJ-ulkNARmkg$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ADHJPDAD5ZLQAE3RA6LXG2DYY4MRDAVCNFSM6AAAAABEWU67DSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBUGQ2TSOJZGA__;!!Hd8rfk_wYxoe!O_djj3n6kTxwpS9pxo6vvPj5t69wrTYbw7z_w0KxSvzW2mQXuEYtT91rT0hEzRhbVfYxKFz92G6aWPwJ-uk8c_ofvA$. You are receiving this because you authored the thread.Message ID: @.**@.>>

-----Message Disclaimer----- This email is intended for the recipient specified in this message. It may contain information that is privileged, confidential and exempt from disclosure under applicable laws. If you are not the intended recipient, sharing or copying this communication is prohibited. If received in error, please send a note to @.*** and delete the original message and any attachments. Principal Financial Group or any of its member companies may retain emails as required by law or regulation. This email is not an electronic signature, unless a specific statement to the contrary is included in this message.

To unsubscribe from marketing communications, go to www.principal.com/do-not-contact-form.

Thrameos commented 8 months ago

I believe the issue is there was a misconfiguration in the Python build. In setuptools there is a file that gives important parameters for the build system. Unfortunately, due to my work schedule I don’t have time to look up the specifics of the Python system enough to provide useful input. I will try to wrap back to this, but it likely won’t be until Sunday.

Thrameos commented 8 months ago

Here is the code to check your compiler settings

from setuptools._distutils import sysconfig

print({p:v for p,v in sysconfig.get_config_vars().items() if isinstance(v,str) and "CC" in p})
print({p:v for p,v in sysconfig.get_config_vars().items() if isinstance(v,str) and "CXX" in p})

This should print out your C and C++ compiler settings.

For my system I get something like this

{'CC': 'x86_64-linux-gnu-gcc', 'CCSHARED': '-fPIC', 'LINKCC': 'x86_64-linux-gnu-gcc', 'MAINCC': 'x86_64-linux-gnu-gcc'}
{'CXX': 'x86_64-linux-gnu-g++', 'LDCXXSHARED': 'x86_64-linux-gnu-g++ -shared -Wl,-O1 -Wl,-Bsymbolic-functions'}

You need to report to your system admin if the CXX and LDCXXSHARED are reporting C rather than C++ versions of the compiler.

I will close this issue as this isn't a problem with JPype but rather a site issue with a build of Python, so it doesn't really apply to anyone else. You can continue to post to this thread and I will respond.