edumorlom commented 9 months ago

Is your feature request related to a problem? Please describe. No

Describe the solution you'd like Support for python 3.12 which is officially out.

bimalkjha commented 9 months ago

Internal: Epic https://jira.rocketsoftware.com/browse/DBC-14681 opened for this enhancement request. Thanks.

LarsMichelsen commented 6 months ago

Can you provide any insight when we can expect a release supporting 3.12?

bchoudhary6415 commented 6 months ago

@LarsMichelsen We have started working on support for python 3.12. The Internal: Epic https://jira.rocketsoftware.com/browse/DBC-14681 is in progress now. Will try to finish as soon as possible.

Thank you for your patience Balram

LarsMichelsen commented 5 months ago

@LarsMichelsen We have started working on support for python 3.12.

Great :)

Will try to finish as soon as possible.

When reading this, shall we think of days, weeks or months? Don't want to be pushy, just asking for better understanding.

bchoudhary6415 commented 5 months ago

@LarsMichelsen We have started working on support for python 3.12.

Great :)

Will try to finish as soon as possible.

When reading this, shall we think of days, weeks or months? Don't want to be pushy, just asking for better understanding.

It may take some weeks, but will try to deliver as soon as possible.

gngrossi commented 5 months ago

Installed Python 3.12.0 on z/OS 2.5. Attached is the output from the pip install ibm-db --verbose

python312_ibm-db.txt

bchoudhary6415 commented 5 months ago

@gngrossi Yes, the error is because of 'distutils' module, the support for this module is removed from python 3.12.x

yadukrishnanperumpillil commented 5 months ago

@bchoudhary6415 any workaround for this other than running on 3.11?

bchoudhary6415 commented 5 months ago

@bchoudhary6415 any workaround for this other than running on 3.11?

@yadukrishnanperumpillil No, as legacy Unicode C APIs have been removed from python 3.12 onwards, Please refer the link - https://docs.python.org/3.12/whatsnew/3.12.html#id10 We have already created the Internal: Epic https://jira.rocketsoftware.com/browse/DBC-14681 to support for python 3.12. Epic is in progress.

Thank you

bimalkjha commented 5 months ago

@gngrossi Current release of ibm_db i.e. v3.2.1 do not support Python 3.12. Hence, error is expected. Next release of ibm_db should support 3.12, till then you can use Python 3.11. Thanks.

gngrossi commented 5 months ago

Will the new release of ibm-db be able to be compiled with IBM C/C++ for Open Enterprise Languages on z/OS 2.0.0? Since there is a need to let the pip installer know where the Python include/library directories are, is there an environment variable like XL_CONFIG that can be specified? (used it for xlclang.cfg).

Would one of these work? CLANG_CONFIG_FILE_USER_DIR CLANG_CONFIG_FILE_SYSTEM_DIR

clang.cfg #

-fzos-le-char-mode=ascii

-mzos-hlq-csslib=SYS1 -mzos-hlq-clang=SYS1

# -Wall -m64 -mrent -march=z15 -mzos-hlq-le=SYS1 -O2 -I/hewitt/zopentools/python-3.12/usr/lpp/IBM/cyp/v3r12/pyz/include/python3.12 -L/hewitt/zopentools/python-3.12/usr/lpp/IBM/cyp/v3r12/pyz/lib

thanks

shehenshah14 commented 5 months ago

@bimalkjha Is there any tentative date or ETA? Not pushing, just curious so we can plan it out.

frbelotto commented 4 months ago

Thanks for your effort. I have the same issue here using 3.12 :(

bimalkjha commented 4 months ago

PR #913 opened to deliver fix for this issue. Thanks.

shehenshah14 commented 4 months ago

@bimalkjha Thank you very much for expediting this. I see PR is merged now, when do we expect the release artifact out.

bimalkjha commented 4 months ago

Next week, by EOM. Thanks.

gngrossi commented 4 months ago

Will the new release of ibm-db be able to be compiled with IBM C/C++ for Open Enterprise Languages on z/OS 2.0.0? Since there is a need to let the pip installer know where the Python include/library directories are, is there an environment variable like XL_CONFIG that can be specified? (used it for xlclang.cfg).

When will the package be available on pypi.org?

thanks

bchoudhary6415 commented 4 months ago

Hello @gngrossi

You can create ". profile" file and specify the python path environment variables such as export PATH=$HOME/bin:/user/python_install/bin:$PATH export LIBPATH=$HOME/lib:/user/python_install/lib:$PATH

For more info regarding ibm_db Installation on z/OS please refer the link - https://github.com/ibmdb/python-ibmdb/blob/master/INSTALL.md#inszos

Thank you

edumorlom commented 4 months ago

@bimalkjha i see that the python3.12 support was merged to main branch but it has not been released to pypi yet.

https://pypi.org/project/ibm-db/

bimalkjha commented 4 months ago

@edumorlom Just released. Please install and verify. Thanks.

gngrossi commented 4 months ago

Do I include this in my install script? thanks

export SETUPTOOLS_USE_DISTUTILS=stdlib

bchoudhary6415 commented 4 months ago

Hello @gngrossi No, need to add "export SETUPTOOLS_USE_DISTUTILS=stdlib" in your install script as from python 3.12 onwards distutils package is removed - please refer the link - https://docs.python.org/3/whatsnew/3.12.html#distutils the setuptools project can be installed: it still provides distutils.

Thanks

gngrossi commented 4 months ago

@bchoudhary6415 Thanks, I will remove the export SETUPTOOLS_USE_DISTUTILS=stdlib from the script.

If I recall, I needed it for Python 3.11.5 but the doc says otherwise. https://docs.python.org/3/whatsnew/3.12.html#distutils

bchoudhary6415 commented 4 months ago

@gngrossi Actually distutils was in deprecation from Python 3.10, And from Python 3.12 onwards it's removed. That is why, we used setuptools in Python 3.12 support.

Thanks

gngrossi commented 4 months ago

Getting an error during the link. I need to use the xlclang.cfg so qarch=13,-qoptimize=2,-I and -L can be used. I pointed to it using export XL_CONFIG="/hewitt/zopentools/installs/xlclang.cfg" and ran the install from the directory.

python312_ibmdb322.txt

xlclang.cfg.txt

bchoudhary6415 commented 4 months ago

@gngrossi Are you able to install ibm_db without setting XL_CONFIG environment variable?

gngrossi commented 4 months ago

@bchoudhary6415 No, receiving the same error. The -L directory doesn't exist.

xlclang build/temp.os390-28.00-3931-cpython-312/ibm_db.o /u/@02858/tmp/pip-install-dg_gnhbu/ibm-db_a0b5efaf392342a0883dd48243edfd68/libdsnao64c.x -L. -L/usr/lpp/IBM/cyp/v3r12/pyz/lib -o build/lib.os390-28.00-3931-cpython-312/ibm_db.cpython-312.so FSUM3248 xlclang: Directory /usr/lpp/IBM/cyp/v3r12/pyz/lib specified by command option -L cannot be searched.

python312_ibmdb322_withoutXL_CONFIG.txt

bchoudhary6415 commented 4 months ago

@gngrossi I can see from the log file you shared the error you got is ERROR: Could not build wheels for ibm-db, which is required to install pyproject.toml-based projects

in case there is error in install due to pyproject.toml file try the following approach

To install the package ibm_db it is necessary at first to install the build dependency package - wheel:
pip3 install wheel
Install ibm_db with the pip flag --no-build-isolation:
pip3 install ibm_db --no-build-isolation

gngrossi commented 4 months ago

@bchoudhary6415 Is installing wheel required Python 3.12 and ibm-db? For Python 3.11.x, it wasn't.

Prior to trying ibm-db with Python 3.12, I was able to install these packages including upgrading pip. pip3 install boto3 pip3 install Flask pip3 install pymongo pip3 install redis pip3 install requests pip3 install zoautil_py-1.3.0.3-cp312-none-any.whl

bash-5.2$ pip3 list Package Version

blinker 1.7.0 boto3 1.34.22 botocore 1.34.22 certifi 2023.11.17 cffi 1.14.6 charset-normalizer 3.3.2 click 8.1.7 cryptography 3.3.2 dnspython 2.4.2 Flask 3.0.1 idna 3.6 itsdangerous 2.1.2 Jinja2 3.1.3 jmespath 1.0.1 MarkupSafe 2.1.3 pip 24.0 pycparser 2.20 pymongo 4.6.1 python-dateutil 2.8.2 redis 5.0.1 requests 2.31.0 s3transfer 0.10.0 six 1.16.0 urllib3 2.0.7 Werkzeug 3.0.1 zoautil-py 1.3.0.3

I still need to be able to use the xlclang.cfg file with the install. thanks

bchoudhary6415 commented 4 months ago

@gngrossi With the above steps, ibm_db is installed or not? Please refer this issue - https://github.com/ibmdb/python-ibmdb/issues/707 for more info With lower python version such as 3.11.x Are you able to install ibm_db with export XL_CONFIG="/hewitt/zopentools/installs/xlclang.cfg ? I tried creating xlclang.cfg file and setting XL_CONFIG variable to that file and I'm able to install ibm_db Please check the attached text file xlclang.cfg.txt Also before installing ibm_db refer this link https://github.com/ibmdb/python-ibmdb/blob/master/INSTALL.md#inszos and follow the steps

Thanks

gngrossi commented 3 months ago

@bchoudhary6415 For 3.12, without using the XL_CONFIG, ibm-db was not installed. With 3.11, I was able to install-db using export XL_CONFIG="$HOME/c/xlclang.cfg" and specifying the -I and -L directories for python 311.

gngrossi commented 3 months ago

@bchoudhary6415 Attached are logs from running after installing wheel 0.42.0

pip3 install ibm-db --no-build-isolation --verbose python312_ibm-db_no-build-isolation_3-11-2024.txt

pip3 install ibm-db --verbose python312_ibm-db_3-11-2024.txt

I referenced the install.md and I'm not seeing where I use an xlclang.cfg file. With the cfg file, besides supplying the Python include and lib directories, I'd like to optimize for running on a z15.

thanks

bchoudhary6415 commented 3 months ago

@gngrossi I see the below error xlclang build/temp.os390-28.00-3931-cpython-312/ibm_db.o /u/@02858/tmp/pip-install-42t3f0qd/ibm-db_c5da6aa13f9344e0a79c2620290f15fe/libdsnao64c.x -L. -L/usr/lpp/IBM/cyp/v3r12/pyz/lib -o build/lib.os390-28.00-3931-cpython-312/ibm_db.cpython-312.so FSUM3248 xlclang: Directory /usr/lpp/IBM/cyp/v3r12/pyz/lib specified by command option -L cannot be searched. error: command '/bin/xlclang' failed with exit code 40 error: subprocess-exited-with-error

Here the error is fsum3248 - please refer the link - https://www.ibm.com/docs/en/zos/3.1.0?topic=fsum-fsum3248 Please ensure that the Directory can be searched. Can you check that directory path is correct and have permission?

Thanks

gngrossi commented 3 months ago

I installed Python from the pax file into /hewitt/zopentools/python312/usr/lpp/IBM/cyp/v3r12/pyz

The xlclang.cfg contains the include and lib directories along with -qarch=13,-qoptimize=2 -I/hewitt/zopentools/python-3.12/usr/lpp/IBM/cyp/v3r12/pyz/include/python3.12,-L/hewitt/zopentools/python-3.12/usr/lpp/IBM/cyp/v3r12/pyz/lib

Here is my Python sourcing profile.

#

profile.python

# export PYZ_HOME="/hewitt/zopentools/python312/usr/lpp/IBM/cyp/v3r12/pyz" export PATH=${PYZ_HOME}/bin:$PATH export LIBPATH=${PYZ_HOME}/lib:$LIBPATH export _BPXK_AUTOCVT=ON export _CEE_RUNOPTS="FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)" export _TAG_REDIR_IN=txt export _TAG_REDIR_OUT=txt export _TAG_REDIR_ERR=txt

Recommended Configuration for IBM z/OS XL C/C++

export _CC_CCMODE=1 export _CXX_CCMODE=1 export _C89_CCMODE=1 export _CC_EXTRA_ARGS=1 export _CXX_EXTRA_ARGS=1 export _C89_EXTRA_ARGS=1

Build packages with setuptools

export CC=xlclang export CXX=xlclang export LDSHARED=xlclang #

gngrossi commented 3 months ago

Here's the portion of the xlclang.cfg file specifying options for -I and -L with directories that should be used. The compile step looks to be using my -I specified directory, how? Others options from the xlclang.cfg file don't appear as being used (-qarch=13,-qoptimize=2). The link step is missing my -L directory as it's defaulting to IBM's which doesn't exist.

Should specifying the environment variable for the location of xlclang.cfg allow /bin/xlclang to use the -F parameter?

Clang C compiler clang: use = DEFLT
Clang C++ compiler clang++: use = DEFLT options = -D_XOPEN_SOURCE=600,-D__static_assert=static_assert,-Wno-parentheses,-Wno-unused-value
common definitions DEFLT: cppcomp = /usr/lpp/cbclib/xlclang/exe/clcdrvr ccomp = /usr/lpp/cbclib/xlclang/exe/clcdrvr as = /bin/c89 ld_c = /bin/c89 ld_cpp = /bin/cxx xlC = /usr/lpp/cbclib/xlclang/bin/xlclang xlCcopt = -D_XOPEN_SOURCE sysobj = sys1.sceeobj:sys1.sceecpp syslib_x = sys1.sceebnd2:sys1.csslib exportlist_c_x = sys1.sceelib(celhs003,celhs001) exportlist_cpp_x = sys1.sceelib(celhs003,celhs001,celhscpp) exportlist_c_64 = sys1.sceelib(celqs003) exportlist_cpp_64 = sys1.sceelib(celqs003,celqscpp,cxxrt64) cinc = -isystem/usr/include/le cppinc = -isystem/usr/include/c++ options = -D_UNIX03_WITHDRAWN,-L/usr/lpp/cbclib/lib,-qarch=13,-qoptimize=2,-I/hewitt/zopentools/python-3.12/usr/lpp/IBM/cyp/v3r12/pyz/include/python3.12,-L/hewitt/zopentools/python-3.12/usr/lpp/IBM/cyp/v3r12/pyz/lib libraries = -libmcmp steplib = sys1.sclccmp

--

/bin/xlclang -DNDEBUG -O3 -D_XOPEN_SOURCE_EXTENDED -D_UNIX03_THREADS -D_POSIX_THREADS -D_OPEN_SYS_FILE_EXT -I/u/@02858/tmp/pip-install-a_0v9ss5/ibm-db_54993ad519164773833cf75400197a34/sdsnc.h -I/hewitt/zopentools/python312/usr/lpp/IBM/cyp/v3r12/pyz/include/python3.12 -c ibm_db.c -o build/temp.os390-28.00-3931-cpython-312/ibm_db.o

/bin/xlclang build/temp.os390-28.00-3931-cpython-312/ibm_db.o /u/@02858/tmp/pip-install-a_0v9ss5/ibm-db_54993ad519164773833cf75400197a34/libdsnao64c.x -L. -L/usr/lpp/IBM/cyp/v3r12/pyz/lib -o build/lib.os390-28.00-3931-cpython-312/ibm_db.cpython-312.so FSUM3248 xlclang: Directory /usr/lpp/IBM/cyp/v3r12/pyz/lib specified by command option -L cannot be searched. error: command '/bin/xlclang' failed with exit code 40

gngrossi commented 3 months ago

workaround until resolved

https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/steven-pitman1/2023/11/17/installing-pypi-packages-with-c-components-on-pyth

ibmdb / python-ibmdb

Python 3.12 support #890

Would one of these work? CLANG_CONFIG_FILE_USER_DIR CLANG_CONFIG_FILE_SYSTEM_DIR

-fzos-le-char-mode=ascii

-mzos-hlq-csslib=SYS1 -mzos-hlq-clang=SYS1

profile.python

Recommended Configuration for IBM z/OS XL C/C++

Build packages with setuptools

workaround until resolved