Cinema4d 2024.0.0

[alexdjulin]

Hey. Cinema4d 2024 is out and I run into the usual PySide2 problem. Are you planning to release a wheel for 2024 any time soon? Thanks for your work and for sharing it with us.

[danbradham]

Hey @alexdjulin,

As far as I know PySide2 5.15.2 is not supported for Python 3.11+ and I see that Cinema 4D 2024 is using Python 3.11.4. I'm actively investigating that but in my initial attempts to build it I was getting several compilation errors that I doubt I'll be able to resolve.

One alternative would be to upgrade to PySide6. You should probably be able to install those wheels directly with pip.

From a cmd prompt...

cd "C:\Program Files\Maxon Cinema 4D 2024\resource\modules\python\libs\win64"
python -m ensurepip
python -m pip install PySide6 --target="%APPDATA%\Maxon\python\python311\libs"

From powershell...

cd "C:\Program Files\Maxon Cinema 4D 2024\resource\modules\python\libs\win64"
.\python.exe -m ensurepip
.\python.exe -m pip install PySide6 --target="$env:APPDATA\Maxon\python\python311\libs"

[danbradham]

So there is a compatibility issue with the prebuilt PySide6 wheels as well, the above will not work after all. I'm still investigating building PySide2 but may end up building PySide6 in the end we'll see.

[alexdjulin]

Thank you for your help! I will keep an eye on it.

[robin-vagner]

Hello, just chiming in to say that this is something that we would be interested in as well!

Is there anything we can do to help ?

[danbradham]

I can give you guys a status update of where I am at in the process. I have attempted to build PySide 6.5.2 and 6.5.3 using the Python included with Cinema 4D. That's Python 3.11.4 built against Microsoft Visual Studio 2019. Both of those builds work if I use the python interpreter included with Cinema 4D, but not c4dpy and not within Cinema 4D =(. I'm not sure why this is the case as this was my typical approach for building PySide2(5.15.2).

I can outline the steps I'm taking if any of you would like to try to build yourself. If you have experience building complex libraries like PySide6 any ideas or help would be greatly appreciated.

Here are my steps - as detailed as I can be. I'll work in D:\build_pyside6. I personally work in powershell as well, so assume any commands below are run from powershell.

1. Install Microsoft Visual Studio 2019 community edition including c++ development tools and the windows SDK. Here's a snapshot of my installation and a download link.
2. Install Qt included the version you're attempting to build (probably 6.5.3 at this point) as well as the included OpenSSL. Here's another snapshots and a download link.
3. Extract libclang binaries from Qt's repository (libclang-release_140-based-windows-vs2019_64.7z) to D:\build_pyside6\libclang.
4. Copy and modify Cinema 4D's Python to a custom location so we can use it for the build process and installing dependencies. This is in lieu of using a virtualenv since neither c4dpy or the included python can be used to create a virtualenv (they are missing some features).

   > cd D:\build_pyside6
   > cp -Recurse "C:\Program Files\Maxon Cinema 4D 2024\resource\modules\python\libs\win64" python
   > cd .\python\
   > mv libs_win64 libs
   > cp -Recurse .\include\python3.11 .\include_flat
   > rm -Recurse -Force .\include
   > mv .\include_flat .\include

5. Clone the pyside-setup repository and checkout the version tag you are attempting to build against (v6.5.3).

   > cd D:\build_pyside6
   > git clone --recursive https://code.qt.io/pyside/pyside-setup
   > cd .\pyside-setup\
   > git checkout v6.5.3

6. Install pyside-setup's python dependencies.

   > cd D:\build_pyside6\pyside-setup
   > & "D:\build_pyside6\python\python.exe" -m ensurepip
   > & "D:\build_pyside6\python\python.exe" -m pip install -r requirements.txt

7. Create this build script D:\build_pyside6\pyside-setup\build.bat.

   
   echo
   echo [DEBUG] Loading MSVC 2019 Build Environment...
   call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" amd64

echo echo [DEBUG] Setting PATH... set OPENSSLBIN=C:\Qt\Tools\OpenSSLv3\Win_x64\bin set QTROOT=C:\Qt\6.5.3\msvc2019_64 set QTBIN=%QTROOT%\bin set PYTHONBIN=D:\build_pyside6\python set LLVM_INSTALL_DIR=D:\build_pyside6\libclang set LLVM_BIN=%LLVM_INSTALL_DIR%\bin set PATH=%QTBIN%;%PYTHONBIN%;%LLVM_BIN%;%PATH%

echo echo [DEBUG] Building PySide 6 python setup.py build --qt-target-path=%QTROOT% --qtpaths=%QTBIN%\qtpaths6.exe --openssl=%OPENSSLBIN%

echo echo [DEBUG] Building Wheels python.exe create_wheels.py

7. Execute `build.bat`.

.\build.bat

At this point...hopefully you will be off to the races. After the build process completes you should have new PySide6 wheels in the D:\build_pyside6\dist_new directory. You can try installing them and seeing if they work.

> cd D:\build_pyside6\pyside-setup\dist_new
> & "D:\build_pyside6\python\python.exe" -m pip install PySide6 --no-index --find-links="." --target="$env:APPDATA\Maxon\python\python311\libs"

To reiterate, in my attempts, these wheels will work when used via C:\Program Files\Maxon Cinema 4D 2024\resource\modules\python\libs\win64\python.exe but not with c4dpy or within Cinema 4D. However, this is where I am, and if you have the time and proclivity to solve build issues, it's a great starting point. I will continue working on this build when I have availability but that's less and less these days.

[robin-vagner]

Thanks, I actually have no experience whatsoever building libraries, but it can't hurt to try!

I'll let you know if I find anything.

[gr4ph0s]

Hi I'm Maxime the responsible Python developer at Maxon for Cinema 4D.

First of all, thanks a lot for your work and all your inputs. Can you try to run c4dpy like so c4dpy -s and tell me if it fix the issue, cause by default since 2024.0 Cinema 4D have the Py_NoUserSiteDirectory flag enabled. But maybe in this case this is preventing some dll discovery. If it does not solve it, may I ask what is the exact stacktrace?

Cheers, Maxime.

[danbradham]

Thank you so much for your attention @gr4ph0s . I was preparing to reach out to Maxon support directly. I'll try the -s flag tomorrow once I'm in front of my build machine and report back.

[danbradham]

Good morning. @gr4ph0s the -s flag didn't make a difference in my tests. Here is the full stacktrace from c4dpy.

❯ & 'C:\Program Files\Maxon Cinema 4D 2024\c4dpy.exe' -s
Welcome to the world of C4D and Python 3.11.4 (pipelines/893915622:c23be21, Jun  8 2023, 15:31:26) [MSC v.1929 64 bit (AMD64)] (c) 2023

>>> from PySide6 import QtWidgets
PySide6/__init__.py: Unable to import Shiboken from C:\Program Files\Maxon Cinema 4D 2024\resource\modules\python\libs\python311, C:\Users\danie\AppData\Roaming\Maxon\Maxon Cinema 4D 2024_A5DBFF93_p\python311, C:\Program Files\Maxon Cinema 4D 2024\resource\modules\python\libs\win64\lib, C:\Program Files\Maxon Cinema 4D 2024\resource\modules\python\libs\win64\dlls, C:\Users\danie, C:\Users\danie\AppData\Roaming\Python\Python311\site-packages, C:\Users\danie\AppData\Roaming\Maxon\python\python311\libs, C:\Users\danie\AppData\Roaming\Maxon\Maxon Cinema 4D 2024_A5DBFF93_p\python311\libs, C:\Program Files\Maxon Cinema 4D 2024\resource\modules\python\libs\win64\lib\site-packages
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\danie\AppData\Roaming\Maxon\python\python311\libs\PySide6\__init__.py", line 124, in <module>
    _setupQtDirectories()
  File "C:\Users\danie\AppData\Roaming\Maxon\python\python311\libs\PySide6\__init__.py", line 64, in _setupQtDirectories
    from shiboken6 import Shiboken
  File "C:\Users\danie\AppData\Roaming\Maxon\python\python311\libs\shiboken6\__init__.py", line 27, in <module>
    from shiboken6.Shiboken import *
ImportError: DLL load failed while importing Shiboken: The specified module could not be found.

I ran procmon from sysinternals to try to find more information about specifically what DLL failed to load here and it seems that when shiboken6.abi3.dll is loading it fails to find the required python3.dll.

Comparing this to a successful import using the python executable in resource\modules\python, the python3.dll is found in the C:\Program Files\Maxon Cinema 4D 2024\resource\modules\python\libs\win64\dlls. Here's my test command for clarity:

❯ $env:PYTHONPATH="$env:APPDATA\Maxon\python\python311\libs"
❯ & "C:\Program Files\Maxon Cinema 4D 2024\resource\modules\python\libs\win64\python.exe"
Python 3.11.4 (pipelines/893915622:c23be21, Jun  8 2023, 15:31:26) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from PySide6 import QtWidgets
>>> exit()

Procmon showing python3.dll loading.

Good details! Now we should be able to add that directory to python's dll lookup.

❯ & 'C:\Program Files\Maxon Cinema 4D 2024\c4dpy.exe'
Welcome to the world of C4D and Python 3.11.4 (pipelines/893915622:c23be21, Jun  8 2023, 15:31:26) [MSC v.1929 64 bit (AMD64)] (c) 2023

>>> import os
>>> os.add_dll_directory("C:/Program Files/Maxon Cinema 4D 2024/resource/modules/python/libs/win64/dlls")
<AddedDllDirectory('C:/Program Files/Maxon Cinema 4D 2024/resource/modules/python/libs/win64/dlls')>
>>> from PySide6 import QtWidgets
>>> app = QtWidgets.QApplication([])
>>> import sys
>>> sys.exit(QtWidgets.QDialog().exec())

Wow, success!

This does lead me to the question of how shiboken6.abi3.dll is looking up python3.dll. It seems to be doing the lookup solely based on the python executables location, and therefore is able to locate it when running an in-place python executable versus a wrapper like c4dpy.exe. I'm unsure of whether or not this is a peculiarity to how shiboken6 is loading python3.dll or if it's an issue with c4dpy.

In a twist, it appears the ABI compatible wheels from PYPI work fine in Cinema 4D after adding the appropriate DLL directory. I misdiagnosed the original DLL load error because I had become accustomed to doing custom builds and the stacktrace was identical and generic. Here's a snapshot from Cinema 4D after I've installed the prebuilt wheels from PYPI.

@gr4ph0s do you think this is something that could be addressed in a future release of Cinema 4D? I imagine this issue could be affecting other ABI compatible libraries from PYPI.

[gr4ph0s]

I can't give you any guarantee, but at least have the correct DLL search path defined is something I will make sure to have. That's pretty weird cause except the python version upgrade and Py_NoUserSiteDirectory change I did not made any others change in this area. So indeed knowing how shiboken6.abi3.dll is looking up python3.dll would be interesting.

Regarding compiling python with the same version that is shipped on the official python version, this is something I will look into but it may not be as easy as it sound on our side sadly.

Anyway thanks a lot for your efforts and glad that it works !

[alexdjulin]

Hey guys. I'm not sure if the initial problem has been solved? @danbradham do you think you will release the updated whees for c4d R2024 any time soon? Thank you and @gr4ph0s for your work.

[danbradham]

Hey @alexdjulin

Yeah, it turns out the PySide6 wheels from pypi do work with a small tweak from Python inside Cinema 4D / c4dpy. So custom wheels will not need to be provided via this repository.

As described in my long debugging post above, all you need to do is add a missing directory to Python's DLL search path prior to importing PySide6 using os.add_dll_directory.

This will hopefully be unnecessary in a future release once @gr4ph0s patches the missing DLL directory in C4D.

I will leave this issue open for now so people can find it easily. If you need more specific instructions let me know.

[alexdjulin]

Amazing, it seems to be working on my side too. Here is what I did, starting from a fresh install of Cinema4d 2024.1.0. Note: My install folder of C4D is not the default one.

First I had to install pip: python.exe -m ensurepip --upgrade

C:\Programs\Cinema4D\Cinema4d_2024.1.0\resource\modules\python\libs\win64>C:\Programs\Cinema4D\Cinema4d_2024.1.0\resource\modules\python\libs\win64\python.exe -m ensurepip --upgrade
WARNING: Ignoring invalid distribution ~ip (C:\Programs\Cinema4D\Cinema4d_2024.1.0\resource\modules\python\libs\win64\Lib\site-packages)
Looking in links: c:\Users\ALEXAN~1.DJ\AppData\Local\Temp\tmpt2uqtc2v
Requirement already satisfied: setuptools in c:\programs\cinema4d\cinema4d_2024.1.0\resource\modules\python\libs\win64\lib\site-packages (65.5.0)
Processing c:\users\alexandre.dj\appdata\local\temp\tmpt2uqtc2v\pip-23.1.2-py3-none-any.whl
WARNING: Ignoring invalid distribution ~ip (C:\Programs\Cinema4D\Cinema4d_2024.1.0\resource\modules\python\libs\win64\Lib\site-packages)
Installing collected packages: pip
  WARNING: The scripts pip3.11.exe and pip3.exe are installed in 'C:\Programs\Cinema4D\Cinema4d_2024.1.0\resource\modules\python\libs\win64\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-23.1.2

Then I installed PySide6 using the pypi wheel as you said: pip3 install PySide6

C:\Programs\Cinema4D\Cinema4d_2024.1.0\resource\modules\python\libs\win64\Scripts>pip3 install PySide6
Collecting PySide6
  Downloading PySide6-6.6.0-cp38-abi3-win_amd64.whl (7.2 kB)
Collecting shiboken6==6.6.0 (from PySide6)
  Downloading shiboken6-6.6.0-cp38-abi3-win_amd64.whl (1.1 MB)
     ---------------------------------------- 1.1/1.1 MB 16.8 MB/s eta 0:00:00
Collecting PySide6-Essentials==6.6.0 (from PySide6)
  Downloading PySide6_Essentials-6.6.0-cp38-abi3-win_amd64.whl (76.7 MB)
     ---------------------------------------- 76.7/76.7 MB 50.4 MB/s eta 0:00:00
Collecting PySide6-Addons==6.6.0 (from PySide6)
  Downloading PySide6_Addons-6.6.0-cp38-abi3-win_amd64.whl (110.3 MB)
     ---------------------------------------- 110.3/110.3 MB 38.5 MB/s eta 0:00:00
Installing collected packages: shiboken6, PySide6-Essentials, PySide6-Addons, PySide6
  WARNING: The scripts pyside6-assistant.exe, pyside6-deploy.exe, pyside6-designer.exe, pyside6-genpyi.exe, pyside6-linguist.exe, pyside6-lrelease.exe, pyside6-lupdate.exe, pyside6-metaobjectdump.exe, pyside6-project.exe, pyside6-qml.exe, pyside6-qmlcachegen.exe, pyside6-qmlformat.exe, pyside6-qmlimportscanner.exe, pyside6-qmllint.exe, pyside6-qmlls.exe, pyside6-qmltyperegistrar.exe, pyside6-qtpy2cpp.exe, pyside6-rcc.exe and pyside6-uic.exe are installed in 'C:\Programs\Cinema4D\Cinema4d_2024.1.0\resource\modules\python\libs\win64\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed PySide6-6.6.0 PySide6-Addons-6.6.0 PySide6-Essentials-6.6.0 shiboken6-6.6.0

Finally I started c4d 2024.1.0 and added the missing dll path like you did. It fixed the problem.

Thanks again for your help!

[gr4ph0s]

Hi just yo let you know that with Cinema 2024.2, I fixed the issue regarding the dll path not being defined by defualt. So running python.exe -m ensurepip --upgrade and python.exe -m pip install PySide6 is enough to get it to works.

Cheers, Maxime.