Open stone-cloud opened 1 month ago
Hi @stone-cloud,
Thank you for your interest! And thanks for the debug prints, that is useful.
I am working on bug templates and a debugging tool for pyqt-crom, but it will be released in the next version, so for now we have to debug "manually".
First of all, I can see that you are using:
Now for your bug that might not be a problem, but for full support, I would recommend you to use:
That's quite restrictive for now because I can't keep track of thousands of specs!
Anyway, regarding your bug:
sip-module didn't create an sdist.
tells me that sip-module
encountered a problemsip-module --sdist PyQt5.sip --abi-version 12
didn't create the source archive required. Information about this can be found in pyqtdeploy-3.3.0/pyqtdeploy/sysroot/plugins/SIP.py
.pyqtdeploy-3.3.0/pyqtdeploy/sysroot/plugins/SIP.py
tells us that SIP requires ['Python', 'Qt'] to be pre-installed. Which does not seem to be the case in your log.Here are some solutions to try out:
PyQt5-sip
module -> Can you run the command pip3 show PyQt5-sip
(so I can know which version you are using -> it should be 12.13.0
according to requirements.txt). sip-module --sdist PyQt5.sip --abi-version 12
(but I am not sure that the flag --verbose
will work) -> so observe the output in case errors are showing up[SIP]
section to the bottom in your sysroot.toml
and remove all the sysroot-android-64
generated folders to avoid caching issues and then rebuild the appHi@achille-martin, First of all, I test Python-3.8.19 and Python-3.10.12 in the same Ubuntu20.04 system, and they all have this Bug. But I still can't be sure if it's unconcerned. I tried all the solutions, but none of them worked. The detailed information is as follows:
There might be something odd with PyQt5-sip module -> Can you run the command pip3 show PyQt5-sip (so I can know which version you are using -> it should be 12.13.0 according to requirements.txt).
Name: PyQt5-sip Version: 12.13.0 Summary: The sip module support for PyQt5 Home-page: https://www.riverbankcomputing.com/software/sip/ Author: Riverbank Computing Limited Author-email: info@riverbankcomputing.com License: SIP Location: /home/gw/anaconda3/envs/torch19/lib/python3.8/site-packages Requires: Required-by: PyQt5
Name: PyQt5 Version: 5.15.10 Summary: Python bindings for the Qt cross platform application toolkit Home-page: https://www.riverbankcomputing.com/software/pyqt/ Author: Riverbank Computing Limited Author-email: info@riverbankcomputing.com License: GPL v3 Location: /home/gw/anaconda3/envs/torch19/lib/python3.8/site-packages Requires: PyQt5-Qt5, PyQt5-sip Required-by: pyqtdeploy
Package Version ---------------------- ------------ coloredlogs 15.0.1 dill 0.3.8 flatbuffers 24.3.25 humanfriendly 10.0 mkl-fft 1.3.8 mkl-random 1.2.4 mkl-service 2.4.0 mpmath 1.3.0 numpy 1.24.3 onnx 1.16.0 onnxconverter-common 1.13.0 onnxruntime-gpu 1.17.1 onnxruntime-tools 1.7.0 opencv-python-headless 4.9.0.80 packaging 23.2 pillow 10.3.0 pip 24.0 pipdeptree 2.18.1 ply 3.11 protobuf 3.20.3 psutil 5.9.8 py-cpuinfo 9.0.0 py3nvml 0.2.7 PyQt-builder 1.15.3 PyQt5 5.15.10 PyQt5-Qt5 5.15.2 PyQt5-sip 12.13.0 pyqtdeploy 3.3.0 setuptools 69.5.1 sip 6.7.12 sympy 1.12 toml 0.10.2 tomli 2.0.1 torch 1.9.0+cu111 torchaudio 0.9.0 torchvision 0.10.0+cu111 typing_extensions 4.11.0 wheel 0.43.0 xmltodict 0.13.0
> Try to run the failed command in verbose mode: sip-module --sdist PyQt5.sip --abi-version 12 (but I am not sure that the flag --verbose will work) -> so observe the output in case errors are showing up
I tried, but no information was printed. š«
> Move the [SIP] section to the bottom in your sysroot.toml and remove all the sysroot-android-64 generated folders to avoid caching issues and then rebuild the app
I tried, but it is not worked.
Ok, let's take it a step at a time then, since there are no debug prints associated to sip-module
.
1) Run sip-module --sdist PyQt5.sip --abi-version 12
anywhere
It should download in the current directory the archive for PyQt5-sip matching with your installed version.
So you should see a folder called PyQt5_sip-12.13.0.tar.gz
in your current directory.
If you don't see it -> there is an issue with your sip
or PyQt5-sip
and I would recommend you pip3 uninstall
and pip3 cache purge && pip3 install
them back again.
If you see it -> then there might be an issue with your pyqtdeploy
and we will need to add debug prints to $PYQT_CROM_DIR/venv/pyqt-crom-venv/lib/python3.10/site-packages/pyqtdeploy/sysroot/plugins/SIP.py
.
Amazingļ¼Base on your guidance, I found pyqt5_sip-12.13.0.tar.gz
in my current directory. So I debug $PYQT_CROM_DIR/venv/pyqt-crom-venv/lib/python3.10/site-packages/pyqtdeploy/sysroot/plugins/SIP.py
.
I found that glob
can't recognize the pyqt5_sip-12.13.0.tar.gz
due to a mismatch in the capitalization of the instructions archives = glob.glob(pattern)
(https://www.riverbankcomputing.com/hg/pyqtdeploy/file/df680b956780/pyqtdeploy/sysroot/plugins/SIP.py). The file name is lowercase and the instructions contain uppercase.
pyqt-crom/examples/demo/demo_project/sysroot.toml
. But it is not worked and print error. archives = glob.glob(pattern.lower())
and it's working!I'm not sure if this is a special case, such as the reason for the version number.
Glad you figured out the issue.
I suspect that the "lowercase" action is performed at some point in the sip
package for some reason.
To investigate a bit further, can you:
print(args.sip_modules)
(at the beginning of the try
block) in $PYQT_CROM_DIR/venv/pyqt-crom-venv/lib/python3.10/site-packages/sipbuild/module/main.py
sip-module --sdist PyQt5.sip --abi-version 12
['PyQt5.sip']
If you get ['PyQt5.sip']
, then the problem is likely later on in the code. Therefore, add debug prints in $PYQT_CROM_DIR/venv/pyqt-crom-venv/lib/python3.10/site-packages/sipbuild/module/module.pyto figure out when the lowercase operation happens (because it should not according to
sip==6.7.12`).
If you get ['pyqt5.sip']
, then the problem likely comes from the ArgumentParser
(so argparse
itself, which is built-in python3).
WARNINGS
1) It is not recommended to modify the site-packages
directory directly
This is because changes cannot be tracked and it creates confusion when you want to change versions.
The recommended way is to use pip in "edit" mode -> https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs
2) It is not recommended to modify the pyqtdeploy
package directly
This is because of the licence which forces you to disclose all the changes you made to the package before a release of your app. If you are happy doing that, then you are fine.
Hi @achille-martin ,
It's been a long time.
I tested the detailed output of '$PYQT_CROM_DIR/venv/pyqt-crom-venv/lib/python3.10/site-packages/sipbuild/module/module.py', and I found that when running the subprocess.run( [sys.executable, 'setup.py', '--quiet', 'sdist', '--dist-dir', '..'])
command, the python package name, 'pyqt5_sip-12.13.0.tar.gz' is changed to lowercase letters.
I asked ChatGPT, and it told me that it might be because some systems are case-insensitive. And it suggested using all lowercase letters. Maybe it needs to consider the system version for testing in the next version.
@stone-cloud thanks for your debugging efforts!
I am a bit surprised by the fact that subprocess.run( [sys.executable, 'setup.py', '--quiet', 'sdist', '--dist-dir', '..'])
modifies the module_name
, I thought it would not affect it as it is installing from setup.py
instructions already populated.
If you are happy to proceed, can you replace your $PYQT_CROM_DIR/venv/pyqt-crom-venv/lib/python3.10/site-packages/sipbuild/module/module.py
file with the one I have attached:
can you replace your $PYQT_CROM_DIR/venv/pyqt-crom-venv/lib/python3.10/site-packages/sipbuild/module/main.py
file with the one I have attached:
I have added a lot of debug prints and I have added verbose tag to suspicious commands - make sure to save your original files (in site-packages) before replacing their content (so you can revert back to your version, alternatively you can reinstall sip
via pip).
Then run sip-module --sdist PyQt5.sip --abi-version 12
.
Then please send me the terminal outputs so we can have a look.
The terminal outputs:
(androidDeploy) gw@gw-System-Product-Name:~/LYH/androidDepoly/QTgui$ sip-module --sdist PyQt5.sip --abi-version 12
main - ARGS: Namespace(abi_version='12', project=None, sdist=True, setup_cfg=None, sip_h=False, sip_rst=False, target_dir=None, sip_modules=['PyQt5.sip'])
module - SIP MODULE: PyQt5.sip
module - PROJECT: None
module - PROJECT 2: PyQt5_sip
_create_patches - SIP MODULE PARTS: ['PyQt5', 'sip']
_create_patches - SIP MODULE NAME: sip
_create_patches - SIP MODULE: PyQt5.sip
module - PATCHES: {'@SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@SIP_MODULE_PROJECT_NAME@': 'PyQt5_sip', '@SIP_MODULE_PACKAGE_NAME@': 'PyQt5', '@SIP_MODULE_VERSION@': '12.13.0', '@_SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@_SIP_MODULE_NAME@': 'sip', '@_SIP_MODULE_SHARED@': '1', '@_SIP_MODULE_ENTRY@': 'PyInit_sip', '@_SIP_MODULE_LEGACY@': '1', '@_SIP_VERSION@': '0x6070c', '@_SIP_VERSION_STR@': '6.7.12'}
module - SDIST DIR: PyQt5_sip-12.13.0
module - SDIST DIR 2: PyQt5_sip-12.13.0
_create_sdist - SDIST DIR: PyQt5_sip-12.13.0
_create_sdist - SDIST DIR 2: PyQt5_sip-12.13.0
_create_sdist - MODULE SOURCE DIR: /home/gw/anaconda3/envs/androidDeploy/lib/python3.10/site-packages/sipbuild/module/source/12
_create_sdist - SETUP CONFIG: None
_create_sdist - NAME FROM SOURCE DIR: MANIFEST.in
_create_sdist - NAME FROM SOURCE DIR: objmap.c
_create_sdist - NAME FROM SOURCE DIR: sip_array.h
_create_sdist - NAME FROM SOURCE DIR: siplib.c
_create_sdist - NAME FROM SOURCE DIR: sipint.h
_create_sdist - NAME FROM SOURCE DIR: LICENSE
_create_sdist - NAME FROM SOURCE DIR: threads.c
_create_sdist - NAME FROM SOURCE DIR: sip.rst.in
_create_sdist - NAME FROM SOURCE DIR: sip_array.c
_create_sdist - NAME FROM SOURCE DIR: setup.py.in
_install_source_file - module_source_dir: /home/gw/anaconda3/envs/androidDeploy/lib/python3.10/site-packages/sipbuild/module/source/12
name: setup.py
target_dir: PyQt5_sip-12.13.0
_install_file - NAME OUT: PyQt5_sip-12.13.0/setup.py
PATCHES: {'@SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@SIP_MODULE_PROJECT_NAME@': 'PyQt5_sip', '@SIP_MODULE_PACKAGE_NAME@': 'PyQt5', '@SIP_MODULE_VERSION@': '12.13.0', '@_SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@_SIP_MODULE_NAME@': 'sip', '@_SIP_MODULE_SHARED@': '1', '@_SIP_MODULE_ENTRY@': 'PyInit_sip', '@_SIP_MODULE_LEGACY@': '1', '@_SIP_VERSION@': '0x6070c', '@_SIP_VERSION_STR@': '6.7.12'}
_create_sdist - NAME FROM SOURCE DIR: bool.cpp
_create_sdist - NAME FROM SOURCE DIR: setup.cfg.in
_create_sdist - NAME FROM SOURCE DIR: LICENSE-GPL3
_create_sdist - NAME FROM SOURCE DIR: int_convertors.c
_create_sdist - NAME FROM SOURCE DIR: voidptr.c
_create_sdist - NAME FROM SOURCE DIR: apiversions.c
_create_sdist - NAME FROM SOURCE DIR: LICENSE-GPL2
_create_sdist - NAME FROM SOURCE DIR: sip.pyi
_create_sdist - NAME FROM SOURCE DIR: pyproject.toml
_create_sdist - NAME FROM SOURCE DIR: sip.h.in
_install_source_file - module_source_dir: /home/gw/anaconda3/envs/androidDeploy/lib/python3.10/site-packages/sipbuild/module/source/12
name: sip.h
target_dir: PyQt5_sip-12.13.0
_install_file - NAME OUT: PyQt5_sip-12.13.0/sip.h
PATCHES: {'@SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@SIP_MODULE_PROJECT_NAME@': 'PyQt5_sip', '@SIP_MODULE_PACKAGE_NAME@': 'PyQt5', '@SIP_MODULE_VERSION@': '12.13.0', '@_SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@_SIP_MODULE_NAME@': 'sip', '@_SIP_MODULE_SHARED@': '1', '@_SIP_MODULE_ENTRY@': 'PyInit_sip', '@_SIP_MODULE_LEGACY@': '1', '@_SIP_VERSION@': '0x6070c', '@_SIP_VERSION_STR@': '6.7.12'}
_create_sdist - NAME FROM SOURCE DIR: qtlib.c
_create_sdist - NAME FROM SOURCE DIR: descriptors.c
_create_sdist - NAME FROM SOURCE DIR: README.in
_install_source_file - module_source_dir: /home/gw/anaconda3/envs/androidDeploy/lib/python3.10/site-packages/sipbuild/module/source/12
name: README
target_dir: PyQt5_sip-12.13.0
_install_file - NAME OUT: PyQt5_sip-12.13.0/README
PATCHES: {'@SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@SIP_MODULE_PROJECT_NAME@': 'PyQt5_sip', '@SIP_MODULE_PACKAGE_NAME@': 'PyQt5', '@SIP_MODULE_VERSION@': '12.13.0', '@_SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@_SIP_MODULE_NAME@': 'sip', '@_SIP_MODULE_SHARED@': '1', '@_SIP_MODULE_ENTRY@': 'PyInit_sip', '@_SIP_MODULE_LEGACY@': '1', '@_SIP_VERSION@': '0x6070c', '@_SIP_VERSION_STR@': '6.7.12'}
_create_sdist - SETUP CONFIG 2: /home/gw/anaconda3/envs/androidDeploy/lib/python3.10/site-packages/sipbuild/module/source/12/setup.cfg.in
_install_file - NAME OUT: PyQt5_sip-12.13.0/setup.cfg
PATCHES: {'@SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@SIP_MODULE_PROJECT_NAME@': 'PyQt5_sip', '@SIP_MODULE_PACKAGE_NAME@': 'PyQt5', '@SIP_MODULE_VERSION@': '12.13.0', '@_SIP_MODULE_FQ_NAME@': 'PyQt5.sip', '@_SIP_MODULE_NAME@': 'sip', '@_SIP_MODULE_SHARED@': '1', '@_SIP_MODULE_ENTRY@': 'PyInit_sip', '@_SIP_MODULE_LEGACY@': '1', '@_SIP_VERSION@': '0x6070c', '@_SIP_VERSION_STR@': '6.7.12'}
_create_sdist - SAVED CWD: /home/gw/LYH/androidDepoly/QTgui
_create_sdist - SDIST DIR 2: PyQt5_sip-12.13.0
_create_sdist - SYS EXEC: /home/gw/anaconda3/envs/androidDeploy/bin/python
running sdist
running egg_info
creating PyQt5_sip.egg-info
writing PyQt5_sip.egg-info/PKG-INFO
writing dependency_links to PyQt5_sip.egg-info/dependency_links.txt
writing top-level names to PyQt5_sip.egg-info/top_level.txt
writing manifest file 'PyQt5_sip.egg-info/SOURCES.txt'
reading manifest file 'PyQt5_sip.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
adding license file 'LICENSE-GPL2'
adding license file 'LICENSE-GPL3'
writing manifest file 'PyQt5_sip.egg-info/SOURCES.txt'
running check
creating pyqt5_sip-12.13.0
creating pyqt5_sip-12.13.0/PyQt5_sip.egg-info
copying files to pyqt5_sip-12.13.0...
copying LICENSE -> pyqt5_sip-12.13.0
copying LICENSE-GPL2 -> pyqt5_sip-12.13.0
copying LICENSE-GPL3 -> pyqt5_sip-12.13.0
copying MANIFEST.in -> pyqt5_sip-12.13.0
copying README -> pyqt5_sip-12.13.0
copying apiversions.c -> pyqt5_sip-12.13.0
copying bool.cpp -> pyqt5_sip-12.13.0
copying descriptors.c -> pyqt5_sip-12.13.0
copying int_convertors.c -> pyqt5_sip-12.13.0
copying objmap.c -> pyqt5_sip-12.13.0
copying pyproject.toml -> pyqt5_sip-12.13.0
copying qtlib.c -> pyqt5_sip-12.13.0
copying setup.cfg -> pyqt5_sip-12.13.0
copying setup.py -> pyqt5_sip-12.13.0
copying sip.h -> pyqt5_sip-12.13.0
copying sip_array.c -> pyqt5_sip-12.13.0
copying sip_array.h -> pyqt5_sip-12.13.0
copying sipint.h -> pyqt5_sip-12.13.0
copying siplib.c -> pyqt5_sip-12.13.0
copying threads.c -> pyqt5_sip-12.13.0
copying voidptr.c -> pyqt5_sip-12.13.0
copying PyQt5_sip.egg-info/PKG-INFO -> pyqt5_sip-12.13.0/PyQt5_sip.egg-info
copying PyQt5_sip.egg-info/SOURCES.txt -> pyqt5_sip-12.13.0/PyQt5_sip.egg-info
copying PyQt5_sip.egg-info/dependency_links.txt -> pyqt5_sip-12.13.0/PyQt5_sip.egg-info
copying PyQt5_sip.egg-info/top_level.txt -> pyqt5_sip-12.13.0/PyQt5_sip.egg-info
copying PyQt5_sip.egg-info/SOURCES.txt -> pyqt5_sip-12.13.0/PyQt5_sip.egg-info
Writing pyqt5_sip-12.13.0/setup.cfg
Creating tar archive
removing 'pyqt5_sip-12.13.0' (and everything under it)
_create_sdist - SAVED CWD 2: /home/gw/LYH/androidDepoly/QTgui
_create_sdist - SDIST DIR 3: PyQt5_sip-12.13.0
and in my root, I get PyQt5_sip-12.13.0
fold and 'pyqt5_sip-12.13.0.tar.gz' file.
Ok I can confirm that you are right about the command setup.py
messing things up.
I dug a bit more into the setup template and I have added a few more debug prints to module.py
:
Again, if you have time, replace module.py
with my extra verbose code and run sip-module --sdist PyQt5.sip --abi-version 12
and we'll inspect the output.
This time, the output will tell us from which python function the problem emerges (it might be the replace
).
If this round comes back inconclusive, I'll close the issue and mark it as "temporarily fixed" with your previous fix.
All right! The print log information is in log file:
If you locate where the problem occurred, please let me know.
Ah! I found it... I was looking at the wrong place...
The "issue" comes from setuptools v69.3.0
Changelog infos here:
sip-module --sdist PyQt5.sip --abi-version 12
calls the command subprocess.run( [sys.executable, 'setup.py', '--quiet', 'sdist', '--dist-dir', '..'])
subprocess.run( [sys.executable, 'setup.py', '--quiet', 'sdist', '--dist-dir', '..'])
calls the make_distribution()
from command/sdist.py
which calls the one from setuptools/_distutils/command/sdist.py
make_distribution()
calls get_fullname()
from setuptools/_core_metadata.py
get_fullname()
calls canonicalize_name()
from setuptools/_vendor/packaging/utils.py
which is effectively distutils
(https://github.com/pypa/distutils)The changes have been made with this chronology:
distutils
has been introduced during this issue: https://github.com/pypa/distutils/pull/244 (in April 2024)sip-module --sdist PyQt5.sip --abi-version 12
is run to get the source distribution.PyQt5_sip-12.13.0.tar.gz
because the name has been canonicalised to pyqt5_sip-12.13.0.tar.gz
by distutils
.1) Constrain setuptools
to be < v69.3.0
in requirements.txt
-> quite restrictive but works for now
2) Modify self.module_name
to be canonical in pyqtdeploy/pyqtdeploy/sysroot/plugins/SIP.py
so that it can find the archive downloaded (the tar.gz
) -> need to check with Phil
3) Modify the value of module_name
to pyqt5.sip
in sysroot.toml
under [SIP]
Thanks for your debugging support!
TL;DR: for now, use pip install setuptools==69.2.0
That's great! We can continue doing this great work. :wink:
Hi,@achille-martin thank you for your amazing meticulous job and selfless open source. according to tutorial, I have an issue while implementing step 1.7. I couldn't find a solution to the relevant problem online, if you know how to solve it, please let me know and Thank you so much! The error message is as follows:
system information: