search

marcelotduarte / cx_Freeze

cx_Freeze creates standalone executables from Python scripts, with the same performance, is cross-platform and should work on any platform that Python itself works on.
https://marcelotduarte.github.io/cx_Freeze/
Other
1.31k stars 215 forks source link

test_hooks_pandas.py::test_pandas FAILS: Possible circular dependency #2513

Closed brianrobt closed 1 month ago

brianrobt commented 1 month ago

System information

Python version: 3.12.4 OS: EndeavourOS (Arch-based) cx_Freeze version: 7.2.0

Problem

Hello,

When trying to run the unit tests, all of the tests pass except for test_hooks_pandas.py::test_pandas. It fails with the message, subprocess.CalledProcessError: Command '['/tmp/pytest-of-brian/pytest-0/test_pandas0/build/exe.linux-x86_64-3.12/test_pandas']' returned non-zero exit status 255..

Executing the script manually gives me a different error about circular imports:

/tmp/pytest-of-brian/pytest-0/test_pandas0/build/exe.linux-x86_64-3.12/test_pandas                                                                                                                                                                        ─╯
Traceback (most recent call last):
  File "/home/brian/workspace/python-cx-freeze/src/cx_Freeze-7.2.0/test_dir/usr/lib/python3.12/site-packages/cx_Freeze/initscripts/__startup__.py", line 141, in run
    module_init.run(name + "__main__")
  File "/home/brian/workspace/python-cx-freeze/src/cx_Freeze-7.2.0/test_dir/usr/lib/python3.12/site-packages/cx_Freeze/initscripts/console.py", line 25, in run
    exec(code, main_globals)
  File "test_pandas.py", line 4, in <module>
  File "/usr/lib/python3.12/site-packages/pandas/__init__.py", line 49, in <module>
    from pandas.core.api import (
  File "/usr/lib/python3.12/site-packages/pandas/core/api.py", line 47, in <module>
    from pandas.core.groupby import (
  File "/usr/lib/python3.12/site-packages/pandas/core/groupby/__init__.py", line 1, in <module>
    from pandas.core.groupby.generic import (
  File "/usr/lib/python3.12/site-packages/pandas/core/groupby/generic.py", line 73, in <module>
    from pandas.core.groupby.groupby import (
  File "/usr/lib/python3.12/site-packages/pandas/core/groupby/groupby.py", line 124, in <module>
    from pandas.core.groupby import (
ImportError: cannot import name 'numba_' from partially initialized module 'pandas.core.groupby' (most likely due to a circular import) (/tmp/pytest-of-brian/pytest-0/test_pandas0/build/exe.linux-x86_64-3.12/lib/pandas/core/groupby/__init__.pyc)

This is preventing me from uploading the v7.2.0 to the AUR.

Thank you, Brian

References

marcelotduarte commented 1 month ago

Hi! The https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=python-cx-freeze installs some packages in depends and checkdepends that were unnecessary to the build and tests. For instance, pyqt5, bcrypt, etc. The pandas package is optional. So you can change it using the MSYS2 PKGBUILD as an example. Naturally, it would be good to fix this error in the pandas hook for the AUR, but I have to see how to test this (Maybe using docker/podman), so leave the issue open for now.

marcelotduarte commented 1 month ago

@brianrobt I tested using an docker image and the tests passed, using the latest versions:

pacman -S python-pandas
resolving dependencies...
looking for conflicting packages...

Package (8)            New Version  Net Change  Download Size

extra/blas             3.12.0-5       0.65 MiB       0.22 MiB
extra/cblas            3.12.0-5       0.34 MiB       0.06 MiB
extra/lapack           3.12.0-5       7.48 MiB       2.59 MiB
extra/python-dateutil  2.9.0-5        0.99 MiB       0.29 MiB
extra/python-numpy     2.0.0-1       46.25 MiB       7.27 MiB
extra/python-pytz      2024.1-2       0.16 MiB       0.05 MiB
extra/python-six       1.16.0-9       0.11 MiB       0.03 MiB
extra/python-pandas    2.2.2-2       92.57 MiB      13.91 MiB
Tests

``` # pip install -e. --no-deps --no-build-isolation Resolved 1 package in 597ms Built cx-freeze @ file:///home/marcelo/github/cx_Freeze Prepared 1 package in 1.39s Uninstalled 1 package in 0.99ms Installed 1 package in 3ms - cx-freeze==7.3.0.dev6 (from file:///home/marcelo/github/cx_Freeze) + cx-freeze==7.2.0 (from file:///home/marcelo/github/cx_Freeze) # python -m pytest -nauto --no-cov =========================================================== test session starts ============================================================ platform linux -- Python 3.12.4, pytest-8.3.1, pluggy-1.5.0 rootdir: /home/marcelo/github/cx_Freeze configfile: pyproject.toml testpaths: tests plugins: timeout-2.3.1, typeguard-4.3.0, cov-5.0.0, datafiles-3.0.0, xdist-3.5.0, mock-3.14.0 6 workers [236 items] skipped ........s.ss.......x....x...........................x...........s.s..x...s.................................x....x................... [ 55%] .....s.....ss........s..................................................sssssss.ssssssssssssss.s........ [100%] ========================================================= short test summary info ========================================================== PASSED tests/test___init__.py::test_exposed_namespaces[None-extra_modules0] PASSED tests/test___init__.py::test_exposed_namespaces[win32-extra_modules1] PASSED tests/test___init__.py::test_exposed_namespaces[darwin-extra_modules2] PASSED tests/test___init__.py::test_exposed_namespaces[linux-extra_modules3] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[excludes='tkinter,unittest'] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[silent=false->0] PASSED tests/test_command_bdist_rpm.py::test_bdist_rpm_not_rpmbuild PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[excludes=['tkinter','unittest']] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[silent=true->1] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[silent-level=none->0] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[replace_paths=*] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[silent-level=0->0] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[replace_paths=*=] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[silent-level=1->1] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[replace_paths=[*=]] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[silent-level=2->2] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[silent=none->0] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[silent-level=3->3] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[no_compress=true] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[zip_include_packages/zip_exclude_packages=none/none] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options_raises[build-exe=build] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[zip_include_package=[*]] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--build-exe(notused)] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--includes=tkinter,unittest] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--build-exe=] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--packages=] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--build-exe=dist] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--packages=tkinter,unittest] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--excludes=] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--replace-paths=*=] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--excludes=tkinter,unittest] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--bin-excludes=] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--includes=] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--bin-includes=] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--include-files=] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--bin-path-excludes=] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--zip-includes=] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--bin-path-includes=] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--zip-include-packages/--zip-exclude-packages(notused)] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--zip-filename=test] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--zip-include-packages=/--zip-exclude-packages=] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--zip-filename=test.zip] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--zip-include-package=*/--zip-exclude-packages=] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--no-compress(notused),--zip-filename(notused)] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--zip-include-packages=/--zip-exclude-packages=*] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--no-compress] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--zip-include-packages/--zip-exclude-packages=namespace/namespace] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--optimize(notused)] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--zip-filename=] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--optimize=0] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--optimize=2] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--optimize=1] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--optimize(-O0] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--silent-level=1->1] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--optimize(-O1] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--silent-level=2->2] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--optimize(-O2] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--silent-level=3->3] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--silent(notused)] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--silent] PASSED tests/test_command_build_exe.py::test_build_exe_script_args[--silent-level=0->0] PASSED tests/test___main__.py::test___main__ PASSED tests/test_cli.py::test_cxfreeze_deprecated_option PASSED tests/test_cli.py::test_cxfreeze_without_options PASSED tests/test_command_build_exe.py::test_build_exe_advanced PASSED tests/test_command_build_exe.py::test_build_exe_sqlite PASSED tests/test_command_bdist_appimage.py::test_bdist_appimage_skip_build PASSED tests/test_cli.py::test_cxfreeze PASSED tests/test_cli.py::test_cxfreeze_help PASSED tests/test_cli.py::test_cxfreeze_include_path PASSED tests/test_command_bdist_appimage.py::test_bdist_appimage_not_posix PASSED tests/test_command_bdist_appimage.py::test_bdist_appimage_target_name PASSED tests/test_cli.py::test_cxfreeze_additional_help PASSED tests/test_command_bdist_appimage.py::test_bdist_appimage_target_name_and_version PASSED tests/test_command_build.py::test_build PASSED tests/test_command_build_exe.py::test_build_exe_asmodule PASSED tests/test_command_bdist_appimage.py::test_bdist_appimage_target_name_and_version_none PASSED tests/test_command_build.py::test_build_raises[build-exe=] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[build-exe=none] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[build-exe=dist] PASSED tests/test_executables.py::test_valid[target_name-None-test] PASSED tests/test_executables.py::test_valid[target_name-test1-test1] PASSED tests/test_executables.py::test_valid[target_name-test-0.1-test-0.1] PASSED tests/test_executables.py::test_valid[target_name-test.exe-test.exe] PASSED tests/test_executables.py::test_valid[icon-icon-result10] PASSED tests/test_executables.py::test_invalid[executables-invalid-empty] PASSED tests/test_executables.py::test_invalid[executables-invalid-string] PASSED tests/test_executables.py::test_invalid[executable-invalid-base] PASSED tests/test_executables.py::test_invalid[executable-invalid-init_script] PASSED tests/test_executables.py::test_invalid[executable-invalid-target_name] PASSED tests/test_executables.py::test_invalid[executable-invalid-target_name-isidentifier] PASSED tests/test_command_build_exe.py::test_zip_include_packages PASSED tests/test_command_bdist_appimage.py::test_bdist_appimage_simple PASSED tests/test_command_bdist_deb.py::test_bdist_deb_not_posix PASSED tests/test_command_bdist_deb.py::test_bdist_deb_not_alien PASSED tests/test_command_bdist_deb.py::test_bdist_deb_not_fakeroot PASSED tests/test_command_bdist_deb.py::test_bdist_deb_dry_run PASSED tests/test_cli.py::test_cxfreeze_deprecated_behavior PASSED tests/test_executables.py::test_executables[setup_toml] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[zip_exclude_packages=*] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[zip_include_packages/zip_exclude_packages=namespace/namespace] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[zip_include_package=*] PASSED tests/test_command_bdist_rpm.py::test_bdist_rpm_not_posix PASSED tests/test_finder.py::TestModuleFinder::test_invalid_syntax PASSED tests/test_finder.py::TestModuleFinder::test_find_spec PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[zip_filename=none] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[zip_filename=test] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[zip_filename=test.zip] PASSED tests/test_executables.py::test_valid_icon PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[zip_filename=0] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[zip_filename=1] PASSED tests/test_command_build_exe.py::test_build_exe_finalize_options[no_compress=false] PASSED tests/test_freezer.py::test_freezer_target_dir_locked PASSED tests/test_command_build_exe.py::test_zip_exclude_packages PASSED tests/test_freezer.py::test_freezer_target_dir_empty PASSED tests/test_command_bdist_appimage.py::test_bdist_appimage_target_name_with_extension PASSED tests/test_executables.py::test_valid[base-None-console-] PASSED tests/test_executables.py::test_valid[base-console-console-] PASSED tests/test_executables.py::test_valid[base-gui-console-] PASSED tests/test_executables.py::test_valid[base-service-console-] PASSED tests/test_executables.py::test_valid[init_script-None-console.py] PASSED tests/test_executables.py::test_valid[init_script-console-console.py] PASSED tests/test_freezer.py::test_freezer_default_bin_includes PASSED tests/test_executables.py::test_executables[setup_py] PASSED tests/test_executables.py::test_not_found_icon PASSED tests/test_freezer.py::test_freezer_options[compress=false] PASSED tests/test_freezer.py::test_freezer_target_dir_dist PASSED tests/test_command_install.py::test_install PASSED tests/test_command_install_exe.py::test_install_exe_finalize_options[install-dir=none] PASSED tests/test_command_install_exe.py::test_install_exe_finalize_options[install-dir=dist] PASSED tests/test_freezer.py::test_freezer_default_bin_includes_emulated PASSED tests/test_freezer.py::test_freezer_populate_zip_options_invalid_values PASSED tests/test_freezer.py::test_freezer_options[compress=true] PASSED tests/test_freezer.py::test_freezer_target_dir_utf8 PASSED tests/test_freezer.py::test_freezer_target_dir_in_path PASSED tests/test_freezer.py::test_freezer_options[compress=none] PASSED tests/test_freezer.py::test_freezer_options[replace_paths=*] PASSED tests/test_freezer.py::test_freezer_options[excludes=['tkinter','unittest']] PASSED tests/test_freezer.py::test_freezer_options[silent=true->1] PASSED tests/test_freezer.py::test_freezer_options[optimize=0->0] PASSED tests/test_executables.py::test_executables[setup_cfg] PASSED tests/test_executables.py::test_executable_rename PASSED tests/test_finder.py::TestModuleFinder::test_scan_code PASSED tests/test_finder.py::TestModuleFinder::test_not_import_invalid_module_name PASSED tests/test_freezer.py::test_freezer_options[replace_paths=[*=]] PASSED tests/test_freezer.py::test_freezer_options[silent=3->3] PASSED tests/test_freezer.py::test_freezer_options[silent=0->0] PASSED tests/test_freezer.py::test_freezer_options[optimize=1->1] PASSED tests/test_freezer.py::test_freezer_options[zip_include_packages/zip_exclude_packages=namespace/namespace] PASSED tests/test_freezer.py::test_freezer_options[optimize=none->0] PASSED tests/test_freezer.py::test_freezer_options[silent=1->1] PASSED tests/test_freezer.py::test_freezer_options[zip_include_packages/zip_exclude_packages=none/none] PASSED tests/test_freezer.py::test_freezer_options[optimize=2->2] PASSED tests/test_executables.py::test_executables[setup_mix] PASSED tests/test_freezer.py::test_freezer_options[silent=2->2] PASSED tests/test_freezer.py::test_freezer_options[zip_include_package=*] PASSED tests/test_freezer.py::test_freezer_options[silent=none->0] PASSED tests/test_freezer.py::test_freezer_zip_filename[zip_filename=none] PASSED tests/test_freezer.py::test_freezer_zip_filename[zip_filename=test.zip/target_dir=utf_8/portuguese] PASSED tests/test_freezer.py::test_freezer_options[zip_exclude_packages=*] PASSED tests/test_freezer.py::test_freezer_options[silent=false->0] PASSED tests/test_executables.py::test_executables[setup_adv_toml] PASSED tests/test_freezer.py::test_freezer_zip_filename[zip_filename=none/compress=false] PASSED tests/test_freezer.py::test_freezer_zip_filename[zip_filename=test.zip/target_dir=utf_8/chinese] PASSED tests/test_freezer.py::test_freezer_zip_filename[zip_filename=test] PASSED tests/test_hooks_multiprocessing.py::test_multiprocessing[sample1-fork] PASSED tests/test_freezer.py::test_freezer_zip_filename[zip_filename=name/compress=false] PASSED tests/test_hooks_multiprocessing.py::test_multiprocessing[sample0-spawn] PASSED tests/test_executables.py::test_executables[setup_adv_py] PASSED tests/test_freezer.py::test_freezer_zip_filename[zip_filename=none/compress=true] PASSED tests/test_freezer.py::test_freezer_zip_filename[zip_filename=test.zip] PASSED tests/test_hooks_multiprocessing.py::test_multiprocessing[sample2-fork] PASSED tests/test_hooks_multiprocessing.py::test_multiprocessing[sample0-fork] PASSED tests/test_modulefinder.py::test_finder[coding_default_utf8_test] PASSED tests/test_modulefinder.py::test_finder[coding_explicit_cp1252_test] PASSED tests/test_modulefinder.py::test_finder[maybe_test] PASSED tests/test_modulefinder.py::test_finder[maybe_test_new] PASSED tests/test_modulefinder.py::test_finder[package_test] PASSED tests/test_modulefinder.py::test_finder[relative_import_test] PASSED tests/test_hooks_multiprocessing.py::test_multiprocessing[sample2-spawn] PASSED tests/test_modulefinder.py::test_finder[relative_import_test_2] PASSED tests/test_modulefinder.py::test_finder[relative_import_test_3] PASSED tests/test_modulefinder.py::test_finder[same_name_as_bad_test] PASSED tests/test_modulefinder.py::test_finder[sub_package_test] PASSED tests/test_modulefinder.py::test_bytecode PASSED tests/test_modulefinder.py::test_zip_include_packages PASSED tests/test_hooks_multiprocessing.py::test_multiprocessing[sample1-spawn] PASSED tests/test_modulefinder.py::test_zip_exclude_packages PASSED tests/test_modulefinder.py::test_finder[coding_explicit_utf8_test] PASSED tests/test_modulefinder.py::test_finder[extended_opargs_test] PASSED tests/test_hooks_stdlib.py::test_ssl PASSED tests/test_modulefinder.py::test_finder[absolute_import_test] PASSED tests/test_hooks_multiprocessing.py::test_multiprocessing[sample3-forkserver] PASSED tests/test_hooks_multiprocessing.py::test_multiprocessing[sample3-fork] PASSED tests/test_modulefinder.py::test_finder[import_call_test] PASSED tests/test_hooks_multiprocessing.py::test_multiprocessing[sample3-spawn] PASSED tests/test_modulefinder.py::test_finder[relative_import_test_4] PASSED tests/test_hooks_pandas.py::test_pandas SKIPPED [1] tests/test_command_bdist_dmg.py:11: macOS tests SKIPPED [1] tests/test_command_bdist_mac.py:16: macOS tests SKIPPED [1] tests/test_command_bdist_msi.py:20: Windows tests SKIPPED [1] tests/test_hooks_mpire.py:15: Depends on extra package: mpire SKIPPED [1] tests/test_hooks_multiprocess.py:15: Depends on extra package: multiprocess SKIPPED [1] tests/test_hooks_torch_multiprocessing.py:15: Depends on extra package: torch SKIPPED [1] tests/test_windows_manifest.py:19: Windows tests SKIPPED [3] tests/test_command_build_exe.py:33: Windows tests SKIPPED [2] tests/test_command_build_exe.py:205: Windows tests SKIPPED [1] tests/test_cli.py:92: Python >= 3.11 SKIPPED [1] tests/test_executables.py:371: Windows tests SKIPPED [3] tests/test_freezer.py:182: Windows tests SKIPPED [1] tests/test_plist_items.py:18: macOS tests SKIPPED [1] tests/test_winmsvcr.py:38: Windows tests SKIPPED [2] tests/test_winmsvcr.py:44: Windows tests SKIPPED [1] tests/test_winversioninfo.py:35: Windows tests SKIPPED [1] tests/test_winversioninfo.py:54: Windows tests SKIPPED [1] tests/test_winversioninfo.py:98: Windows tests SKIPPED [9] tests/test_winversioninfo.py:106: Windows tests SKIPPED [1] tests/test_winversioninfo.py:128: Windows tests SKIPPED [3] tests/test_winversioninfo.py:148: Windows tests SKIPPED [1] tests/test_winversioninfo.py:162: Windows tests SKIPPED [1] tests/test_winversioninfo.py:167: Windows tests XFAIL tests/test_command_bdist_rpm.py::test_bdist_rpm_options[spec_only] - reason: rpmbuild not installed XFAIL tests/test_command_bdist_rpm.py::test_bdist_rpm_options_run[spec_only] - reason: rpmbuild not installed XFAIL tests/test_command_bdist_rpm.py::test_bdist_rpm_simple - reason: rpmbuild not installed XFAIL tests/test_command_bdist_rpm.py::test_bdist_rpm - reason: rpmbuild not installed XFAIL tests/test_command_bdist_deb.py::test_bdist_deb_simple - reason: alien not installed XFAIL tests/test_command_bdist_deb.py::test_bdist_deb - reason: alien not installed =============================================== 198 passed, 39 skipped, 6 xfailed in 37.24s ================================================ ```

brianrobt commented 1 month ago

Thanks, @marcelotduarte!

brianrobt commented 1 month ago

I worked around this by skipping the test case since it runs if python-pandas is installed from Arch extras, regardless of whether python-pandas is included in the checkdepends.

marcelotduarte commented 1 month ago

Have you tested it with pandas to see if it fails? extra/python-pandas 2.2.2-2

brianrobt commented 1 month ago

I'll try to test it out this week.

brianrobt commented 1 month ago

I get the same error.