search

apache / arrow

Apache Arrow is a multi-language toolbox for accelerated data interchange and in-memory processing
https://arrow.apache.org/
Apache License 2.0
14.3k stars 3.48k forks source link

[Python] Test failure on verifying 15.0.0 RC1 #39679

Open zanmato1984 opened 8 months ago

zanmato1984 commented 8 months ago

Describe the bug, including details regarding any error messages, version, and platform.

This seems a trivial one, but I did get 1 test failure when verifying 15.0.0 RC1 by:

TEST_DEFAULT=0 TEST_PYTHON=1 ./verify-release-candidate.sh 15.0.0 1
========================================================================================================================================== warnings summary ===========================================================================================================================================
pyarrow/tests/test_dataset.py::test_make_fragment
  /private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/venv-source/lib/python3.9/site-packages/numpy/core/fromnumeric.py:59: FutureWarning: 'DataFrame.swapaxes' is deprecated and will be removed in a future version. Please use 'DataFrame.transpose' instead.
    return bound(*args, **kwds)

pyarrow/tests/test_pandas.py::TestConvertMisc::test_category
pyarrow/tests/test_pandas.py::TestConvertMisc::test_category
pyarrow/tests/test_pandas.py::TestConvertMisc::test_category
  /private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/apache-arrow-15.0.0/python/pyarrow/tests/test_pandas.py:118: FutureWarning: Mismatched null-like values None and nan found. In a future version, pandas equality-testing functions (e.g. assert_frame_equal) will consider these not-matching and raise.
    tm.assert_frame_equal(result, expected, check_dtype=check_dtype,

pyarrow/tests/test_pandas.py::TestConvertMisc::test_category
pyarrow/tests/test_pandas.py::TestConvertMisc::test_category
pyarrow/tests/test_pandas.py::TestConvertMisc::test_category
  /private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/apache-arrow-15.0.0/python/pyarrow/tests/test_pandas.py:154: FutureWarning: Mismatched null-like values None and nan found. In a future version, pandas equality-testing functions (e.g. assert_frame_equal) will consider these not-matching and raise.
    tm.assert_series_equal(pd.Series(result), expected, check_names=False)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================================================================================================= short test summary info =======================================================================================================================================
FAILED pyarrow/tests/test_cython.py::test_cython_api - subprocess.CalledProcessError: Command '['/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/venv-source/bin/python3', '-c', "if 1:\n            import sys\n            import os\n\n            try:\n                # Add dll directory was added on pytho...
===================================================================================================== 1 failed, 6848 passed, 603 skipped, 12 xfailed, 2 xpassed, 7 warnings in 318.47s (0:05:18) ======================================================================================================

Component(s)

Python

kou commented 8 months ago

Do you have a log for failure details part?

zanmato1984 commented 8 months ago

Do you have a log for failure details part?

Sorry I forgot to paste the failures section in report:

============================================================================================================================================== FAILURES ===============================================================================================================================================
___________________________________________________________________________________________________________________________________________ test_cython_api ___________________________________________________________________________________________________________________________________________

tmpdir = local('/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/pytest-of-zanmato/pytest-1/test_cython_api0')

    @pytest.mark.cython
    def test_cython_api(tmpdir):
        """
        Basic test for the Cython API.
        """
        # Fail early if cython is not found
        import cython  # noqa

        with tmpdir.as_cwd():
            # Set up temporary workspace
            pyx_file = 'pyarrow_cython_example.pyx'
            shutil.copyfile(os.path.join(here, pyx_file),
                            os.path.join(str(tmpdir), pyx_file))
            # Create setup.py file
            setup_code = setup_template.format(pyx_file=pyx_file,
                                               compiler_opts=compiler_opts,
                                               test_ld_path=test_ld_path)
            with open('setup.py', 'w') as f:
                f.write(setup_code)

            # ARROW-2263: Make environment with this pyarrow/ package first on the
            # PYTHONPATH, for local dev environments
            subprocess_env = test_util.get_modified_env_with_pythonpath()

            # Compile extension module
            subprocess.check_call([sys.executable, 'setup.py',
                                   'build_ext', '--inplace'],
                                  env=subprocess_env)

            # Check basic functionality
            orig_path = sys.path[:]
            sys.path.insert(0, str(tmpdir))
            try:
                mod = __import__('pyarrow_cython_example')
                check_cython_example_module(mod)
            finally:
                sys.path = orig_path

            # Check the extension module is loadable from a subprocess without
            # pyarrow imported first.
            code = """if 1:
                import sys
                import os

                try:
                    # Add dll directory was added on python 3.8
                    # and is required in order to find extra DLLs
                    # only for win32
                    for dir in {library_dirs}:
                        os.add_dll_directory(dir)
                except AttributeError:
                    pass

                mod = __import__({mod_name!r})
                arr = mod.make_null_array(5)
                assert mod.get_array_length(arr) == 5
                assert arr.null_count == 5
            """.format(mod_name='pyarrow_cython_example',
                       library_dirs=pa.get_library_dirs())

            path_var = None
            if sys.platform == 'win32':
                if not hasattr(os, 'add_dll_directory'):
                    # Python 3.8 onwards don't check extension module DLLs on path
                    # we have to use os.add_dll_directory instead.
                    delim, path_var = ';', 'PATH'
            elif sys.platform == 'darwin':
                delim, path_var = ':', 'DYLD_LIBRARY_PATH'
            else:
                delim, path_var = ':', 'LD_LIBRARY_PATH'

            if path_var:
                paths = sys.path
                paths += pa.get_library_dirs()
                paths += [subprocess_env.get(path_var, '')]
                paths = [path for path in paths if path]
                subprocess_env[path_var] = delim.join(paths)
>           subprocess.check_call([sys.executable, '-c', code],
                                  stdout=subprocess.PIPE,
                                  env=subprocess_env)

pyarrow/tests/test_cython.py:161:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

popenargs = (['/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/venv-source/bin/python3', '-..._null_array(5)\n            assert mod.get_array_length(arr) == 5\n            assert arr.null_count == 5\n        '],)
kwargs = {'env': {'ALACRITTY_LOG': '/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/Alacritty-1575.log', 'ALACRITTY_SOCKET': '...yg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/apache-arrow-15.0.0/cpp/gdb_arrow.py', ...}, 'stdout': -1}, retcode = -6
cmd = ['/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/venv-source/bin/python3', '-c...ke_null_array(5)\n            assert mod.get_array_length(arr) == 5\n            assert arr.null_count == 5\n        ']

    def check_call(*popenargs, **kwargs):
        """Run command with arguments.  Wait for command to complete.  If
        the exit code was zero then return, otherwise raise
        CalledProcessError.  The CalledProcessError object will have the
        return code in the returncode attribute.

        The arguments are the same as for the call function.  Example:

        check_call(["ls", "-l"])
        """
        retcode = call(*popenargs, **kwargs)
        if retcode:
            cmd = kwargs.get("args")
            if cmd is None:
                cmd = popenargs[0]
>           raise CalledProcessError(retcode, cmd)
E           subprocess.CalledProcessError: Command '['/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/venv-source/bin/python3', '-c', "if 1:\n            import sys\n            import os\n\n            try:\n                # Add dll directory was added on python 3.8\n                # and is required in order to find extra DLLs\n                # only for win32\n                for dir in ['/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/apache-arrow-15.0.0/python/pyarrow', '/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/install/lib']:\n                    os.add_dll_directory(dir)\n            except AttributeError:\n                pass\n\n            mod = __import__('pyarrow_cython_example')\n            arr = mod.make_null_array(5)\n            assert mod.get_array_length(arr) == 5\n            assert arr.null_count == 5\n        "]' died with <Signals.SIGABRT: 6>.

/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py:373: CalledProcessError
---------------------------------------------------------------------------------------------------------------------------------------- Captured stdout call -----------------------------------------------------------------------------------------------------------------------------------------
Compiling pyarrow_cython_example.pyx because it changed.
[1/1] Cythonizing pyarrow_cython_example.pyx
Extension module: <setuptools.extension.Extension('pyarrow_cython_example') at 0x11d3919a0> ['/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/venv-source/lib/python3.9/site-packages/numpy/core/include', '/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/apache-arrow-15.0.0/python/pyarrow/include'] ['arrow_python', 'arrow'] ['/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/apache-arrow-15.0.0/python/pyarrow', '/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/install/lib']
running build_ext
building 'pyarrow_cython_example' extension
creating build
creating build/temp.macosx-10.9-universal2-cpython-39
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -I/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/venv-source/lib/python3.9/site-packages/numpy/core/include -I/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/apache-arrow-15.0.0/python/pyarrow/include -I/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/venv-source/include -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -c pyarrow_cython_example.cpp -o build/temp.macosx-10.9-universal2-cpython-39/pyarrow_cython_example.o -std=c++17
creating build/lib.macosx-10.9-universal2-cpython-39
clang++ -bundle -undefined dynamic_lookup -arch arm64 -arch x86_64 -Wl,-headerpad,0x1000 build/temp.macosx-10.9-universal2-cpython-39/pyarrow_cython_example.o -L/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/apache-arrow-15.0.0/python/pyarrow -L/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/install/lib -larrow_python -larrow -o build/lib.macosx-10.9-universal2-cpython-39/pyarrow_cython_example.cpython-39-darwin.so
copying build/lib.macosx-10.9-universal2-cpython-39/pyarrow_cython_example.cpython-39-darwin.so ->
---------------------------------------------------------------------------------------------------------------------------------------- Captured stderr call -----------------------------------------------------------------------------------------------------------------------------------------
pyarrow_cython_example.cpp:4743:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_buffer' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_buffer)(std::shared_ptr< arrow::Buffer>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4744:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_resizable_buffer' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_resizable_buffer)(std::shared_ptr< arrow::ResizableBuffer>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4745:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_data_type' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_data_type)(std::shared_ptr< arrow::DataType>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4746:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_field' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_field)(std::shared_ptr< arrow::Field>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4747:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_schema' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_schema)(std::shared_ptr< arrow::Schema>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4750:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_chunked_array' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_chunked_array)(std::shared_ptr< arrow::ChunkedArray>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4751:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_coo_tensor' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_coo_tensor)(std::shared_ptr< arrow::SparseCOOTensor>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4752:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csc_matrix' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csc_matrix)(std::shared_ptr< arrow::SparseCSCMatrix>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4753:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csf_tensor' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csf_tensor)(std::shared_ptr< arrow::SparseCSFTensor>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4754:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csr_matrix' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csr_matrix)(std::shared_ptr< arrow::SparseCSRMatrix>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4755:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_tensor' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_tensor)(std::shared_ptr< arrow::Tensor>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4756:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_batch' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_batch)(std::shared_ptr< arrow::RecordBatch>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4757:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_table' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_table)(std::shared_ptr< arrow::Table>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4758:43: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_buffer' [-Wunused-variable]
static std::shared_ptr< arrow::Buffer>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_buffer)(PyObject *); /*proto*/
                                          ^
pyarrow_cython_example.cpp:4760:42: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_field' [-Wunused-variable]
static std::shared_ptr< arrow::Field>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_field)(PyObject *); /*proto*/
                                         ^
pyarrow_cython_example.cpp:4761:43: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_schema' [-Wunused-variable]
static std::shared_ptr< arrow::Schema>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_schema)(PyObject *); /*proto*/
                                          ^
pyarrow_cython_example.cpp:4764:49: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_chunked_array' [-Wunused-variable]
static std::shared_ptr< arrow::ChunkedArray>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_chunked_array)(PyObject *); /*proto*/
                                                ^
pyarrow_cython_example.cpp:4765:52: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_coo_tensor' [-Wunused-variable]
static std::shared_ptr< arrow::SparseCOOTensor>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_coo_tensor)(PyObject *); /*proto*/
                                                   ^
pyarrow_cython_example.cpp:4766:52: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csc_matrix' [-Wunused-variable]
static std::shared_ptr< arrow::SparseCSCMatrix>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csc_matrix)(PyObject *); /*proto*/
                                                   ^
pyarrow_cython_example.cpp:4767:52: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csf_tensor' [-Wunused-variable]
static std::shared_ptr< arrow::SparseCSFTensor>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csf_tensor)(PyObject *); /*proto*/
                                                   ^
pyarrow_cython_example.cpp:4768:52: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csr_matrix' [-Wunused-variable]
static std::shared_ptr< arrow::SparseCSRMatrix>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csr_matrix)(PyObject *); /*proto*/
                                                   ^
pyarrow_cython_example.cpp:4769:43: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_tensor' [-Wunused-variable]
static std::shared_ptr< arrow::Tensor>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_tensor)(PyObject *); /*proto*/
                                          ^
pyarrow_cython_example.cpp:4770:48: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_batch' [-Wunused-variable]
static std::shared_ptr< arrow::RecordBatch>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_batch)(PyObject *); /*proto*/
                                               ^
pyarrow_cython_example.cpp:4771:42: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_table' [-Wunused-variable]
static std::shared_ptr< arrow::Table>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_table)(PyObject *); /*proto*/
                                         ^
24 warnings generated.
pyarrow_cython_example.cpp:4743:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_buffer' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_buffer)(std::shared_ptr< arrow::Buffer>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4744:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_resizable_buffer' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_resizable_buffer)(std::shared_ptr< arrow::ResizableBuffer>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4745:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_data_type' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_data_type)(std::shared_ptr< arrow::DataType>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4746:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_field' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_field)(std::shared_ptr< arrow::Field>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4747:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_schema' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_schema)(std::shared_ptr< arrow::Schema>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4750:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_chunked_array' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_chunked_array)(std::shared_ptr< arrow::ChunkedArray>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4751:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_coo_tensor' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_coo_tensor)(std::shared_ptr< arrow::SparseCOOTensor>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4752:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csc_matrix' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csc_matrix)(std::shared_ptr< arrow::SparseCSCMatrix>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4753:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csf_tensor' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csf_tensor)(std::shared_ptr< arrow::SparseCSFTensor>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4754:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csr_matrix' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_sparse_csr_matrix)(std::shared_ptr< arrow::SparseCSRMatrix>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4755:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_tensor' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_tensor)(std::shared_ptr< arrow::Tensor>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4756:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_batch' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_batch)(std::shared_ptr< arrow::RecordBatch>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4757:20: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_wrap_table' [-Wunused-variable]
static PyObject *(*__pyx_f_7pyarrow_3lib_pyarrow_wrap_table)(std::shared_ptr< arrow::Table>  const &); /*proto*/
                   ^
pyarrow_cython_example.cpp:4758:43: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_buffer' [-Wunused-variable]
static std::shared_ptr< arrow::Buffer>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_buffer)(PyObject *); /*proto*/
                                          ^
pyarrow_cython_example.cpp:4760:42: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_field' [-Wunused-variable]
static std::shared_ptr< arrow::Field>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_field)(PyObject *); /*proto*/
                                         ^
pyarrow_cython_example.cpp:4761:43: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_schema' [-Wunused-variable]
static std::shared_ptr< arrow::Schema>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_schema)(PyObject *); /*proto*/
                                          ^
pyarrow_cython_example.cpp:4764:49: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_chunked_array' [-Wunused-variable]
static std::shared_ptr< arrow::ChunkedArray>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_chunked_array)(PyObject *); /*proto*/
                                                ^
pyarrow_cython_example.cpp:4765:52: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_coo_tensor' [-Wunused-variable]
static std::shared_ptr< arrow::SparseCOOTensor>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_coo_tensor)(PyObject *); /*proto*/
                                                   ^
pyarrow_cython_example.cpp:4766:52: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csc_matrix' [-Wunused-variable]
static std::shared_ptr< arrow::SparseCSCMatrix>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csc_matrix)(PyObject *); /*proto*/
                                                   ^
pyarrow_cython_example.cpp:4767:52: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csf_tensor' [-Wunused-variable]
static std::shared_ptr< arrow::SparseCSFTensor>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csf_tensor)(PyObject *); /*proto*/
                                                   ^
pyarrow_cython_example.cpp:4768:52: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csr_matrix' [-Wunused-variable]
static std::shared_ptr< arrow::SparseCSRMatrix>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_sparse_csr_matrix)(PyObject *); /*proto*/
                                                   ^
pyarrow_cython_example.cpp:4769:43: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_tensor' [-Wunused-variable]
static std::shared_ptr< arrow::Tensor>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_tensor)(PyObject *); /*proto*/
                                          ^
pyarrow_cython_example.cpp:4770:48: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_batch' [-Wunused-variable]
static std::shared_ptr< arrow::RecordBatch>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_batch)(PyObject *); /*proto*/
                                               ^
pyarrow_cython_example.cpp:4771:42: warning: unused variable '__pyx_f_7pyarrow_3lib_pyarrow_unwrap_table' [-Wunused-variable]
static std::shared_ptr< arrow::Table>  (*__pyx_f_7pyarrow_3lib_pyarrow_unwrap_table)(PyObject *); /*proto*/
                                         ^
24 warnings generated.
ld: warning: ignoring file '/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/install/lib/libarrow.1500.0.0.dylib': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/apache-arrow-15.0.0/python/pyarrow/libarrow_python.dylib': found architecture 'x86_64', required architecture 'arm64'
dyld[72672]: Symbol not found: _Py_Initialize
  Referenced from: <B2C18537-4A92-3012-A067-DC43E7CEC51B> /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/bin/python3.9
  Expected in:     <B2C18537-4A92-3012-A067-DC43E7CEC51B> /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/bin/python3.9

I'm not really familiar with these Python tests and this is my first time trying to verify Python for a RC. So it is possible that I did something wrong with my env setup.

kou commented 8 months ago

Thanks. Architecture may be related:

ld: warning: ignoring file '/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/install/lib/libarrow.1500.0.0.dylib': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/private/var/folders/kd/xcyg9h791l181yj3ltm5syph0000gn/T/arrow-15.0.0.XXXXX.7YGOHf0MrG/apache-arrow-15.0.0/python/pyarrow/libarrow_python.dylib': found architecture 'x86_64', required architecture 'arm64'

@pitrou or @jorisvandenbossche may know why.

raulcd commented 8 months ago

This is weird indeed. @zanmato1984 you are using macOS M1 right?

zanmato1984 commented 8 months ago

This is weird indeed. @zanmato1984 you are using macOS M1 right?

No, this was on my Intel Mac.

kou commented 8 months ago

It may be related.

The log shows that you tried to create universal (that includes x86_64 and arm64 binary) binary:

creating build/temp.macosx-10.9-universal2-cpython-39

But we don't need to create universal binary for the test. Because the test just wants to test Cython integration.

We may need to specify _PYTHON_HOST_PLATFORM like https://github.com/apache/arrow/blob/55afcf0450aa2b611e78335bdbfd77e55ae3bc9f/ci/scripts/python_wheel_macos_build.sh#L36 to prevent building universal binary.