search

apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.
https://beam.apache.org/
Apache License 2.0
7.9k stars 4.27k forks source link

[Bug]: Error installing Beam in Python 3.11 #26458

Closed davidcavazos closed 1 year ago

davidcavazos commented 1 year ago

What happened?

Installing apache-beam on a fresh virtual environment fails with Python 3.11.

Python version: 3.11.3 Pip version: 23.1.2

Steps to reproduce

python3.11 -m venv env
source env/bin/activate

pip install -U pip
pip install apache-beam

There's an error building pyarrow. Here's the (truncated) output:

Click to show command outputs ``` Collecting pyarrow<10.0.0,>=3.0.0 (from apache-beam) Using cached pyarrow-9.0.0.tar.gz (873 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam) Using cached docopt-0.6.2-py2.py3-none-any.whl Collecting six>=1.9.0 (from hdfs<3.0.0,>=2.1.0->apache-beam) Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) Collecting pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 (from httplib2<0.22.0,>=0.8->apache-beam) Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB) Collecting charset-normalizer<4,>=2 (from requests<3.0.0,>=2.24.0->apache-beam) Using cached charset_normalizer-3.1.0-cp311-cp311-macosx_11_0_arm64.whl (121 kB) Collecting idna<4,>=2.5 (from requests<3.0.0,>=2.24.0->apache-beam) Using cached idna-3.4-py3-none-any.whl (61 kB) Collecting urllib3<1.27,>=1.21.1 (from requests<3.0.0,>=2.24.0->apache-beam) Using cached urllib3-1.26.15-py2.py3-none-any.whl (140 kB) Collecting certifi>=2017.4.17 (from requests<3.0.0,>=2.24.0->apache-beam) Using cached certifi-2022.12.7-py3-none-any.whl (155 kB) Building wheels for collected packages: pyarrow Building wheel for pyarrow (pyproject.toml) ... error error: subprocess-exited-with-error × Building wheel for pyarrow (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [215 lines of output] running bdist_wheel running build running build_py creating build creating build/lib.macosx-11.1-arm64-cpython-311 creating build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/orc.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/conftest.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_generated_version.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/benchmark.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_compute_docstrings.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/ipc.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/util.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/flight.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/cffi.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/substrait.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/filesystem.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/__init__.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/plasma.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/types.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/dataset.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/cuda.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/feather.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/pandas_compat.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/fs.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/csv.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/jvm.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/hdfs.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/json.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/serialization.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/compute.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow creating build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_tensor.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_ipc.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/conftest.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_convert_builtin.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_misc.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/arrow_16597.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_gandiva.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/strategies.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_adhoc_memory_leak.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/arrow_7980.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/util.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_orc.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_table.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_array.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_deprecations.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_serialization_deprecated.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/__init__.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_io.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_util.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_cuda_numba_interop.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_cffi.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_schema.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_jvm.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_plasma_tf_op.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_fs.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_udf.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_filesystem.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/pandas_threaded_import.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/pandas_examples.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_cython.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_sparse_tensor.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_dataset.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_builder.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_cuda.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_extension_type.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_feather.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_pandas.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_memory.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_exec_plan.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_flight.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/read_record_batch.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_json.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_serialization.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_compute.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_hdfs.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/deserialize_buffer.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_strategies.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_csv.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_plasma.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_scalars.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_gdb.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_types.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/test_substrait.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests creating build/lib.macosx-11.1-arm64-cpython-311/pyarrow/vendored copying pyarrow/vendored/version.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/vendored copying pyarrow/vendored/__init__.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/vendored copying pyarrow/vendored/docscrape.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/vendored creating build/lib.macosx-11.1-arm64-cpython-311/pyarrow/parquet copying pyarrow/parquet/encryption.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/parquet copying pyarrow/parquet/__init__.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/parquet creating build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/parquet copying pyarrow/tests/parquet/test_basic.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/parquet copying pyarrow/tests/parquet/conftest.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/parquet copying pyarrow/tests/parquet/encryption.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/parquet copying pyarrow/tests/parquet/test_parquet_writer.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/parquet copying pyarrow/tests/parquet/test_metadata.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/parquet copying pyarrow/tests/parquet/__init__.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/parquet copying pyarrow/tests/parquet/test_datetime.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/parquet copying pyarrow/tests/parquet/common.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/parquet copying pyarrow/tests/parquet/test_dataset.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/parquet copying pyarrow/tests/parquet/test_data_types.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/parquet copying pyarrow/tests/parquet/test_pandas.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/parquet copying pyarrow/tests/parquet/test_parquet_file.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/parquet copying pyarrow/tests/parquet/test_compliant_nested_type.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/parquet copying pyarrow/tests/parquet/test_encryption.py -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/parquet running egg_info writing pyarrow.egg-info/PKG-INFO writing dependency_links to pyarrow.egg-info/dependency_links.txt writing entry points to pyarrow.egg-info/entry_points.txt writing requirements to pyarrow.egg-info/requires.txt writing top-level names to pyarrow.egg-info/top_level.txt listing git files failed - pretending there aren't any reading manifest file 'pyarrow.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching '../LICENSE.txt' warning: no files found matching '../NOTICE.txt' warning: no previously-included files matching '*.so' found anywhere in distribution warning: no previously-included files matching '*.pyc' found anywhere in distribution warning: no previously-included files matching '*~' found anywhere in distribution warning: no previously-included files matching '#*' found anywhere in distribution warning: no previously-included files matching '.git*' found anywhere in distribution warning: no previously-included files matching '.DS_Store' found anywhere in distribution no previously-included directories found matching '.asv' writing manifest file 'pyarrow.egg-info/SOURCES.txt' copying pyarrow/__init__.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_compute.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_compute.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_csv.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_csv.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_cuda.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_cuda.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_dataset.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_dataset.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_dataset_orc.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_dataset_parquet.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_exec_plan.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_feather.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_flight.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_fs.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_fs.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_gcsfs.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_hdfs.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_hdfsio.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_json.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_orc.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_orc.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_parquet.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_parquet.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_parquet_encryption.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_parquet_encryption.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_plasma.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_s3fs.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/_substrait.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/array.pxi -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/benchmark.pxi -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/builder.pxi -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/compat.pxi -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/config.pxi -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/error.pxi -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/gandiva.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/io.pxi -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/ipc.pxi -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/lib.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/lib.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/memory.pxi -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/pandas-shim.pxi -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/public-api.pxi -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/scalar.pxi -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/serialization.pxi -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/table.pxi -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/tensor.pxi -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow copying pyarrow/types.pxi -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow creating build/lib.macosx-11.1-arm64-cpython-311/pyarrow/includes copying pyarrow/includes/common.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/includes copying pyarrow/includes/libarrow_feather.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/includes copying pyarrow/includes/__init__.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/includes copying pyarrow/includes/libgandiva.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/includes copying pyarrow/includes/libarrow_python.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/includes copying pyarrow/includes/libarrow_flight.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/includes copying pyarrow/includes/libarrow_dataset_parquet.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/includes copying pyarrow/includes/libarrow_dataset.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/includes copying pyarrow/includes/libarrow_substrait.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/includes copying pyarrow/includes/libarrow.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/includes copying pyarrow/includes/libarrow_cuda.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/includes copying pyarrow/includes/libarrow_fs.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/includes copying pyarrow/includes/libplasma.pxd -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/includes copying pyarrow/tests/bound_function_visit_strings.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests copying pyarrow/tests/pyarrow_cython_example.pyx -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests creating build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tensorflow copying pyarrow/tensorflow/plasma_op.cc -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tensorflow creating build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data creating build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/feather copying pyarrow/tests/data/feather/v0.17.0.version.2-compression.lz4.feather -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/feather creating build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/orc copying pyarrow/tests/data/orc/README.md -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/orc copying pyarrow/tests/data/orc/TestOrcFile.emptyFile.jsn.gz -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/orc copying pyarrow/tests/data/orc/TestOrcFile.emptyFile.orc -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/orc copying pyarrow/tests/data/orc/TestOrcFile.test1.jsn.gz -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/orc copying pyarrow/tests/data/orc/TestOrcFile.test1.orc -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/orc copying pyarrow/tests/data/orc/TestOrcFile.testDate1900.jsn.gz -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/orc copying pyarrow/tests/data/orc/TestOrcFile.testDate1900.orc -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/orc copying pyarrow/tests/data/orc/decimal.jsn.gz -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/orc copying pyarrow/tests/data/orc/decimal.orc -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/orc creating build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/parquet copying pyarrow/tests/data/parquet/v0.7.1.all-named-index.parquet -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/parquet copying pyarrow/tests/data/parquet/v0.7.1.column-metadata-handling.parquet -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/parquet copying pyarrow/tests/data/parquet/v0.7.1.parquet -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/parquet copying pyarrow/tests/data/parquet/v0.7.1.some-named-index.parquet -> build/lib.macosx-11.1-arm64-cpython-311/pyarrow/tests/data/parquet running build_ext creating /private/var/folders/kr/n7z09n6x1w9425ztkx73nlbc00dsyy/T/pip-install-ex1inbin/pyarrow_d776e1b164cb465384f22889a10cd1df/build/temp.macosx-11.1-arm64-cpython-311 -- Running cmake for pyarrow cmake -DPYTHON_EXECUTABLE=/Users/dcavazos/src/sandbox/env/bin/python3.11 -DPython3_EXECUTABLE=/Users/dcavazos/src/sandbox/env/bin/python3.11 "" -DPYARROW_BUILD_CUDA=off -DPYARROW_BUILD_SUBSTRAIT=off -DPYARROW_BUILD_FLIGHT=off -DPYARROW_BUILD_GANDIVA=off -DPYARROW_BUILD_DATASET=off -DPYARROW_BUILD_ORC=off -DPYARROW_BUILD_PARQUET=off -DPYARROW_BUILD_PARQUET_ENCRYPTION=off -DPYARROW_BUILD_PLASMA=off -DPYARROW_BUILD_GCS=off -DPYARROW_BUILD_S3=off -DPYARROW_BUILD_HDFS=off -DPYARROW_USE_TENSORFLOW=off -DPYARROW_BUNDLE_ARROW_CPP=off -DPYARROW_BUNDLE_BOOST=off -DPYARROW_GENERATE_COVERAGE=off -DPYARROW_BOOST_USE_SHARED=on -DPYARROW_PARQUET_USE_SHARED=on -DCMAKE_BUILD_TYPE=release /private/var/folders/kr/n7z09n6x1w9425ztkx73nlbc00dsyy/T/pip-install-ex1inbin/pyarrow_d776e1b164cb465384f22889a10cd1df error: command 'cmake' failed: No such file or directory [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for pyarrow Failed to build pyarrow ERROR: Could not build wheels for pyarrow, which is required to install pyproject.toml-based projects ```

Installing pyarrow by itself works, and even with it pre-installed, apache-beam still cannot install.

Issue Priority

Priority: 1 (data loss / total loss of function)

Issue Components

davidcavazos commented 1 year ago

FYI: @tvalentyn you might have some insights on this

davidcavazos commented 1 year ago

It looks like cmake is a new system dependency, is this right?

But installations on prior Python versions work. Is there any reason we can't use the prebuilt pyarrow packages?

Abacn commented 1 year ago

pip install apache-beam installs 2.36.0 for now which does not support Python 3.11. The first beam version supports Python 3.11 is 2.47.0 which has not yet released. You can install the release candidate with pip install apache-beam==2.47.0rc1

tvalentyn commented 1 year ago

+1 to @Abacn - @davidcavazos could you give it a try? Thanks!

davidcavazos commented 1 year ago

Yes, it looks like 2.47.0rc1 works. Thanks!