Crash of modal due to pyarrow incompatibility

[OrfeasBourchas]

I am currently trying to run the codes for the Data Validation in Production ML Pipelines course, and I run into the following problem both on my local machine and on the modal remote. I think it is the latest version of python+the pyarrow that cause this problem.

  running build_ext
      creating /tmp/pip-install-y2x7mnua/pyarrow_11f173a8029a4b4aafed72e11e381502/build/temp.linux-x86_64-cpython-312
      -- Running cmake for PyArrow
      cmake -DCMAKE_INSTALL_PREFIX=/tmp/pip-install-y2x7mnua/pyarrow_11f173a8029a4b4aafed72e11e381502/build/lib.linux-x86_64-cpython-312/pyarrow -DPYTHON_EXECUTABLE=/usr/local/bin/python -DPython3_EXECUTABLE=/usr/local/bin/python -DPYARROW_CXXFLAGS= -DPYARROW_BUILD_CUDA=off -DPYARROW_BUILD_SUBSTRAIT=off -DPYARROW_BUILD_FLIGHT=off -DPYARROW_BUILD_GANDIVA=off -DPYARROW_BUILD_DATASET=off -DPYARROW_BUILD_ORC=off -DPYARROW_BUILD_PARQUET=off -DPYARROW_BUILD_PARQUET_ENCRYPTION=off -DPYARROW_BUILD_PLASMA=off -DPYARROW_BUILD_GCS=off -DPYARROW_BUILD_S3=off -DPYARROW_BUILD_HDFS=off -DPYARROW_USE_TENSORFLOW=off -DPYARROW_BUNDLE_ARROW_CPP=off -DPYARROW_BUNDLE_BOOST=off -DPYARROW_BUNDLE_CYTHON_CPP=off -DPYARROW_BUNDLE_PLASMA_EXECUTABLE=on -DPYARROW_GENERATE_COVERAGE=off -DPYARROW_BOOST_USE_SHARED=on -DPYARROW_PARQUET_USE_SHARED=on -DCMAKE_BUILD_TYPE=release /tmp/pip-install-y2x7mnua/pyarrow_11f173a8029a4b4aafed72e11e381502
      error: command 'cmake' failed: No such file or directory
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.

May be changing to fixed pyarrow might solve the problem as during installation I noticed that it was requiring a version between >= 11.0.0 and <12.0.0

[shreyashankar]

Thanks for flagging! will update pyarrow