anelendata / tap-bigquery

Singer.io tap for extracting data from BigQuery tables
Apache License 2.0
16 stars 33 forks source link

Pin numpy to "<2" #30

Closed AlejandroUPC closed 4 months ago

AlejandroUPC commented 4 months ago

Description

Using meltano==3.4.0 and installing the tap as defined in the meltano labs page here results in an error (had to cut off some verbose):

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.
ImportError: numpy.core.multiarray failed to import

How to reproduce

Using:

Steps:

  1. Initialize a pyproject (I do not think using pyproject is really relevant here, any dependency will do) and install meltano==3.4.0
  2. Add the bq tap as described in the meltano labs page meltano add extractor tap-bigquery
  3. Create a valid config and run the command meltano config tap-bigquery test

Current work around

Requieres manual intervation and a bit hacky, go into the virtual environment where meltano installed the tap and downgrade numpy:

  1. pip uninstall numpy
  2. pip install "numpy<2"
  3. Run the command again meltano config tap-bigquery test and it should work.

Proposed solution

Pin numpy to < 2 on the main branch.

AlejandroUPC commented 4 months ago

merged here, thank you @daigotanaka