search

GoogleCloudPlatform / python-docs-samples

Code samples used on cloud.google.com
Apache License 2.0
7.4k stars 6.41k forks source link

bigquery.pandas-gbq-migration.samples_test: test_client_library_upload_from_dataframe failed #5313

Closed flaky-bot[bot] closed 3 years ago

flaky-bot[bot] commented 3 years ago

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and I will stop commenting.

commit: c3a65998973d7010d0f77edb84034f05ed40adeb buildURL: Build Status, Sponge status: failed

Test output
Traceback (most recent call last):
  File "/workspace/bigquery/pandas-gbq-migration/samples_test.py", line 231, in test_client_library_upload_from_dataframe
    df, table_id, job_config=job_config
  File "/workspace/bigquery/pandas-gbq-migration/.nox/py-3-7/lib/python3.7/site-packages/google/cloud/bigquery/client.py", line 2333, in load_table_from_dataframe
    parquet_compression=parquet_compression,
  File "/workspace/bigquery/pandas-gbq-migration/.nox/py-3-7/lib/python3.7/site-packages/google/cloud/bigquery/_pandas_helpers.py", line 472, in dataframe_to_parquet
    arrow_table = dataframe_to_arrow(dataframe, bq_schema)
  File "/workspace/bigquery/pandas-gbq-migration/.nox/py-3-7/lib/python3.7/site-packages/google/cloud/bigquery/_pandas_helpers.py", line 436, in dataframe_to_arrow
    bq_to_arrow_array(get_column_or_index(dataframe, bq_field.name), bq_field)
  File "/workspace/bigquery/pandas-gbq-migration/.nox/py-3-7/lib/python3.7/site-packages/google/cloud/bigquery/_pandas_helpers.py", line 218, in bq_to_arrow_array
    return pyarrow.Array.from_pandas(series, type=arrow_type)
  File "pyarrow/array.pxi", line 887, in pyarrow.lib.Array.from_pandas
  File "pyarrow/array.pxi", line 292, in pyarrow.lib.array
  File "pyarrow/array.pxi", line 83, in pyarrow.lib._ndarray_to_array
  File "pyarrow/error.pxi", line 107, in pyarrow.lib.check_status
pyarrow.lib.ArrowTypeError: Did not pass numpy.dtype object
flaky-bot[bot] commented 3 years ago

Looks like this issue is flaky. :worried:

I'm going to leave this open and stop commenting.

A human should fix and close this.

When run at the same commit (c3a65998973d7010d0f77edb84034f05ed40adeb), this test passed in one build (Build Status, Sponge) and failed in another build (Build Status, Sponge).

shollyman commented 3 years ago

Looks like this broke due to the bump to pyarrow 3? Compatability problem in the veneer?

tswast commented 3 years ago

Tests pass locally. Unable to reproduce.

(nox) swast@tims-cloud.c ~/src/python-docs-samples/bigquery/pandas-gbq-migration% mkdir -p .nox/py-3-7
(nox) swast@tims-cloud.c ~/src/python-docs-samples/bigquery/pandas-gbq-migration% ~/bin/python3.7 -m venv .nox/py-3-7
(nox) swast@tims-cloud.c ~/src/python-docs-samples/bigquery/pandas-gbq-migration% nox -s py-3.7 -r
nox > Running session py-3.7
nox > Re-using existing virtual environment at .nox/py-3-7.
nox > python -m pip install -r requirements.txt
nox > python -m pip install -r requirements-test.txt
nox > pytest --junitxml=sponge_log.xml
==================================================== test session starts =====================================================
platform linux -- Python 3.7.9, pytest-6.2.1, py-1.10.0, pluggy-0.13.1 -- /usr/local/google/home/swast/src/python-docs-samples/bigquery/pandas-gbq-migration/.nox/py-3-7/bin/python
cachedir: .pytest_cache
rootdir: /usr/local/google/home/swast/src/python-docs-samples, configfile: pytest.ini
collected 10 items

samples_test.py::test_client_library_query PASSED                                                                      [ 10%]
samples_test.py::test_pandas_gbq_query PASSED                                                                          [ 20%]
samples_test.py::test_client_library_query_bqstorage PASSED                                                            [ 30%]
samples_test.py::test_pandas_gbq_query_bqstorage PASSED                                                                [ 40%]
samples_test.py::test_client_library_legacy_query PASSED                                                               [ 50%]
samples_test.py::test_pandas_gbq_legacy_query PASSED                                                                   [ 60%]
samples_test.py::test_client_library_query_with_parameters PASSED                                                      [ 70%]
samples_test.py::test_pandas_gbq_query_with_parameters PASSED                                                          [ 80%]
samples_test.py::test_client_library_upload_from_dataframe PASSED                                                      [ 90%]
samples_test.py::test_pandas_gbq_upload_from_dataframe PASSED                                                          [100%]

====================================================== warnings summary ======================================================
bigquery/pandas-gbq-migration/samples_test.py::test_pandas_gbq_upload_from_dataframe
  /usr/local/google/home/swast/src/python-docs-samples/bigquery/pandas-gbq-migration/.nox/py-3-7/lib/python3.7/site-packages/pandas_gbq/gbq.py:1283: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = self.client.dataset(self.dataset_id).table(table_id)

bigquery/pandas-gbq-migration/samples_test.py::test_pandas_gbq_upload_from_dataframe
  /usr/local/google/home/swast/src/python-docs-samples/bigquery/pandas-gbq-migration/.nox/py-3-7/lib/python3.7/site-packages/pandas_gbq/gbq.py:1390: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    self.client.get_dataset(self.client.dataset(dataset_id))

bigquery/pandas-gbq-migration/samples_test.py::test_pandas_gbq_upload_from_dataframe
  /usr/local/google/home/swast/src/python-docs-samples/bigquery/pandas-gbq-migration/.nox/py-3-7/lib/python3.7/site-packages/pandas_gbq/gbq.py:1320: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    table_ref = self.client.dataset(self.dataset_id).table(table_id)

bigquery/pandas-gbq-migration/samples_test.py::test_pandas_gbq_upload_from_dataframe
  /usr/local/google/home/swast/src/python-docs-samples/bigquery/pandas-gbq-migration/.nox/py-3-7/lib/python3.7/site-packages/pandas_gbq/load.py:59: PendingDeprecationWarning: Client.dataset is deprecated and will be removed in a future version. Use a string like 'my_project.my_dataset' or a cloud.google.bigquery.DatasetReference object, instead.
    destination_table = client.dataset(dataset_id).table(table_id)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
--- generated xml file: /usr/local/google/home/swast/src/python-docs-samples/bigquery/pandas-gbq-migration/sponge_log.xml ----
========================================= 10 passed, 4 warnings in 65.74s (0:01:05) ==========================================
nox > Session py-3.7 was successful.