apache / arrow-cookbook

Apache Arrow Cookbook
https://arrow.apache.org/
Apache License 2.0
95 stars 46 forks source link

[Python] Python cookbooks are failing against current development #331

Closed raulcd closed 10 months ago

raulcd commented 10 months ago

After updating the release version to 14.0.0 and the development to 15 dev the development cookbooks for arrow seem to be failing due to the following:

File "schema.rst", line 407, in default
Failed example:
    third_schema = pa.schema([
        ("country_code", pa.int32()),
        ("lat", pa.float32()),
        ("long", pa.float32()),
    ])

    try:
        union_schema =  pa.unify_schemas([first_schema, second_schema, third_schema])
    except pa.ArrowInvalid as e:
        print(e)
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python3.10/doctest.py", line 1350, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest default[0]>", line 8, in <module>
        union_schema =  pa.unify_schemas([first_schema, second_schema, third_schema])
      File "pyarrow/types.pxi", line 3326, in pyarrow.lib.unify_schemas
      File "pyarrow/error.pxi", line 154, in pyarrow.lib.pyarrow_internal_check_status
      File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status
    pyarrow.lib.ArrowTypeError: Unable to merge: Field country_code has incompatible types: string vs int32

See: https://github.com/apache/arrow-cookbook/actions/runs/6788921789/job/18454962099

raulcd commented 10 months ago

cc @AlenkaF @jorisvandenbossche

AlenkaF commented 10 months ago

The example is failing due to the change in the error raised by pa.unify_schemas. The change happened in https://github.com/apache/arrow/pull/36846/files and is now raising ArrowTypeError instead of ArrowInvalid. Will make a PR to update the Cookbook example.