geopandas / pyogrio

Vectorized vector I/O using OGR
https://pyogrio.readthedocs.io
MIT License
272 stars 22 forks source link

BUG: integer overflow when reading int64 column #259

Closed theroggy closed 1 year ago

theroggy commented 1 year ago

When reading a .gpkg file with an integer column with values > the max 32 bit int value (2.147.483.647), those values overflow to negative values when read.

The use_arrow=True code path doesn't have this issue.

Code to reproduce behaviour

from pathlib import Path

import geopandas as gpd
import pyogrio
import shapely

script_dir = Path(__file__).resolve().parent
path = script_dir / "int_overflow_tmp.gpkg"
gdf = gpd.GeoDataFrame(
    data={"overflowing_int": [2192502720]}, geometry=[shapely.Point(0,1)], crs="epsg:31370"
)
pyogrio.write_dataframe(gdf, path)

gdf = pyogrio.read_dataframe(path, use_arrow=False)
print(f"with use_arrow=False:\n{gdf}")
gdf = pyogrio.read_dataframe(path, use_arrow=True)
print(f"with use_arrow=True:\n{gdf}")