OvertureMaps / schema

Overture Maps Schema
http://docs.overturemaps.org/schema
Creative Commons Attribution 4.0 International
145 stars 6 forks source link

Reading the data using C++ #164

Open jakobvinkas opened 7 months ago

jakobvinkas commented 7 months ago

I have a parquet file containing transportation information that I have been able to parse in python and extract the needed data as follows:

import pandas as pd
import shapely.wkb
import shapely.geometry
import matplotlib.pyplot as plt
import utm

df = pd.read_parquet("lanes_small.parquet", engine="fastparquet")

plt.figure()
for index, object in df.iterrows():
    if not "segment" in object["id"]:
        continue
    linestring = shapely.wkb.loads(object["geometry"])
    coordinates = shapely.geometry.mapping(linestring)["coordinates"]
    xy_coordinates = [utm.from_latlon(lat, lon) for lon, lat in coordinates]
    plt.plot(
        [xy_coord[1] for xy_coord in xy_coordinates],
        [xy_coord[0] for xy_coord in xy_coordinates],
    )

plt.show()

But I am unable to do the same in C++, I have tried about 100 different ways but none of them work, maybe the closest I have gotten is this:

arrow::MemoryPool* pool = arrow::default_memory_pool();
std::shared_ptr<arrow::io::RandomAccessFile> input;
input = *arrow::io::ReadableFile::Open(path);

std::unique_ptr<parquet::arrow::FileReader> arrow_reader;
arrow::Status open_status = parquet::arrow::OpenFile(input, pool, &arrow_reader);
if(!open_status.ok()){
    std::cout << "Failed open: " << open_status.ToString();
}

std::shared_ptr<arrow::Table> table;
arrow::Status read_table_status = arrow_reader->ReadTable(&table);
if(!read_table_status.ok()){
    std::cout << "Failed read: " << read_table_status.ToString() << std::endl;
}

Which ends with Failed read: NotImplemented: Support for codec 'snappy' not built

And I have added snap to the conan project:

[requires]
snappy/1.2.0
arrow/15.0.0

[options]
arrow/*:parquet=True
arrow/*:with_boost=True
arrow/*:with_thrift=True
arrow/*:snappy=True

And even then I have no idea what I am supposed to get out from the table and the documentation is of little help.

Any input of what I should do?

jakobvinkas commented 7 months ago

Could anyone point in a direction here?