search

wcjochem / sfarrow

R package for reading/writing `sf` objects from/to parquet files with `arrow`.
https://wcjochem.github.io/sfarrow/
Other
75 stars 4 forks source link

error in st_read_parquet when geometry is not in col_select #8

Closed floriandeboissieu closed 3 years ago

floriandeboissieu commented 3 years ago

Many thanks for this very useful package!

In st_read_parquet, when the geometry column is not in col_select it fails:

Error in arrow_to_sf(tbl, geo) : Malformed file and geo metatdata.

As the geometry column is kind of special here, could it be possible to detect and include it, even if not in col_select?

wcjochem commented 3 years ago

Hi,

Thanks for the raising this issue. I was being strict when I set this check. If a user is not reading in a geometry field, then it will probably be more efficient to use arrow. It would be possible to detect and add any geometry cols to the selection. I can see where that might be useful. I'll consider adding an option to allow that. And I'll at least document this a bit better in st_read_parquet.

Thanks.

wcjochem commented 3 years ago

@floriandeboissieu I'm testing out a find_geom parameter which should add the functionality to find and add any geometry columns to the selected columns.

floriandeboissieu commented 3 years ago

@wcjochem sorry I forgot to answer to your previous message. thanks for your first answer and new developments. I don't know why I had in mind that using arrow on files written with sfarrow was giving bad structure even when removing the geometry column, but actually it is working perfectly, so it is solving half of the issue. Thanks for considering the option, that will be really useful.