JuliaGeo / Shapefile.jl

Parsing .shp files in Julia
http://juliageo.org/Shapefile.jl/
MIT License
82 stars 14 forks source link

Lazy loading #76

Open rafaqz opened 1 year ago

rafaqz commented 1 year ago

A key advantage of using native julia code instead of GDAL is we can do everything lazily.

Recently using GeoJSON.jl with Rasters.jl I found it effortlessly handled rasterizing 3GB files that crashed GDAL. The same should be true of this package (it likely already is in some cases, but we can make it a lot more so).

I've spoken with @visr abouit lazy loading here, but making this issue to track implementation.

The idea would be to eagerly load shape metadata like bounds and type, and lazily load the point vectors.

(And although this is an old format disciplines like ecology are totally dependent on it, often with multi-GB files)