Open rafaqz opened 1 year ago
Specifically, it's slow to go from JSON3 Array into a nested type like Vector{Vector{Vector{Float64}}} (sometimes deeper).
Reading directly into a struct is fast, but loses the laziness. Ideally we load lazily and then convert subsets of the JSON 3 Object/Array into known types directly.
Can we just call read
on the codepoints in the tape
at the leaf?
Maybe allowing something like JSON3.read(A::JSON.Array, Vector{Tuple{Float64,Float64}})
with these leaf arrays would be possible?
From what I gathered; given an JSON3.Array
A
, A.tape[A.inds[begin]:A[A.inds[end]]
would give you the tape for that array, but it's unclear to me how to parse the tape to find the correct indices into A.buf
. It could just take a single utility function.
In GeoJSON.jl we are loading JSON lazily, which is great over large files.
But the final vectors of points are loaded as arrays of length 2/3 arrays, which is very slow. We need to load these as length 2/3 Tuples instead.
Is it possible to read a
JSON3.Array
forcing the eltype to beTuple{Float64,Float64}
? Just loading the whole array at that point asVector{Tuple{Float64,Float64}}
.