quinnj / JSON3.jl

Other
214 stars 47 forks source link

Reading multiple Objects in jsonlines format #171

Closed jfb-h closed 3 years ago

jfb-h commented 3 years ago

Im struggling to read an array of custom type instances from a .jsonl file.

Let's say you have a file looking like this:

{"a":2,"b":"x"}
{"a":3,"b":"y"}

and a struct like this:

struct Test
    a::Int
    b::String
end

 StructTypes.StructType(::Type{Test}) = StructTypes.Struct()

Then reading the file like this:

JSON3.read(read("data/structtypes_test.jsonl"), Test, jsonlines=true)

results in only the first object being read, while reading it into an array, like so:

JSON3.read(read("data/structtypes_test.jsonl"), JSON3.Array{Test}, jsonlines=true)

results in the following error message:

julia> JSON3.read(read("data/structtypes_test.jsonl"), JSON3.Array{Test}, jsonlines=true)
ERROR: ArgumentError: invalid JSON at byte position 1 while parsing type JSON3.Array{Test, S, TT} where {S<:AbstractVector{UInt8}, TT<:AbstractVector{UInt64}}: ExpectedOpeningArrayChar
{"a":2,"b":"x"}
{"a":3,"b

Is this a bug or am I just doing something wrong?

mcmcgrath13 commented 3 years ago

It looks like the jsonlines keyword is currently only implemented for the JSON3.read(str; jsonlines=true) method and not reading into a type. @quinnj should it be implemented?

jfb-h commented 3 years ago

Thanks for the heads up! I guess I can just readlines() and then parse each line individually but having the keyword supported directly for the type method would certainly be convenient.

quinnj commented 3 years ago

Hmmm, yes, I think this should probably work. I don't remember if I looked into it specifically when we implemented jsonlines for the non-typed case. I'll take a look.

kmsquire commented 3 years ago

I created a fix for this in #178