ancapdev / LightBSON.jl

High performance encoding and decoding of BSON data in Julia
MIT License
20 stars 4 forks source link

partial parsing #28

Closed poncito closed 1 year ago

poncito commented 1 year ago

I'd like to be able to partially parse a document. I would need to read a header before knowing the type I should use to parse the payload. Partial parsing would allow me to avoid type instabillity.

struct MyStruct{T}
    field_encoding_the_type::Int
    payload::T
end

x = reader[MyStruct{AbstractBSONReader}]
if x.field_encoding_the_type == 0
    payload = x.payload[T] # using the corresponding type
    ...
elseif....
end
codecov-commenter commented 1 year ago

Codecov Report

Merging #28 (78f582c) into master (69e129a) will decrease coverage by 1.04%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master      #28      +/-   ##
==========================================
- Coverage   89.88%   88.84%   -1.05%     
==========================================
  Files          13       13              
  Lines         811      753      -58     
==========================================
- Hits          729      669      -60     
- Misses         82       84       +2     
Flag Coverage Δ
unittest 88.84% <100.00%> (-1.05%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/reader.jl 91.08% <100.00%> (-0.73%) :arrow_down:
src/object_id.jl 80.00% <0.00%> (-13.03%) :arrow_down:
src/indexed_reader.jl 92.00% <0.00%> (-4.00%) :arrow_down:
src/validator.jl 94.44% <0.00%> (-0.91%) :arrow_down:
src/writer.jl 88.38% <0.00%> (-0.44%) :arrow_down:
src/index.jl 94.44% <0.00%> (-0.43%) :arrow_down:
src/convenience.jl 79.31% <0.00%> (+2.64%) :arrow_up:
src/representations.jl 93.18% <0.00%> (+4.05%) :arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

poncito commented 1 year ago
julia> struct S{T}
           header::Int
           payload::T
       end

       testf(reader) = reader[S{typeof(reader)}].header

       buf = UInt8[]
       writer = BSONWriter(buf)
       writer[] = (; header = 1, payload = 45)
       close(writer)

       reader = BSONReader(buf)
       @benchmark testf($reader)

BenchmarkTools.Trial: 10000 samples with 997 evaluations.
 Range (min … max):  20.186 ns … 46.640 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     20.270 ns              ┊ GC (median):    0.00%
 Time  (mean ± σ):   20.492 ns ±  0.879 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▄█▇▇▅▃▄▁       ▁▃▁▂▁ ▁           ▂▁ ▁▁                      ▂
  ████████▅██▅██▄█████▇██▅▇▆▄▇▇▅▆█▆██▆██▇██▆█▇▅▆▅▄▄▅▃▆▅▁▃▄▁▁▃ █
  20.2 ns      Histogram: log(frequency) by time      22.7 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.
poncito commented 1 year ago

Hi @ancapdev, Would you have time to look into this small feature? Thank you,

ancapdev commented 1 year ago

Hi @ancapdev, Would you have time to look into this small feature? Thank you,

Been a little snowed under, but this looks good to me, just a tiny formatting change

ancapdev commented 1 year ago

https://github.com/JuliaRegistries/General/pull/74027