search

Nadrieril / dhall-rust

Maintainable configuration files, for Rust users
Other
303 stars 27 forks source link

Deserialize binary dhall? #199

Closed bsaul closed 3 years ago

bsaul commented 3 years ago

(not sure if this the appropriate place to ask)

Do the dhall or serde_dhall crates support reading binary-encoded dhall files? It seems like the dhall crate supports encoding/decoding, but I couldn't find any examples of reading from a binary.

Nadrieril commented 3 years ago

It does, but I haven't exposed it in the public API because it's meant to be mostly reserved for caching. Do you have a use-case for it?

bsaul commented 3 years ago

Do you have a use-case for it?

My dhall configuration files could potentially be large-ish, and I was looking for way to store and transmit smaller versions. I was also curious if the binary version would marshal into my application faster.

Nadrieril commented 3 years ago

This is available in v0.9.0!

bsaul commented 3 years ago

Awesome! Thanks!

On Nov 20, 2020, at 5:57 PM, Nadrieril notifications@github.com wrote:

 This is available in v0.9.0!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

Nadrieril commented 3 years ago

I'd be curious to know if you get a noticeable speedup, let me know if you measure it

bsaul commented 3 years ago

I ran a quick test using criterion. In the results below, Dhall/dhall is benchmarking importing a single 27KB dhall file that represents a very simple use case for my application. Dhall/dhallb benchmarks importing the encoded version (7KB) of that same file.

Dhall/dhall             time:   [3.7877 s 3.8678 s 3.9583 s]                           
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe
Dhall/dhallb            time:   [10.662 ms 11.189 ms 11.811 ms]                         
Found 10 outliers among 100 measurements (10.00%)
  2 (2.00%) high mild
  8 (8.00%) high severe

If I did the tests correctly, that's quite dramatic (3.86s vs 11.19ms).

Nadrieril commented 3 years ago

Ok indeed it's impressive. Thanks for that! That's with a warm file cache I assume?

bsaul commented 3 years ago

That's with a warm file cache I assume?

I'm not entirely sure. Is there a way to tell.

FWIW, I don't know that it matters for my particular application, since the application will generally be run in containerized, one run and done way.

Nadrieril commented 3 years ago

I don't know how to tell but usually just running the test twice in a row and discarding the first run is good enough. It's just no know if the difference is mostly because of the file size or processing size. You're right though, for your purposes you want to know the overall cost on a cold cache probably.