shrink moonfire-nvr binary

[scottlamb]

There's some significant bloat in the binary right now:

• [ ] debug symbols. These are handy, but they're enormous. This is the vast majority of the reason for the large binaries. (Unstripped binary is 150M. Stripped is 9.2M.) When I release, it'd be nice to do split these to a separate file as Ubuntu does. See: https://github.com/rust-lang/rust/issues/34651
• [ ] protobuf. It's surprisingly large; maybe there's some low-hanging fruit or we could switch to prost.
• [x] regex. This was a convenient way of doing my parsing, but it's not strictly necessary, and cargo bloat says it's pretty large.
  • [ ] cursive. It's a lovely library, but long-term I want a web config UI (#35) instead.
  • [x] duplicate crypto libraries: openssl and ring. libpasta uses ring iirc. I could just stop using openssl, although that might not be a good choice when I actually have built-in https (#27) [Currently I've eliminated openssl in the new-schema branch. It's not yet merged to master.]
  • [x] two arguments parsing libraries: clap and docopt. I use docopt. prettydiff is pulling in clap, which at first glance seems silly. I might convert to clap anyway though so maybe there's no point in messing with prettydiff.
  • [x] two versions of parking_lot. (tokio currently uses 0.7; I use 0.9 in moonfire-{nvr,db,base} and in mylog.) tokio is likely to update soon-ish, so I can probably just wait for this to be fixed.

These aren't causing material problems right now; the big binaries are just a bit embarrassing. So no particular size we need to get under or anything.

$ cargo bloat --release --crates
Compiling ...
Analyzing target/release/moonfire-nvr

File  .text     Size Crate
0.8%  21.0%   1.0MiB std
0.4%  12.0% 602.5KiB moonfire_db
0.4%  10.3% 521.0KiB protobuf
0.2%   5.7% 286.6KiB regex
0.2%   5.3% 268.5KiB moonfire_nvr
0.2%   4.8% 242.2KiB cursive
0.2%   4.3% 215.5KiB regex_syntax
0.1%   3.7% 187.3KiB h2
0.1%   3.0% 148.8KiB docopt
0.1%   2.4% 121.9KiB hyper
0.1%   2.3% 116.2KiB futures
...

[scottlamb]

The split debug symbols (aka "fission") would currently break backtraces, unfortunately:

https://github.com/rust-lang/backtrace-rs/issues/287

[IronOxidizer]

There are many (10+) duplicate packages with different versions. This is likely increasing the size to some degree. These can be reduced by using dependencies with matching dependency versions.

Here's a quick list:

ansi_term v0.11.0
ansi_term v0.9.0
error-chain v0.11.0
error-chain v0.12.2
proc-macro2 v0.4.30
proc-macro2 v1.0.10
quote v0.6.13
quote v1.0.3
rand v0.3.23
rand v0.4.6
rand v0.5.6
rand v0.7.3
rand_core v0.3.1
rand_core v0.4.2
rand_core v0.5.1
strsim v0.8.0
strsim v0.9.3
structopt v0.2.18
structopt v0.3.13
structopt-derive v0.2.18
structopt-derive v0.4.6
syn v0.15.44
syn v1.0.17
time v0.1.43
time v0.2.10
unicode-xid v0.1.0
unicode-xid v0.2.0

For the depedency graph, use cargo tree -d

[scottlamb]

Definitely not ideal, although I suspect none of the ones in that list are a major contribution to the binary size given the cargo bloat output.

[scottlamb]

As of 0.7.0, no duplicate packages!

[slamb@slamb-workstation ~/git/moonfire-nvr/server]$ cargo bloat --release --crates
    Finished release [optimized + debuginfo] target(s) in 0.05s
    Analyzing target/release/moonfire-nvr

File  .text     Size Crate
0.6%  18.7% 982.2KiB std
0.6%  16.8% 884.4KiB moonfire_db
0.3%  10.4% 545.1KiB protobuf
0.2%   6.4% 335.4KiB moonfire_nvr
0.2%   6.2% 328.0KiB clap
0.2%   4.7% 248.3KiB cursive_core
0.2%   4.5% 235.2KiB retina
0.1%   4.1% 213.7KiB tokio
0.1%   3.2% 170.8KiB hyper
0.1%   1.8%  93.0KiB backtrace
0.0%   1.1%  60.2KiB http
0.0%   1.1%  60.2KiB cursive
0.0%   1.0%  53.9KiB hashbrown
0.0%   1.0%  52.7KiB serde_json
0.0%   1.0%  51.3KiB rustc_demangle
0.0%   0.9%  47.6KiB url
0.0%   0.8%  42.9KiB miniz_oxide
0.0%   0.8%  40.2KiB addr2line
0.0%   0.7%  36.7KiB idna
0.0%   0.7%  35.3KiB sha2
0.4%  11.2% 589.7KiB And 88 more crates. Use -n N to show more.
3.4% 100.0%   5.1MiB .text section size, the file size is 152.2MiB

Note: numbers above are a result of guesswork. They are not 100% correct and never will be.

Biggest likely-fixable contributors that are still around: debug info (file size minus .text section is 147 MiB, so still the vast majority), protobuf, and cursive:

• debug info: the ecosystem seems to be slowly making progress on split debug info working properly, so I'm just waiting it out.
• protobuf: I might try switching to prost. I'm likely to stop using the protobuf text format stuff anyway in favor of JSON. I'll likely use JSON-format permissions for #153 and #133.
• cursive: I still intend to obsolete this in favor of a web configuration UI.

moonfire-db has grown, I think largely as a result of using JSON stuff in the schema (serde_json code is large). The v6->v7 schema upgrade funcs in particular are pretty large. Maybe eventually I can drop older schema upgrade code.

[scottlamb]

With 0406e09, the binary shrunk from 154 MiB to 70 MiB. I'm still hoping to reduce it further with split debug info. From this comment and this PR it looks like work is actively being done on that.