search

vmware / differential-datalog

DDlog is a programming language for incremental computation. It is well suited for writing programs that continuously update their output in response to input changes. A DDlog programmer does not write incremental algorithms; instead they specify the desired input-output mapping in a declarative manner.
MIT License
1.37k stars 117 forks source link

"Duplicate" dependencies #442

Open d-e-s-o opened 4 years ago

d-e-s-o commented 4 years ago

Filing this issue to track cleaning up of "duplicate" Rust dependencies. Mostly out of curiosity when we were looking a bit at how to speed up the CI pipeline, I thought about reducing the number of dependencies. cargo-tree helps visualize our dependency (for the Rust part of the project). Here is the tree:

datalog_example v0.1.0 (differential-datalog/rust/template)
├── abomonation v0.7.3
├── cmd_parser v0.1.0 (differential-datalog/rust/template/cmd_parser)
│   ├── differential_datalog v0.1.0 (differential-datalog/rust/template/differential_datalog)
│   │   ├── abomonation v0.7.3 (*)
│   │   ├── differential-dataflow v0.9.0
│   │   │   ├── abomonation v0.7.3 (*)
│   │   │   ├── abomonation_derive v0.3.0
│   │   │   │   ├── quote v0.3.15
│   │   │   │   ├── syn v0.11.11
│   │   │   │   │   ├── quote v0.3.15 (*)
│   │   │   │   │   ├── synom v0.11.3
│   │   │   │   │   │   └── unicode-xid v0.0.4
│   │   │   │   │   └── unicode-xid v0.0.4 (*)
│   │   │   │   └── synstructure v0.6.1
│   │   │   │       ├── quote v0.3.15 (*)
│   │   │   │       └── syn v0.11.11 (*)
│   │   │   ├── fnv v1.0.6
│   │   │   ├── serde v1.0.102
│   │   │   │   └── serde_derive v1.0.102
│   │   │   │       ├── proc-macro2 v1.0.6
│   │   │   │       │   └── unicode-xid v0.2.0
│   │   │   │       ├── quote v1.0.2
│   │   │   │       │   └── proc-macro2 v1.0.6 (*)
│   │   │   │       └── syn v1.0.7
│   │   │   │           ├── proc-macro2 v1.0.6 (*)
│   │   │   │           ├── quote v1.0.2 (*)
│   │   │   │           └── unicode-xid v0.2.0 (*)
│   │   │   │   [dev-dependencies]
│   │   │   │   └── serde_derive v1.0.102 (*)
│   │   │   ├── serde_derive v1.0.102 (*)
│   │   │   ├── timely v0.9.0
│   │   │   │   ├── abomonation v0.7.3 (*)
│   │   │   │   ├── abomonation_derive v0.3.0 (*)
│   │   │   │   ├── serde v1.0.102 (*)
│   │   │   │   ├── serde_derive v1.0.102 (*)
│   │   │   │   ├── timely_bytes v0.9.0
│   │   │   │   ├── timely_communication v0.9.0
│   │   │   │   │   ├── abomonation v0.7.3 (*)
│   │   │   │   │   ├── abomonation_derive v0.3.0 (*)
│   │   │   │   │   ├── getopts v0.2.21
│   │   │   │   │   │   └── unicode-width v0.1.6
│   │   │   │   │   ├── serde v1.0.102 (*)
│   │   │   │   │   ├── serde_derive v1.0.102 (*)
│   │   │   │   │   ├── timely_bytes v0.9.0 (*)
│   │   │   │   │   └── timely_logging v0.9.0
│   │   │   │   └── timely_logging v0.9.0 (*)
│   │   │   └── timely_sort v0.1.6
│   │   ├── fnv v1.0.6 (*)
│   │   ├── graph_map v0.1.0 (https://github.com/frankmcsherry/graph-map.git#f2634f29)
│   │   │   ├── memmap v0.5.2
│   │   │   │   └── libc v0.2.65
│   │   │   └── rand v0.4.6
│   │   │       └── libc v0.2.65 (*)
│   │   ├── libc v0.2.65 (*)
│   │   ├── num v0.2.0
│   │   │   ├── num-bigint v0.2.3
│   │   │   │   ├── num-integer v0.1.41
│   │   │   │   │   └── num-traits v0.2.8
│   │   │   │   │       [build-dependencies]
│   │   │   │   │       └── autocfg v0.1.7
│   │   │   │   │   [build-dependencies]
│   │   │   │   │   └── autocfg v0.1.7 (*)
│   │   │   │   ├── num-traits v0.2.8 (*)
│   │   │   │   └── serde v1.0.102 (*)
│   │   │   │   [build-dependencies]
│   │   │   │   └── autocfg v0.1.7 (*)
│   │   │   ├── num-complex v0.2.3
│   │   │   │   ├── num-traits v0.2.8 (*)
│   │   │   │   └── serde v1.0.102 (*)
│   │   │   │   [build-dependencies]
│   │   │   │   └── autocfg v0.1.7 (*)
│   │   │   ├── num-integer v0.1.41 (*)
│   │   │   ├── num-iter v0.1.39
│   │   │   │   ├── num-integer v0.1.41 (*)
│   │   │   │   └── num-traits v0.2.8 (*)
│   │   │   │   [build-dependencies]
│   │   │   │   └── autocfg v0.1.7 (*)
│   │   │   ├── num-rational v0.2.2
│   │   │   │   ├── num-bigint v0.2.3 (*)
│   │   │   │   ├── num-integer v0.1.41 (*)
│   │   │   │   ├── num-traits v0.2.8 (*)
│   │   │   │   └── serde v1.0.102 (*)
│   │   │   │   [build-dependencies]
│   │   │   │   └── autocfg v0.1.7 (*)
│   │   │   └── num-traits v0.2.8 (*)
│   │   ├── sequence_trie v0.3.6
│   │   ├── serde v1.0.102 (*)
│   │   └── timely v0.9.0 (*)
│   ├── libc v0.2.65 (*)
│   ├── nom v4.2.3
│   │   └── memchr v2.2.1
│   │   [build-dependencies]
│   │   └── version_check v0.1.5
│   ├── num v0.2.0 (*)
│   └── rustyline v1.0.0
│       ├── encode_unicode v0.1.3
│       ├── libc v0.2.65 (*)
│       ├── nix v0.5.1
│       │   ├── bitflags v0.4.0
│       │   └── libc v0.2.65 (*)
│       └── unicode-width v0.1.6 (*)
├── cpuprofiler v0.0.3
│   ├── error-chain v0.5.0
│   │   └── backtrace v0.2.3
│   │       ├── backtrace-sys v0.1.32
│   │       │   └── libc v0.2.65 (*)
│   │       │   [build-dependencies]
│   │       │   └── cc v1.0.46
│   │       ├── cfg-if v0.1.10
│   │       ├── dbghelp-sys v0.2.0
│   │       │   └── winapi v0.2.8
│   │       │   [build-dependencies]
│   │       │   └── winapi-build v0.1.1
│   │       ├── kernel32-sys v0.2.2
│   │       │   └── winapi v0.2.8 (*)
│   │       │   [build-dependencies]
│   │       │   └── winapi-build v0.1.1 (*)
│   │       ├── libc v0.2.65 (*)
│   │       ├── rustc-demangle v0.1.16
│   │       └── winapi v0.2.8 (*)
│   └── lazy_static v0.2.11
├── ddlog_ovsdb_adapter v0.1.0 (differential-datalog/rust/template/ovsdb)
│   ├── differential_datalog v0.1.0 (differential-datalog/rust/template/differential_datalog) (*)
│   ├── num v0.2.0 (*)
│   ├── serde_json v1.0.41
│   │   ├── itoa v0.4.4
│   │   ├── ryu v1.0.2
│   │   └── serde v1.0.102 (*)
│   └── uuid v0.7.4
│       └── byteorder v1.3.2
├── differential-dataflow v0.9.0 (*)
├── differential_datalog v0.1.0 (differential-datalog/rust/template/differential_datalog) (*)
├── fnv v1.0.6 (*)
├── lazy_static v1.4.0
├── libc v0.2.65 (*)
├── num-traits v0.2.8 (*)
├── rustop v1.0.4
├── serde v1.0.102 (*)
├── time v0.1.42
│   └── libc v0.2.65 (*)
│   [dev-dependencies]
│   └── winapi v0.3.8
├── timely v0.9.0 (*)
└── twox-hash v1.5.0
    └── rand v0.7.2
        ├── getrandom v0.1.13
        │   ├── cfg-if v0.1.10 (*)
        │   └── libc v0.2.65 (*)
        ├── libc v0.2.65 (*)
        ├── rand_chacha v0.2.1
        │   ├── c2-chacha v0.2.3
        │   │   └── ppv-lite86 v0.2.6
        │   └── rand_core v0.5.1
        │       └── getrandom v0.1.13 (*)
        └── rand_core v0.5.1 (*)
        [dev-dependencies]
        └── rand_hc v0.2.0
            └── rand_core v0.5.1 (*)
[build-dependencies]
└── libtool v0.1.1

I noticed a few low hanging fruit that have the potential to marginally speed up compilation and/or reduce memory foot print. Even if there is little effect, it is probably worth the (presumably small) effort to clean things up as newer versions typically contains bug fixes or other improvements.

In particular, in the tree above we can see that syn, proc-macro2, and quote are compiled in different versions. syn in turn has a transitive dependency on unicode-xid, which we now also compile twice. Lastly, there is lazy_static being used in two versions.

I've upstreamed changes to make differential-dataflow and timely use 1.0 versions of the crates. All we have to do there is wait for new releases and pick them up.

lazy_static is consumed in an old version by cpuprofiler and it seems the project is dead (based on a pull request in it that never received a response). But I'll create a pull request myself and see.

d-e-s-o commented 4 years ago

Update: