Lightening up LightGraphs

[sglyon]

I think we've arrived in a position LightGraphs with all of its dependencies isn't currently very "light".

I have in mind dependencies like GZip and JLD, that I believe are just used for graph persistence, not within algorithms.

How can we lighten up the core algorithms from extra features like persistence?

[sbromberger]

👍 for this discussion. Right now,

GZip 0.2.20
EzXML 0.3.0
ParserCombinator 1.7.11
JLD 0.6.3

can all be moved if we refactor persistence. My opinion is that we should do this in two phases: 1) move persistence into its own module 2) migrate to FileIO.jl, thereby removing these deps and going to a standard IO framework.

(we'd still need these dependencies.) Thoughts?

[sbromberger]

PS: This should probably happen AFTER the big 1.0 transition.

[sbromberger]

cc @jpfairbanks @juliohm

[jpfairbanks]

I agree that we could move the persistence and some of the analysis codes into their own packages, this was part of the decision to make LightGraphsExtras into its own repo. The only downside is that we have a more complicated dependency graph where you need certain versions of GraphPersistence to be compatible with certain versions of LightGraphs.

[sbromberger]

@jpfairbanks yes - that's true. Given the "heaviness" of persistence, though, perhaps we keep the standard lightgraph file format in "Base LG" and move the others into LGE? It would still require gzip, though.

@sglyon - is a dependence on gzip problematic for your use case?

[sglyon]

I just took a look at GZip.jl and it looks like it doesn't have deps/build.jl file. Does that mean it doesn't have any binary dependencies above and beyond what it expects to find on the user system?

[sbromberger]

Does that mean it doesn't have any binary dependencies above and beyond what it expects to find on the user system?

I think it requires zlib.

[sglyon]

From what I can see in GZip.jl it doesn't look like the julia package tries to download/install zlib. is that right?

If so, then yes, having a dependency on GZip.jl would work for my use case

[sbromberger]

it doesn't look like the julia package tries to download/install zlib. is that right?

That's how I'm reading it too.

having a dependency on GZip.jl would work for my use case

Awesome. That allows LG to have a native persistence method in Base, which makes things much better.

[sbromberger]

Just FYI, here's the current dependency list on a fresh Julia install:

julia> Pkg.add("LightGraphs")
INFO: Initializing package repository /home/seth/.julia/v0.5
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
INFO: Cloning cache of AutoHashEquals from https://github.com/andrewcooke/AutoHashEquals.jl.git
INFO: Cloning cache of BinDeps from https://github.com/JuliaLang/BinDeps.jl.git
INFO: Cloning cache of Blosc from https://github.com/stevengj/Blosc.jl.git
INFO: Cloning cache of Calculus from https://github.com/johnmyleswhite/Calculus.jl.git
INFO: Cloning cache of Compat from https://github.com/JuliaLang/Compat.jl.git
INFO: Cloning cache of DataStructures from https://github.com/JuliaCollections/DataStructures.jl.git
INFO: Cloning cache of Distributions from https://github.com/JuliaStats/Distributions.jl.git
INFO: Cloning cache of EzXML from https://github.com/bicycle1885/EzXML.jl.git
INFO: Cloning cache of FileIO from https://github.com/JuliaIO/FileIO.jl.git
INFO: Cloning cache of GZip from https://github.com/JuliaIO/GZip.jl.git
INFO: Cloning cache of HDF5 from https://github.com/JuliaIO/HDF5.jl.git
INFO: Cloning cache of JLD from https://github.com/JuliaIO/JLD.jl.git
INFO: Cloning cache of LegacyStrings from https://github.com/JuliaArchive/LegacyStrings.jl.git
INFO: Cloning cache of LightGraphs from https://github.com/JuliaGraphs/LightGraphs.jl.git
INFO: Cloning cache of PDMats from https://github.com/JuliaStats/PDMats.jl.git
INFO: Cloning cache of ParserCombinator from https://github.com/andrewcooke/ParserCombinator.jl.git
INFO: Cloning cache of QuadGK from https://github.com/JuliaMath/QuadGK.jl.git
INFO: Cloning cache of Rmath from https://github.com/JuliaStats/Rmath.jl.git
INFO: Cloning cache of SHA from https://github.com/staticfloat/SHA.jl.git
INFO: Cloning cache of SpecialFunctions from https://github.com/JuliaMath/SpecialFunctions.jl.git
INFO: Cloning cache of StatsBase from https://github.com/JuliaStats/StatsBase.jl.git
INFO: Cloning cache of StatsFuns from https://github.com/JuliaStats/StatsFuns.jl.git
INFO: Cloning cache of URIParser from https://github.com/JuliaWeb/URIParser.jl.git

[sbromberger]

@sglyon we're planning on moving persistence out of LG core by v1.1.0.

[sglyon]

Great! Thanks for the heads up!

[sbromberger]

Making this a list:

• [x] "LightGraphs" (required, obv)
• [x] "DataStructures"
• [x] "Compat" (secondary dep.)
• [x] "StatsBase"
• [x] "SpecialFunctions" (secondary dep.)
• [x] "SimpleTraits"
• [x] "MacroTools" (secondary dep.)
• [x] "GZip" (required if we keep lightgraphs native graph persistence, which I think we should.)
• [ ] "Distributions"
• [ ] "QuadGK"
• [ ] "Calculus"
• [ ] "StatsFuns"
• [ ] "Rmath"
• [ ] "BinDeps"
• [ ] "SHA"
• [ ] "URIParser"
• [ ] "PDMats"
• [ ] "JLD"
• [ ] "FileIO"
• [ ] "HDF5"
• [ ] "WinRPM"
• [ ] "Libz"
• [ ] "BufferedStreams"
• [ ] "LibExpat"
• [ ] "Homebrew"
• [ ] "JSON"
• [ ] "Blosc"
• [ ] "ParserCombinator"
• [ ] "AutoHashEquals"
• [ ] "EzXML"

[sbromberger]

Down to 8.