Graphstore

[samertm]

This PR adds the package "graphstore", which stores graph data separately from the build cache in formats that facilitate retrieval. Once the graph store has reached featured parity with the build cache, we can merge the two packages.

Graph store directory layout:

                 <defs> := SRCLIBPATH/defs/<def-path>
             <def-path> := <repo>/<unit-type>/<unit>/<path>/<commit-id>
<def-path-no-commit-id> := <repo>/<unit-type>/<unit>/<path>

                 <refs> := SRCLIBPATH/refs/<ref-path>
             <ref-path> := <def-path-no-commit-id>/.refs/<ref-repo>

Refs are currently stored as "all.refs" files, and are indexed by ref-path. We don't save a ref's history or differentiate between commits: when you import refs, old refs for that repository are overwritten.

TODO:

• [x] Store references
• [x] List references
• [ ] Store definitions
• [ ] Def accessors
• [ ] Store docs
• [ ] Doc accessors
• [ ] Store annotations
• [ ] Ann accessors
• [ ] Compress data (pack files for old commits)

TODO for sourcegraph/sourcegraph:

• [ ] Interface with S3
• [ ] Call "import" on import step
• [ ] Import builds from the srclib-cache bucket

Unknowns:

• [ ] How much time does the import step take?
• [ ] What is the memory usage like for large builds?