Sprint P

[jbenet]

(a bit late in consolidating this here, sorry)

Sprint Goals

Sprint Discussions

See https://github.com/ipfs/pm#sprint-discussion-schedule

Sprint Deliverables

• Add yourself below, that way we only have people listed who are really in the sprint
• Add links to issues down here. Only add things you can finish this sprint.

  @jbenet

• [ ] lots of CR
• [ ] land signed objects and records
• [ ] chain record
• [x] meetup venue
• [ ] json-ld / cbor-ld / ipfs-ld tooling
• [x] cube feedback
• [ ] find designer for cube
• [x] limits: bandwidth, connections, disk (discussion)
• [ ] dag path notation (discussion)
• [x] docker blog post gif

  @mappum

• [ ] cube frontend - initial UX mockups/previews
• [ ] cube frontend - React
• [ ] cube backend - file accounting
• [ ] cube backend - IPFS daemon integration (adding, pinning, unpinning)

  @lgierth

• [x] land monitoring https://github.com/ipfs/infrastructure/pull/44
• [x] submit battlemesh talks
• [ ] discover peers from cjdns https://github.com/ipfs/go-ipfs/pull/1316
• [ ] spec out relevant cjdns internals https://github.com/ipfs/notes/issues/14
• [ ] migrate IRC bots to solarnet https://github.com/ipfs/infrastructure/issues/33
• [ ] investigate goroutine spikes https://github.com/ipfs/go-ipfs/issues/1470

  @whyrusleeping

• [ ] fix too many open files issue with bitswap/flatfs https://github.com/ipfs/go-ipfs/issues/1456
• [x] make ping its own protocol
• [ ] fix pinbot
• [ ] ipfs-ld parser
• [x] faster daemon shutdown
• [x] rebase dev0.4.0
• [x] ping as its own protocol
• [x] fix path parsing bug
• [x] bitswap writeblock retries
• [x] improve test reliability

  @heems

• [ ] programmatically set link settings between nodes in bssim based on real latency/bandwidths
• [ ] more/better graphs in bssim
• [ ] make pipeline configurable from 1 command/config file

  @diasdavid

• [ ] abstract-stream-muxer
  • [x] Finish heavy duty tests
  • [x] Polish and Publish node-spdy-stream-muxer - https://github.com/diasdavid/node-spdy-stream-muxer
  • [x] Make node-multiplex-stream-muxer - https://github.com/diasdavid/node-multiplex-stream-muxer
  • [x] Report multiplex under 'mega' stress conditions (https://github.com/maxogden/multiplex/issues/25)
  • [ ] ~Timing reports
• [ ] node-ipfs
  • [ ] swarm
  • [x] attach Stream Muxer through interface
  • [x] make Identify work with protobufs
  • [x] send proper observed addrs
  • [x] write swarm spec (https://github.com/ipfs/specs/pull/18)
  • [x] avoid multiaddr explosion (https://github.com/diasdavid/node-ipfs-swarm/issues/1)
  • [ ] dht
  • [ ] ping
  • [ ] attach swarm for pings
• [ ] Other
  • [ ] travis-ci all the things

    @rht

• [ ] https://github.com/ipfs/go-ipfs/issues/1353 blocks versioning
• [ ] https://github.com/ipfs/go-ipfs/issues/1418 stats
• [ ] https://github.com/ipfs/go-ipfs/issues/143 ()

  @wking

• [ ] support symlinks (discussion)
• [ ] open PRs
• [ ] example or blog post with runC (?)

  @chriscool

• [X] https://github.com/jbenet/ipfs-paste/pull/3

  Backlog

Anyone can pick these up:

• rabin fingerprinting
• limits: bandwidth, connections, disk (implementation)
• implementing dag path notation on bitswap
• jsonld protoc equivalent (with cbor and/or ubjson)
• dht records + ipns
• support symlinks (implementation)
• memleaks, goroutine spikes, gateway "lock-ups"

[jbenet]

I've spent a bunch of time looking for golang cbor encoders. this is the most promising: https://github.com/ugorji/go/issues/86 -- but isn't perfect yet. i made this tool to test them: https://github.com/jbenet/go-json2cbor/

so we can try the ipfs-ld thing, but not sure cbor's going to work reliably this week. i think cbor is the right choice so far (it's the closest thing to json).

ubjson is worse-- there only seems to be one golang implementation, and no idea if it's complete: https://github.com/imipolexg/go-ubjson -- we could try it with {bson, bjson, msgpack} but those all have problems too.

[rht]

Is ipfs-ld meant to augment https://github.com/ipfs/go-ipfs/blob/master/merkledag/node.go? Regardless of current implementation detail, what is the recommended canonical reading here?

[rht]

One use case of json-ld's RDF is to add standardized contexts to the premises of a proposition in a proposition tree? From e.g. http://ethics.spinozism.org/graphs/js/ethica5.json. Though the example doesn't require semversioning (or could be since there are versions of the manuscript) nor content-addressing (or could be for highly composite citations/footnotes, but this can be handled by linked data too).

(but content address is already a kind of semver, ipns is another, but for major versions and "manual" naming, e.g. "Diseased Newt")

[daviddias]

Sprint Update:

abstract-stream-muxer:

• Made heavy duties test, which lead to find some bugs in spdy-transport (now fixed) and a load problem in multiplex (still unresolved, but reported https://github.com/maxogden/multiplex/issues/25)
• Make, polish and publish node-spdy-stream-muxer - https://github.com/diasdavid/node-spdy-stream-muxer
• Make, polish and publish node-multiplex-stream-muxer - https://github.com/diasdavid/node-multiplex-stream-muxer

ipfs-swarm:

• Make ipfs-swarm to use stream muxer interface (for easy change of stream muxer)
• Make Identify use protobufs
• Send the right observedAddr (Node.js IPv6 default representation workaround)
• Write swarm spec (complement the network spec - https://github.com/ipfs/specs/pull/18)

ipfs-routing:

• good discussions around how the DHT is implemented (or will be implemented)

community:

• push for the Seattle IPFS meetup
• push for the Portland IPFS meetup

other updates

• Cory Benfield started an extension for HTTP/2 P2P - https://www.ietf.org/internet-drafts/draft-benfield-http2-p2p-00.txt
• dignifiedquire is interested in helping out with node-ipfs. This is cool because it forces me to be more verbose, creating issues for tasks, which might lead for other contributors

[jbenet]

• lots of CR.
• bad news is, i still haven't landed the records. (sorry whyrusleeping)
• but the good news is i have ipfsld working with cbor! \o/ which would cause the records to be rewritten - anyway, so that's good.
• i'm making a bit of tooling for making schemas, though the base one already works :)
• worked with mappum on cube things
• figured out the process limits discussion
• figured out more about dag path stuff (subsets of XPath, JSON Pointer, JSON Select). still need a decision here. worked through dht/routing/network interfaces with whyrusleeping and daviddias still need to finish CRing daviddias's network spec PR. i spent way too much time messing with asciinema, but the good part is we now have asciinemas of ipfs tooling that get rehosted to ipfs: http://gateway.ipfs.io/ipfs/QmTDAsNvNGAWmJ9Eu3NRFhQxW1Tc8eYBd2KG8UzDqeZWaP (will push out the tool i made later today), so i'll be making a lot of them this week.

[ghost]

• monitoring is finally ready for service. happy to answer any questions re: setting up access
• submitted battlemesh talks re: ipfs, cjdns -- also travel planning (bbq in vienna with @dch of ppsp)
• migrated pinbot to solarnet
• started rebuilding gateway hosts. pluto got resized and is rebuilt. plenty of little fixes for bootstrapping vanilla hosts (PR incoming)
• no news on the goroutine spikes yet, same for cjdns discovery

[jbenet]

etherpad for next week https://etherpad.mozilla.org/OsdMG3LCWe