Sprint Sep 21

[RichardLitt]

Sprint Goals

• Finalize roadmap and ship 0.3.8

  Sprint Discussions

See https://github.com/ipfs/pm#sprint-discussion-schedule

Please take notes in a separate pad, if you can, and link it here.

• main pad: https://etherpad.mozilla.org/nTqGOVynPT
• infrastructure: https://etherpad.mozilla.org/mJ1ryjJLeH

  Sprint Deliverables

• Add yourself below, that way we only have people listed who are really in the sprint
• Add links to issues down here. Only add things you can finish this sprint.

  @RichardLitt

• [ ] Take a crack at node-ipfs-api docs!
• [x] Work with lgierth to make sure that set-dns-record tool works as expected.
• [x] Streamline ipfs/support into website and community flow.
• [ ] Build a community page listing all of the repos currently in development and make sure that their internal READMEs are streamlined. Write a starlog about this once it is done.
• [x] Close https://github.com/ipfs/website/pull/14 and all baseURL website issues. Also see https://github.com/ipfs/website/issues/2.
• [ ] Resolve issues on ipfs/community as much as possible. Don't go overboard.
• [ ] Fix textarea scrolling on https://github.com/ipfs/starlog/issues/26
• [ ] See if I can codify how I can help manage sprints and project management better.
• [ ] Give IPFS install another test run through for ease of use.
• [x] Work on IPFS-textbook
• [ ] Finish job spec.
• [x] Get information on good DNS providers; open an issue about it.
• [x] Ask for cool websites we want to have IPFS look like.
• [x] Cake day issues. JSON in ipfs/community for cake days and travel plans.
• [ ] Make a list of things to do for new repos: for instance, adding Contribute, Adding Readme, adding badges, Add language contribute link, add travis, add CI, add to roadmap / repo list.
• [ ] Add -contribute.md files to community list. See https://github.com/ipfs/go-ipfs/pull/1734
• [x] Review at least one node-ipfs module
• [ ] Fill out https://github.com/ipfs/ipfs/blob/master/project-directory.md

  @diasdavid || @daviddias

• [x] Open a call for volunteers + Captain.log https://github.com/ipfs/node-ipfs/issues/30
• [x] https://github.com/diasdavid/node-spdy-stream-muxer/pull/5
• [x] Organize ipfs-lx
• [x] Complete Swarm Revisit https://github.com/diasdavid/node-ipfs-swarm/tree/revisit
• [x] Make node-libp2p bindings
• [x] Merge https://github.com/ipfs/node-ipfsd-ctl/pull/21 to fix https://github.com/ipfs/node-ipfs-api/pull/65

  @dignifiedquire

  Workable state of the electron app

• [ ] Finish current efforts and clean up https://github.com/ipfs/electron-app/pull/43
• [ ] Test new dist version of electron app on windows and mac and find people to test on linux
• [ ] Start thinking of an improved webui (design)

  @lgierth

• [x] gateway-dmca-denylist https://github.com/ipfs/gateway-dmca-denylist
• [ ] dmca.denylists.lists.refs.ipfs.io https://github.com/ipfs/go-ipfs/pull/1551
• [~] peer discovery using cjdns https://github.com/ipfs/go-ipfs/pull/1316
• [~] fix the pluto private key situation https://github.com/ipfs/go-ipfs/pull/1748
• cjdns on solarnet https://github.com/ipfs/infrastructure/pull/94
  • [x] update cjdns
  • [x] update peering tools
  • [x] allow alexandria peering with us
  • [x] peer with ansuz' new node in london
• [ ] update infrastructure documentation
• [ ] move pollux to a protocol labs account https://github.com/ipfs/infrastructure/issues/73
• [ ] gateway caching headers

  @whyrusleeping

• [ ] finish private-blocks PR (ready for CR again) (https://github.com/ipfs/go-ipfs/pull/1538)
• [ ] finish mfs extraction (https://github.com/ipfs/go-ipfs/pull/1715)
• [ ] epoll fixes (https://github.com/jbenet/go-reuseport/issues/8)
• [ ] integrate UDT into go-ipfs
  • [x] go-udtwrapper fixes (https://github.com/jbenet/go-udtwrapper/pull/13)
  • [ ] PR it into multiaddr-net
  • [ ] finally, PR that into go-ipfs
• [ ] add chunking algo to root unixfs file blocks
• [ ] ipfs update command
• [x] dht command tweaks (allow for easier key input)

  @amstocker

• [ ] python-ipfs-api:
  • [ ] pydoc documentation
  • [ ] make readthedocs.org page
  • [ ] make it easy to integrate python-ipfs-api into async environment
  • [ ] make filestreaming chunk large files/multiple files
  • [x] python 3 support
  • [x] testing framework
  • [x] tavisci setup
• [ ] python-multiaddr
  • [ ] make testing framework
  • [ ] python 3 support
  • [ ] feature parity with go-multiaddr/node-multiaddr

    @jbenet

• [ ] put ipld into go-ipfs
• [x] meet with people @stanford
• [ ] make libp2p go interfaces file
• [ ] node-ipfs repo module listings
• [ ] api docs designdoc

  @rht

• [ ] refactor ipfs add (I have a local branch where the bulk of core/commands/add.go is moved to core/coreunix/add.go...if this is preferable?) (and optionally investigate slow add but this is not guaranteed to finish)
• [ ] fix gc test (pr has no merge conflict with 0.4.0)

[jbenet]

Food for go-ipfs https://github.com/ipfs/pm/issues/28#issuecomment-133756372

[jbenet]

needed in go-ipfs@0.3.8

• [ ] ipfs update
• [ ] UDT transport
• [ ] IPNS patches

[daviddias]

@jbenet are we going full UDT or just supporting that primarily for now?

[whyrusleeping]

@diasdavid see https://github.com/ipfs/go-ipfs/issues/1738

[jbenet]

Reminder: Only add things you can finish this sprint.

[jbenet]

@rht add tasks to etherpad?

[daviddias]

SPRINT CHECKIN

• [x] Started a 'captain.log' on node-ipfs repo, like a dev news letter for all things node-ipfs https://github.com/ipfs/node-ipfs/issues/30 so that folks can feel welcome to chip in to the current endeavours.
• [x] Lisbon IPFS Meetup - Got the Venue and the date (Oct 7th) and some awareness around the Lisbon community. Unfortunately meetup.com deleted our meetup (+ the Portland one) for lack of information in the description, reached out to support to get it back, waiting to hear from them.
• [x] Swarm revisit https://github.com/diasdavid/node-libp2p-swarm/pull/10, now there is organization to add multiple transports and how to upgrade connections, following https://github.com/diasdavid/node-libp2p-swarm/issues/8
• [x] Reviewed and merged a bunch of PRs across the board, from node-ipfs-api, npm-go-ipfs, libp2p stuff and more
• [x] Discussed with @whyrusleeping on what is Bitswap to start implementing it on node-ipfs
• [x] Got some notes from @jbenet on how the chunking of IPFS is done and structured
• [x] Reviewed and posted feedback to PANDO, the PhD proposal from Erick Lavoie
• [x] Landed the greenfield for libp2p-website https://github.com/diasdavid/libp2p-website (really happy how nice it is to use IPFS to publish static websites with simple npm scripts), continue working on the plan of ip-surge tool (that resembles https://surge.sh/)

[ghost]

Pad for next sprint (Sep 28): https://etherpad.mozilla.org/juEv7EcG5l

[whyrusleeping]

• [x] UDT cleanup and tests
  • [x] more cross platform makefile
• [ ] ipns patches (writing some more tests)
• [x] iptb can use port 0
  • [x] misc other iptb improvements
• [x] fix api file for port zero
• [ ] ipfs update command (wrote scaffolding, still needs infra work)
• [x] bitswap stat returns number of bytes wasted
• [x] docs for ipfs object patch
• [x] better input parsing for 'ipfs dht' commands
• [x] wrote up beginnings of 0.3.8 changelog
• [x] began discussion of udt transition steps
• [x] bitswap discussion with @daviddias

[hackergrrl]

• [x] rig up a crude 'ipget' prototype (https://github.com/noffle/ipget/)

Lots of learning around both the go-ipfs codebase and how to do things The Right Way(tm) with golang -- loving the unixy tooling. Getting the ipget tool working was pretty easy once @whyrusleeping showed me how to get a fresh node up. No blockers.

[ghost]

• [x] the little gateway-dmca-denylist tool finally does something -- renders and adds a structure of notices and to-be-blocked objects https://github.com/ipfs/gateway-dmca-denylist
• [ ] ipfs.io/refs/lists/denylists/dmca should resolve to dmca.denylists.lists.refs.ipfs.io, which is a dnslink to the latest version of the denylist https://github.com/ipfs/go-ipfs/pull/1551
• [~] peer discovery using cjdns, i'm updating the branch right now and will add the missing dialing parts https://github.com/ipfs/go-ipfs/pull/1316
• [~] fix the pluto private key situation -- we lost pluto's private key a while ago, why is mining for a new one starting in SoL[Pp] https://github.com/ipfs/go-ipfs/pull/1748
• cjdns on solarnet https://github.com/ipfs/infrastructure/pull/94
  • [x] update cjdns
  • [x] update peering tools
  • [x] allow alexandria peering with us
  • [x] peer with ansuz' new node in london
• [ ] update infrastructure documentation
• [ ] move pollux to a protocol labs account https://github.com/ipfs/infrastructure/issues/73
• [ ] gateway caching headers

edit:

• [x] made dnslink-deploy work with @RichardLitt https://github.com/ipfs/dnslink-deploy

[dignifiedquire]

Main goal: Workable state of the electron app

• [ ] Finish current efforts and clean up ipfs/electron-app#43
• [ ] Test new dist version of electron app on windows and mac and find people to test on linux
• [ ] Start thinking of an improved webui (design)
• [x] Being sick

[jbenet]

• [ ] put ipld into go-ipfs
• [x] meet with people @stanford
• [~] make libp2p go interfaces file (posting later today)
• [~] node-ipfs repo module listings (posting later today)
• [~] api docs designdoc (posting later today)
• [x] reached out to golang list re golang/build https://groups.google.com/forum/#!topic/golang-nuts/kRq8Ow4qEeU
• [x] reached out to quic list re auth + asymmetry https://groups.google.com/a/chromium.org/forum/#!topic/proto-quic/b8c39UbgmKA and https://groups.google.com/a/chromium.org/forum/#!topic/proto-quic/Gf6Hi7XJFAc
• [x] eval CurveCP as a transport
• [x] eval QUIC, https://github.com/devsisters/goquic, and feasibility
• [x] eval uTP/LEDBAT, https://github.com/anacrolix/utp, https://github.com/jbenet/go-libutp
• [x] call with ethereum swarm people
• [x] review + feedback for http://www.ietf.org/id/draft-benfield-http2-p2p-01.txt
• [x] figure out details re UTF-8 on paths (mailed plan9ers)
• [x] addressed windows' users problems and triggered gobuilder manually (wtf is going on? why is it still broken?)
• [x] met with OKFN Ireland -- Denis Parfenov, Ingo, and Zach

[RichardLitt]

• [x] Work with lgierth to make sure that set-dns-record tool works as expected.
• [x] Streamline ipfs/support into website and community flow.
• [x] Close https://github.com/ipfs/website/pull/14 and all baseURL website issues. Also see https://github.com/ipfs/website/issues/2.
• [x] Work on IPFS-textbook
• [x] Get information on good DNS providers; open an issue about it.
• [x] Ask for cool websites we want to have IPFS look like.
• [x] Cake day issues. JSON in ipfs/community for cake days and travel plans.
• [x] Review at least one node-ipfs module
• [x] Fill out https://github.com/ipfs/ipfs/blob/master/project-directory.md

Worked on:

• [ ] Build a community page listing all of the repos currently in development and make sure that their internal READMEs are streamlined. Write a starlog about this once it is done.
• [ ] Make a list of things to do for new repos: for instance, adding Contribute, Adding Readme, adding badges, Add language contribute link, add travis, add CI, add to roadmap / repo list.
• [ ] Fix textarea scrolling on https://github.com/ipfs/starlog/issues/26

Not done:

• [ ] Take a crack at node-ipfs-api docs!
• [ ] Resolve issues on ipfs/community as much as possible. Don't go overboard.
• [ ] See if I can codify how I can help manage sprints and project management better.
• [ ] Give IPFS install another test run through for ease of use.
• [ ] Finish job spec.
• [ ] Add -contribute.md files to community list. See https://github.com/ipfs/go-ipfs/pull/1734

[rht]

• [ ] Refactor ipfs add has to wait for 0.4.0 rebase.
• [ ] gc: disk size stuff on os x

[jbenet]

@rht is that done this week? or upcoming? you'll want to add next tasks to etherpad in #35

[jbenet]

@jbenet's sprint feedback

(sorry for misisng the timing, hopefully this compensates a bit)

@richardlitt:

• love the dnslink-deploy tool! def was a missing piece. https://github.com/ipfs/dnslink-deploy for those interested
• +1 for ipfs/support. may want to advertise it over at https://groups.google.com/forum/#!forum/ipfs-users too.
• +1 on all the website push stuff

@diasdavid

• +1 for captain's log. great work. will do one for go-ipfs
• sorry about the meetup.com fail
• yay on swarm revisit.
• thanks for keeping on top of maintaining all the js modules

@dignifiedquire

• hope you're feeling all better!
• and nice designs

@lgierth

• good progress with denylist
• love the dnslink-deploy tool! def was a missing piece. https://github.com/ipfs/dnslink-deploy for those interested

@whyrusleeping

• yay for all the cpp UDT work. will come in very handy
• thanks for the iptb update, much better tests incoming
• 0.3.8 so close!

@rht

• the benchmark will be very useful. thanks for pressing on that

@noffle

• :clap: on ipget -- it's so nice and simple. now we can plug IPFS into important things! \o/

@amstocker

• +1 on all the testing. great to have.

[jbenet]

• @mappum - want to drop a sprint update for all the ipfs-node stuff here?
• @davidar - want to drop an update here for archives and so on?
• @chriscool - want to drop an update here re tests?

also, thanks to:

• @ianopolous finished https://github.com/ipfs/java-ipfs-api :clap: :clap: (feel free to drop updates here!)
• @mildred - you pushed on very important IPLD stuff + discussion points. lmk if i'm blocking on the code (will look back)
• @willglynn thanks for implementing path support -- will take closer look this week

and thanks to lots of other people that made IPFS lots better this week! \o/ :fireworks:

[chriscool]

• [x] Added shellquote() thanks to @ion1, see https://github.com/ipfs/go-ipfs/pull/1757

Started to fix it as @ion1 suggested and to add tests, but it's not so easy to add tests because of all the quoting.

[ianopolous]

All these are in java-ipfs-api

• [X] cleaned up api, removed MerkleNode where it was only used as a proxy for Multihash.
• [X] ported MultiAddress, including binary encoding and all the protocols as well as varints from Go
• [X] gave IPFS a constructor taking a MultiAddress (nice suggestion @jbenet )
• [X] wrote blog post about it (http://ianopolous.github.io/java/IPFS/)

To Do

• [ ] Use the API in Peergos, which will probably result in further improvements
• [ ] Javadoc the API
• [ ] make sure the relation between the different types of pinning to normal add is clear to new comers
• [ ] give it an O(1) memory streaming api - ipfs.add(InputStream)
• [ ] Toying with the idea of a cross platform IPFS installer (in Java) which we would also use in Peergos

[davidar]

• IPFS-hosted markup editor thanks to @osener
• discussed getting webrecorder.io onto IPFS with @ikreymer ipfs/archives#28
• made a little bit of progress with the BASE data ipfs/archives#3 (HELP WANTED, if anyone's interested)
• mirrored PGP pubkeys ipfs/archives#27
• tried to get the bookreader working with BPG (please ping me if you're an emscripten expert)
• pushed the first complete arXiv archive
• argued about IPLD some more ;)
• agreed on a standard format for archive metadata

[mappum]

• [x] planned initial npm-bootstrapped ipfs-npm prototype
• [x] implemented inpm with /ipfs/HASH symlink install
• [x] filed issues on node-ipfsd-ctl for bugs that were blocking inpm (thanks for fixing stuff, @bcomnes)
• [x] went to Tokyo :airplane: :jp:

[rht]

pushed the first complete arXiv archive

Thanks a lot for this. There is e.g. https://blog.lateral.io/2015/07/harvesting-research-arxiv/ (https://arxiv.lateral.io/), but definitely a lot more stuff can be done, etc. (haven't dumped my thoughts yet...)

No, finished neither of the 2 checklists.

[ozanmakes]

Great initiative! Is there anything I can do to make markup.rocks easier to integrate?

[davidar]

Is there anything I can do to make markup.rocks easier to integrate?

@osener That would be great :). What I'd really like to do is replace the Dropbox open/save links with IPFS equivalents (similar to the IPFS wiki by @jamescarlyle). I've started on an IPFS Haskell API (ipfs/ipfs#4), which may be able to help with this.

Feel free to ping me (davidar) on http://chat.ipfs.io (or #ipfs on freenode) if you want to discuss this further.

[davidar]

@rht yes, definitely interested in hearing ideas for utilising the arxiv data. The ones I've thought of are:

• search engine for LaTeX equations
• publishing papers in alternative formats (html, etc)
• building a citation graph
• training a topic model (latent Dirichlet allocation / hierarchical Dirichlet processes)
• automatically extracting definitions to build a dictionary of terminology
• semantic markup of math (connecting variables in equations to their textual descriptions, disambiguating notation, etc)

Edit: see ipfs/apps#1

[jamescarlyle]

FWIW I'm planning to add a markdown parser to my ipfs-wiki, so that standard md syntax can be used (and then rendered as HTML)