Sprint Nov 2

[daviddias]

Sprint Nov 2

Sprint Goals

• Get go-libp2p out

  Sprint Discussions

Schedule

Please take notes in a separate pad, if you can, and link it here.

Endeavour	Lead	Pad	Permanent Link
Apps on IPFS	Juan Benet and Friedel Ziegelmayer	https://public.etherpad-mozilla.org/p/nov-2-apps-on-ipfs
infrastructure	Lars Gierth	https://public.etherpad-mozilla.org/p/nov-2-infrastructure
libp2p	David Dias	https://public.etherpad-mozilla.org/p/nov-2-libp2p	https://github.com/ipfs/pm/issues/46#issuecomment-153132086
js-ipfs	David Dias	https://public.etherpad-mozilla.org/p/nov-2-js-ipfs	https://github.com/ipfs/pm/issues/46#issuecomment-153136248
go-ipfs	Jeromy Johnson	https://public.etherpad-mozilla.org/p/nov-2-go-ipfs	https://github.com/ipfs/pm/issues/46#issuecomment-153152451
testing + ci	Jeromy Johnson	https://public.etherpad-mozilla.org/p/nov-2-testing-ci
protocol + specs	Juan Benet	https://public.etherpad-mozilla.org/p/nov-2-protocol-specs
bitswap ml	Juan Benet	https://public.etherpad-mozilla.org/p/nov-2-bitswap-ml
Data Structures	Juan Benet	https://public.etherpad-mozilla.org/p/nov-2-data-structures

Please add the Agenda to the Pad before the endeavour sprint starts.

Sprint Deliverables

• Add yourself below, that way we only have people listed who are really in the sprint
• Add links to issues down here. Only add things you can finish this sprint.

  [<% name %>](@<% your_GitHub_handle %)

  David Dias

• libp2p
  • Continue making modules work in the browser
  • [x] js-peer-id https://github.com/diasdavid/js-peer-id/pull/3 (also make it use RSA instead of ECDH)
  • [x] js-multiaddr https://github.com/jbenet/js-multiaddr/pull/4
  • [x] js-peer-info https://github.com/diasdavid/js-peer-info/pull/3
  • [ ] Get webrtc-explorer as one of the transports (blocked by node-webrtc not compiling on the Mac OS X, problem introduced with XCode7 and El Capitan)
• npm's registry on IPFS
  • [x] update ipfs-blob-store to latest js-ipfs-api
  • [ ] download the entire npm
  • [x] make a read-only reginabox
  • [x] patch reginabox so that it can use a blob-store instead of ipfs directly
  • [ ] install modules via IPFS using reginabox+registry-static
• js-ipfs-api
• js-ipfs
  • [ ] jsipfs module that offers same API as full node, but delegates all the work to a node in the network https://github.com/ipfs/js-ipfs/pull/35
• extra
  • [x] cli tool to add commit signature automatically - https://github.com/diasdavid/sign-commit

[daviddias]

Endeavour leads, please update your sprint meeting pads with a proposed agenda for tomorrow

@jbenet @whyrusleeping @dignifiedquire @lgierth

[dignifiedquire]

I won't make it tomorrow, could someone take over for me please? @jbenet @diasdavid ?

[daviddias]

@Dignifiedquire absolutely, could you to add some notes to the pad with info you would like everyone to know (or to think/decide)? Thanks :)

[daviddias]

libp2p

Lead: David Dias

Agenda

• quick update on the webrtc-update
• quick update on go-libp2p
• can we invest time in this sprint on go-libp2p? (Y/N)
• we need to make everything browser compatible (things like multihash can't depend on Node.js crypto)
• WebCrypto

You have 30 minutes for this agenda, 5 minutes before the meeting ends, consider moving the remaining items to a github discussion thread so that all the other sprint meetings can start in time.

Participants

• David Dias
• Jeromy Johnson
• Johan Kiviniemi (Ion)
• Victor
• Aaron (from ethereum)

Notes

• tentatively:
• Jeromy finishes the utp merge Monday
• Juan CR by Tuesday
• We can start ripping go-libp2p Wednesday
• go through every module and make sure it runs in browser
• post an entry on captain log about this and see if someone wants to chip in

After sprint meeting is finished, create the respective action items on the Github sprint issue

[daviddias]

js-ipfs

Lead: David Dias

Agenda

• quick update
• jsipfs next

You have 30 minutes for this agenda, 5 minutes before the meeting ends, consider moving the remaining items to a github discussion thread so that all the other sprint meetings can start in time.

Participants

• David Dias
• Johan Kiviniemi (Ion)
• Victor
• Aaron (from ethereum)

Notes

• bottom up appoarch (libp2p first)
• top down approach (jsipfs with js-ipfs-api, then bitswap, then merge with the bottom up)

After sprint meeting is finished, create the respective action items on the Github sprint issue

[whyrusleeping]

go-ipfs

Agenda

• watch juan walk through new york
• ipfs-update
• transport
• utp
• go-log
• ipld

Action items

• [ ] refactor transports
• [ ] rebase utp on transport code when done
• [ ] ipfs-update to run migrations
• [ ] ipfs-update integrated into go-ipfs
• [ ] go-log fixes
• [ ] ipld review
• [ ] bssim graph integrations

Participants

• @whyrusleeping
• @jbenet
• @chriscool
• @ion1
• @lgierth

[whyrusleeping]

@whyrusleeping's todos:

• [ ] refactor transports
• [ ] rebase utp on transport code when done
• [x] ipfs-update to run migrations
• [ ] ipfs-update integrated into go-ipfs
• [x] go-log fixes
• [ ] ipld review
• [ ] gx repos
• [x] discuss gx as a generalized extensible package manager
• [ ] ipfs add perf

[jbenet]

@jbenet todos:

• [ ] Code Review
  • [ ] CR go-ipfs utp
  • [x] CR go-ipfs refactored transports
  • [ ] CR ipfs-update -- if others want to help here pls do
  • [x] CR go-log -- if others want to help here pls do
  • [x] CR https://github.com/ipfs/blog/pull/8
• [x] consensys demo
• [x] web of trust conference: https://github.com/WebOfTrustInfo/rebooting-the-web-of-trust
• [x] archive visit -- discuss: https://github.com/ipfs/archives open web, and other things
• [x] org things

[ghost]

Infrastructure

Lead: Lars Gierth

Agenda

• dmca / refs
• keeping track of storage
• openipfs / pin-federation
• writable gateway
• metrics

You have 30 minutes for this agenda, 5 minutes before the meeting ends, consider moving the remaining items to a github discussion thread so that all the other sprint meetings can start in time.

Participants

@whyrusleeping @VictorBjelkholm @chriscool @lgierth

Notes

• candidates for ipfs.io/refs apart from dmca keep popping up: pinned items, storage inventory, releases of other software projects, etc.
• What's the point of having writeable gateway? Content disappears after a while... Ask Juan about use case
  • https://github.com/ipfs/infrastructure/issues/105
• using the ansible scripts in infrastructure.git
  • edit hosts file
  • secrets.yml => secrets_plaintext/secrets.yml
• wanna open up metrics.i.ipfs.io to the public. if you'd like to see certain ipfs or host metrics, talk to @lgierth, it's not too hard

After sprint meeting is finished, create the respective action items on the Github sprint issue

[ghost]

@lgierth's todos:

• dmca / ipfs.io/refs
  • [~] deploy refs.ipfs.io https://github.com/ipfs/gateway-dmca-denylist
  • [~] point ipfs.io/refs to refs.ipfs.io ipfs/infrastructure#95
  • [ ] ping @VictorBjelkholm re: refs/openipfs
• gateway
  • [ ] hope: writable gateway ipfs/infrastructure#105
  • [~] open up cors on gateways (allow origin *) ipfs/infrastructure#109
• org
  • [x] move pollux to protocol labs account ipfs/infrastructure#73
  • more and more back and forth
  • [~] periodic ovh renewal
• archives
  • [ ] hope: get archive inventory together ipfs/infrastructure#106
  • [ ] hope: add notes about cdnjs ipfs/archives#35
  • [ ] hope: add openwrt 15.05 on castor
• cjdns
  • [ ] discover peers from cjdns ipfs/go-ipfs#1316
  • [ ] invite to fc00 development (aka go-cjdns)

[RichardLitt]

@RichardLitt's todos:

• [ ] 15 hours of work (minimum) on IFPS HTTP docs.
  • [ ] Look into automatically generating
  • [ ] Manually do the rest if needed. curl everything to make sure it works.
• [ ] Help out with designing a community infrastructure for meetups
• [ ] Sprint (GitHub) management
• [ ] Make the documentation around Commit Sign-Offs more friendly
• [ ] Parse IPFS readme's automatically using a CLI tool for ipfs-readme-standard.

[masylum]

@diasdavid why don't we get karma runner (:wink: @Dignifiedquire) in the js- repos to test against phantom? Checking manually repo by repo seems cumbersome. We have to setup a CI for most of those repos anyway.

This week I'm toying with:

• [ ] Ruby sdk
• [ ] Abstract connection test suite (its unlikely that I will have any time but...)

[daviddias]

@masylum would it be that simple? Also, the idea is not only to say 'it is broken', but actually have a proper docs of how to use it in the browser (for example, how to use Streams or Buffers), see https://github.com/jbenet/js-multihashing as an example.

[chriscool]

@chriscool's todo:

• [x] Get started with Google App Engine: https://cloud.google.com/appengine/docs/go/gettingstarted/introduction
• [x] Come up with a todo list for https://github.com/ipfs/golang-build
• [x] Come up with a question list for https://github.com/ipfs/golang-build
• [ ] start working on items from the above lists
• [ ] Start installing ipfs/golang-build on Google App Engine
• [ ] help @whyrusleeping if he needs help about git command extension mechanism

[masylum]

@diasdavid may not be that simple but I think its a first step. We have to setup CI to those repos anyway, so at least we could ensure that the ones that can run in the browser can run the tests both in node and browser-land.

[daviddias]

@masylum my current strategy is going one by one, but would be awesome to have that 'CI test machine' you are proposing that just 'shots at every repo'! :D

[kyledrake]

@kyledrake:

• [ ] libp2p site work
• [ ] finish DMCA work
• [ ] changes to blog post

A bunch of other stuff is churning fast right now, expect changes. Two conferences and a lot going on this week.

@masylum I've made some initial ruby related code that you may be interested in. I was definitely hoping to sit down and bang out a ruby API for go-ipfs at least. Check out https://github.com/neocities/ruby-multihash

[masylum]

@kyledrake nice! I'm just looking to do the API sdk. Like the python and node ones.

[hackergrrl]

Hopping in late on this sprint!

apps

• [x] design /w @whyrusleeping on gx package manager on ipfs
• [x] wrote ipcat (https://github.com/noffle/ipcat)

go-ipfs

• [x] design pondering on a common ipfs shell interface (https://github.com/ipfs/go-ipfs/issues/1945)
• [x] fallback-ipfs-shell: uses local node and falls back to ephemeral (https://github.com/noffle/fallback-ipfs-shell)

[daviddias]

SPRINT CHECKIN

• libp2p
  • Continue making modules work in the browser
  • [x] js-peer-id https://github.com/diasdavid/js-peer-id/pull/3 (also make it use RSA instead of ECDH)
  • [x] js-multiaddr https://github.com/jbenet/js-multiaddr/pull/4
  • [x] js-peer-info https://github.com/diasdavid/js-peer-info/pull/3
  • [ ] Get webrtc-explorer as one of the transports - I'm currently "blocked" in this one, I can make it work in Linux, but node-webrtc doesn't compile for Mac OS X now because incompatibilities introduced with XCode7 and El Capitan. This is a known problem, both from the Chrome team and the node-webrtc people, fixing it requires comprehension of Node.js native bindings and how libwebrtc works, fortunately, there are several people involved, testing and working on it.
• npm's registry on IPFS
  • [x] update ipfs-blob-store to latest js-ipfs-api https://github.com/ipfs/ipfs-blob-store/pull/5
  • [x] make a read-only reginabox https://github.com/diasdavid/reginabox
  • [x] patch reginabox so that it can use a blob-store instead of ipfs directly https://github.com/diasdavid/reginabox - I also tried to make the st module to be ipfs-blob-store compatible, so that folks could serve static files from network storage, but that PR started to become way too long and I just changed reginabox instead (reginabox was using st to as a static server)
  • [ ] download the entire npm - Started downloading npm with a fs-blob-store (> 48 hours), but got cut a bunch of times, fortunately, there is an option to resume. However the goal is to download it into IPFS (I started downloading it with fs-blob-store so that then it would be faster to add through ipfs-blob-store). Currently, js-ipfs-api doesn't work 100% with 0.4.0 due to API changes introduced in 0.3.9, I really need for 0.3.9 to be rebased on 0.4.0, which is kind of blocked until this PR https://github.com/ipfs/go-ipfs/pull/1937 gets merged. I've been sync with @whyrusleeping on this.
  • [ ] install modules via IPFS using reginabox+registry-static
• js-ipfs
  • [~] jsipfs module that offers same API as full node, but delegates all the work to a node in the network https://github.com/ipfs/js-ipfs/pull/35 - This is a big effort and PR, but actually good one because it layers a ton of foundations to start upgrading jsipfs gradually till it becomes a full IPFS node
• extra
  • [x] cli tool to add commit signature automatically - https://github.com/diasdavid/sign-commit
  • [x] organize things for IPFS-Lisbon
  • [x] prepare my talk to IPFS-Lisbon
  • [x] work towards organising an IPFS-Porto (there are some people really interested :D)
  • [x] review and merge some PRs
  • [x] calendaring stuff

[RichardLitt]

• [x] Opened https://github.com/ipfs/website/issues/78
• [x] Updated dates on Calendar
• [x] Make a security.md for IPFS
• [x] Opened https://github.com/ipfs/go-ipfs/issues/1946: "Issue Reporting Guide"
• [x] Copy-edited news PR. https://github.com/ipfs/website/pull/71
• [~] Looked at Victor's PR. https://github.com/ipfs/js-ipfs-api/pull/102. More to do here
• [~] 15 hours of work (minimum) on IFPS HTTP docs. Did roughly 8 hours.
  • [x] Look into automatically generating
  • [~] Manually do the rest if needed. curl everything to make sure it works. Got stuck figuring out curl, wasn't very clear
• [ ] Help out with designing a community infrastructure for meetups.
• [x] Sprint (GitHub) management
• [ ] Make the documentation around Commit Sign-Offs more friendly
• [ ] Parse IPFS readme's automatically using a CLI tool for ipfs-readme-standard.

[ghost]

• dmca / ipfs.io/refs
  • [x] deploy refs.ipfs.io https://github.com/ipfs/refs https://ipfs.io/ipns/refs.ipfs.io
  • [x] point ipfs.io/refs to refs.ipfs.io ipfs/infrastructure#95
  • [ ] actual usage blocked by directory listings bug ipfs/go-ipfs#1955
  • [ ] ping @VictorBjelkholm re: refs/openipfs
• gateway
  • [x] gzip compression ipfs/infrastructure@e80ea375010882b674cd0a49c896d25ab162c482
  • [~] open up cors on gateways (allow origin *) ipfs/infrastructure#109
  • [ ] hope: writable gateway ipfs/infrastructure#105
• org
  • [x] move pollux to protocol labs account ipfs/infrastructure#73
  • more and more back and forth
  • [x] periodic ovh renewal
• cjdns
  • [x] catch up with cjd's new cryptoauth dev
  • [~] discover peers from cjdns ipfs/go-ipfs#1316
  • [ ] invite to fc00 development (aka go-cjdns)
• infrastructure
  • [x] dashboard cleanup and style improvements
  • [x] daemon dashboard http://metrics.i.ipfs.io/dashboard/db/ipfs-daemon
  • [x] added jbenet's SSH tunnel script (metrics.sh)
• archives
  • [ ] hope: get archive inventory together ipfs/infrastructure#106
  • [ ] hope: add notes about cdnjs ipfs/archives#35
  • [ ] hope: add openwrt 15.05 on castor

[ion1]

• [~] Worked further on https://github.com/ipfs/go-ipfs/issues/1818 “max-age, ETag for /ipns requests” and https://github.com/ipfs/go-ipfs/issues/1934 “Tests for ipns-caching”, CR pending.

[kyledrake]

@kyledrake checkin

• [ ] libp2p site work
• [ ] finish DMCA work
• [ ] changes to blog post

Very busy week, as predicted.

[jbenet]

@jbenet checkin

This week I talked at futurismnyc.com/schedule/blockchain-demo-night/. Then I went to SF. It was super productive. went to http://www.weboftrust.info, met with people, and visited the Archive. Then went to London and talked at devcon1 -- http://devcon.ethereum.org.

• [ ] Code Review
  • [ ] CR go-ipfs utp
  • [x] CR go-ipfs refactored transports
  • [ ] CR ipfs-update -- if others want to help here pls do
  • [x] CR go-log -- if others want to help here pls do
  • [x] CR https://github.com/ipfs/blog/pull/8
• [x] org things
• [x] latest IPLD spec draft https://github.com/ipfs/specs/pull/37
• [x] merged @diasdavid's PRs for browser + republish etc.
• [x] Visits:
  • [x] archive visit -- discuss: https://github.com/ipfs/archives open web, and other things
  • [x] consensys demo futurismnyc.com/schedule/blockchain-demo-night/
  • [x] web of trust conference: https://github.com/WebOfTrustInfo/rebooting-the-web-of-trust + http://www.weboftrust.info/
  • [x] DPKI and "updating master keys" protocols
  • [x] keychain description
  • [x] ethereum devcon1 -- http://devcon.ethereum.org
  • [x] make devcon1 talk

[masylum]

@masylum checkin

Couldn't do anything last week :(

We are organizing a hack day in Barcelona though: https://github.com/ipfs/community/issues/67#issuecomment-154474364

[whyrusleeping]

planned from sprint start

• [ ] refactor transports
• [ ] rebase utp on transport code when done
• [x] ipfs-update to run migrations
• [ ] ipfs-update integrated into go-ipfs
• [x] go-log fixes
• [ ] ipld review
• [x] gx repos
• [x] discuss gx as a generalized extensible package manager
• [ ] ipfs add perf (#1954)

added along the way

• [x] query-vis tool

[dignifiedquire]

• [x] bisect go-ipfs https://github.com/ipfs/go-ipfs/issues/1950
• [x] Upgrade dependencies for webui https://github.com/ipfs/webui/pull/92

[NeoTeo]

Hi guys. A quick note to let you know my first pass at the Swift IPFS API client/shell is now done and passing tests and is here. Comments welcome. :)

[RichardLitt]

@NeoTeo Nice! That's awesome!

Nice pictures Juan!

Closing this in favor of ipfs/kubo#51.