Sprint January 19th

[RichardLitt]

Sprint January 19

Moved from the 18th to the 19th.

Sprint Goals

Sprint Discussions

Schedule

Please take notes in a separate pad, if you can, and link it here.

Endeavour	Lead	Time (PDT - UTC/Z - CET)	Pad
infrastructure	Lars Gierth	18:45Z	https://public.etherpad-mozilla.org/p/ipfs-jan-18-infrastructure
libp2p	David Dias	19:00Z	https://public.etherpad-mozilla.org/p/ipfs-jan-18-libp2p
js-ipfs	David Dias	19:15Z	https://public.etherpad-mozilla.org/p/ipfs-jan-18-js-ipfs
go-ipfs	Jeromy Johnson	19:30Z	https://public.etherpad-mozilla.org/p/ipfs-jan-18-go-ipfs
apps on ipfs	Juan Benet	19:45Z	https://public.etherpad-mozilla.org/p/ipfs-jan-18-apps-on-ipfs

Please add the Agenda to the Pad before the endeavour sprint starts.

Sprint Deliverables

• Add your tasks below in a comment, that way we only have people listed who are really in the sprint
• Add links to issues down here. Only add things you can finish this sprint.

[whyrusleeping]

@whyrusleeping's TODOs:

• [ ] debug random multistream goroutines...
• [ ] fix ipfs files 'deadlocks'
• [ ] private networks
  • implement 'encrypted transport' thing
• [ ] thin out go-libp2p deps
  • go-net needs to die
• [ ] directory sharding
• [ ] node aliases (should be easy to add to config)
  • keep map in IpfsNode object, maybe in resolver too

Down the road:

• change importer to name file links
• merkledag insertInOrder

[daviddias]

David Dias TODO

• js-ipfs
  • js-ipfs (cli + http api)
  • [x] block
    • [x] put
    • [x] del
    • [x] stat
    • [x] https://github.com/webpack/webpack/pull/1930 + https://github.com/webpack/webpack/pull/1931
  • [ ] init
  • [x] CR+M https://github.com/ipfs/js-ipfs/pull/55 (browser tests)
  • [x] Debug, figure out why Buffers coming from local storage were bigger and then give support to binary data to local-storage-blob-store https://github.com/xicombd/local-storage-blob-store/pull/1
  • js-ipfs-repo
  • Work with Francisco on getting IPFS repo to work with idb https://github.com/ipfs/js-ipfs-repo/pull/26
  • js-ipfs-merkle-dag
  • [x] Be able to successfully read (and write) MerkleDAG nodes that were stored into an IPFS Repo through go-ipfs, in JavaScript https://github.com/vijayee/js-ipfs-merkle-dag/pull/6
  • [x] Documentation https://github.com/vijayee/js-ipfs-merkle-dag/pull/7
  • js-ipfs-data-importing
  • [ ] Create a protobuf version of the layouting (similar to what go-ipfs does) (a good test is creating it, adding to the repo and then let go-ipfs use it <- this will be so sweet)
  • [ ] Extrapolate learnings and merge them into the spec - https://github.com/ipfs/specs/pull/57
• api documentation
  • [x] Work on Docs with @RichardLitt
  • [x] Finish with @RichardLitt https://github.com/ipfs/http-api-spec/pull/17
• js-ipfs-api
  • [x] docs update https://github.com/ipfs/js-ipfs-api/pull/191
  • [x] CR+M https://github.com/ipfs/js-ipfs-api/pull/177
  • [x] https://github.com/ipfs/js-ipfs-api/pull/191
  • [x] https://github.com/ipfs/js-ipfs-api/pull/192
  • [x] https://github.com/ipfs/js-ipfs-api/pull/193
  • [x] https://github.com/ipfs/js-ipfs-api/pull/194
• libp2p
  • utp
  • [x] ucat with utp-native
• npm on ipfs
  • [ ] Debug and solve with @whyrusleeping latests issues on files API to get it completely ready
• specs
• extra
  • [x] R https://github.com/ipfs/pm/pull/82
  • [x] R https://github.com/ipfs/awesome-ipfs/pull/40
  • [x] CR https://github.com/whyrusleeping/js-mafmt/pull/1
  • [x] R https://github.com/ipfs/awesome-ipfs/pull/42
  • [x] CR+M https://github.com/ipfs/js-ipfsd-ctl/pull/47
  • [x] CR+M https://github.com/ipfs/js-ipfs-api/pull/196
  • [x] R https://github.com/ipfs/community/pull/96
  • [x] R https://github.com/ipfs/webui/pull/217/files
  • [x] check everyday https://github.com/issues?utf8=%E2%9C%93&q=is%3Aopen+org%3Aipfs+label%3A%22needs+review%22
  • [x] CR+M https://github.com/ipfs/pm/pull/85

[hackergrrl]

@noffle TODO

Keeping my list on the short side, since I expect ArcticJS (and my cold) to keep me pretty busy.

• [ ] go-ipfs
  • [ ] finish https://github.com/ipfs/go-ipfs/issues/2155
  • [ ] https://github.com/ipfs/go-ipfs/issues/2187
• [ ] ipget
  • [ ] create distributions entry

[whyrusleeping]

I think due to extenuating circumstances, we only really need 15 minutes for each of the sprint hangouts. Thoughts? @RichardLitt @diasdavid @lgierth @noffle

[dignifiedquire]

sounds good

[RichardLitt]

Works for me.

[dignifiedquire]

Can we start 15 minutes later than with the apps hangout please?

[RichardLitt]

Yes.

[RichardLitt]

How about:

(See above)

[ghost]

yeah shortening is cool for me

[RichardLitt]

Cool. Changed times. See first comment.

[RichardLitt]

To Dos

• [ ] Sprint Management
• [ ] Write the weekly roundup
• [ ] IPFS API
  • [ ] Add a link to the api from the specs repo
  • [ ] Add curl commands to all method descriptions
  • [ ] Check all responses have maximal entries
  • [ ] Log any errors in go-ipfs
  • [ ] Standardize formatting for ipfs --help entries, basing on 0.4
  • [ ] Open an issue about sharness testing.
  • [ ] Open a feature issue about having multiple language implementations for curl requests

[dignifiedquire]

@dignifiedquire To Dos

Work

Webui

• [ ] Extract the globe
• [ ] Finish and merge "remove jquery"
• [ ] Finish and merge "ES6/7 transition"
• [ ] Remove local storage
• [ ] New design (Photoshop)

Distributions

• [ ] Ship it
• [ ] Review blog post
• [ ] Fix download hierarchy
• [ ] Add better links

Learn

• [ ] Go

[jbenet]

@jbenet TODO

• [ ] finalize IPLD pathing issue
• [ ] finalize IPLD protobuf compat issue
• [ ] finalize IPLD spec
• [ ] finish transcribing seminar talk into blog post
• [ ] CR more things

[RichardLitt]

This Week

• [x] Sprint Management
• [~] Write the weekly roundup
• [~] IPFS API
  • [x] Split into current functions master branch and expected spec branch.
  • [x] Add a link to the api from the specs repo. PR specs/pull/69
  • [ ] Add curl commands to all method descriptions
  • [~] Check all responses have maximal entries
  • [x] Done for ipfs/add.
  • [ ] Log any errors in go-ipfs
  • [~] Standardize formatting for ipfs --help entries, basing on 0.4
  • [x] Added docs-styleguide.md to ifps/community.
  • [~] PR docs-styleguide.md links where needed.
  • [x] Open an issue about sharness testing. http-api-spec/issues/23
  • [ ] Open a feature issue about having multiple language implementations for curl requests

Extra

• [x] CR+M project-repos/pull/36
• [x] CR+M js-ipfs-api/pull/191
• [x] CR+M project-repos/pull/37
• [x] CR go-ipfs/pull/2231
• [x] Added 'needs review', 'difficulty:*' labels to every repo. community/issues/94, PR community/pulls/95, community/pulls/96
• [x] Created RichardLitt/gh-description to help with IPFS standardization, eventually.
• [x] Added log to go-ipfs about C-c. PR go-ipfs/pull/2243
• [x] Capitalized and period-appended logs in go-ipfs. PR go-ipfs/pull/2244

[hackergrrl]

This Week

As expected, ArcticJS and being sick kept me pretty busy! No blockers.

• [ ] go-ipfs
  • [x] finish https://github.com/ipfs/go-ipfs/issues/2155
  • [ ] https://github.com/ipfs/go-ipfs/issues/2187
• [x] ipget
  • [x] create distributions entry

[whyrusleeping]

DONE (week of Jan 18):

• fix go-ipfs-api object patch code
• wrote a basic multicast package (https://github.com/whyrusleeping/multicast)
• distributions fixes
• :star: wrote 'just a dht' binary (https://github.com/whyrusleeping/ipfs-dht)
• wrote simple ipfs keygen program (https://github.com/whyrusleeping/ipfs-key)
• arcticJS
  • met richardlitt and noffle in person finally
  • hackathon with mafintosh feross wa7son and others
  • :star: gave a talk (https://www.youtube.com/watch?v=fVSu_t0CsWA)
• gx work
  • gx update was doing odd things, fixed that
  • added '.gx' dir for storing repo info
  • more tests (up to four test files now!)
• learned metric ass-tons about linux network routing
  • building a distributed networking sim tool
  • dude, wow, networking. hard shit.
• started a 'generic progress' lib, frustrated with how difficult progress bars are
• made progress on ipfs files deadlocks, registry mirror is running now

[ghost]

@lgierth sprint update

Really messy sprint on my end, but the dev040 work is largely done, and provsn works nicely. jbenet had to clean up after me today, and wrote a post mortem.

• dev040
  • [x] deploy dist.ipfs.io
  • [x] dev040 gateways/bootstrappers
  • :star: that means all of ipfs.io is now served by both v03x and v04x, whatever responds faster. v03x.ipfs.io and v04x.ipfs.io themselves are exempt from this.
  • [ ] dev040 pinbot
  • [x] CR ipfs/go-ipfs#2191
• infrastructure
  • [x] use provsn for deploying ipfs
  • more about provsn in the coming days
  • [ ] ipv6
  • [ ] wildcard cert
  • [ ] wget benchmark

[daviddias]

David Dias Check In

• js-ipfs
  • js-ipfs (cli + http api)
  • [x] block
    • [x] put
    • [x] del
    • [x] stat
    • [x] https://github.com/webpack/webpack/pull/1930 + https://github.com/webpack/webpack/pull/1931
  • [x] CR+M https://github.com/ipfs/js-ipfs/pull/55 (browser tests)
  • [x] Debug, figure out why Buffers coming from local storage were bigger and then give support to binary data to local-storage-blob-store https://github.com/xicombd/local-storage-blob-store/pull/1
  • js-ipfs-repo
  • [x] Work with Francisco on getting IPFS repo to work with idb https://github.com/ipfs/js-ipfs-repo/pull/26
  • [x] Later we dropped IndexedDB in favor of localStorage, see https://github.com/xicombd/local-storage-blob-store, now fixing some issues when it comes about storing binary data, see https://github.com/ipfs/js-ipfs/issues/36#issuecomment-174563637
  • js-ipfs-merkle-dag
  • [x] Be able to successfully read (and write) MerkleDAG nodes that were stored into an IPFS Repo through go-ipfs, in JavaScript https://github.com/vijayee/js-ipfs-merkle-dag/pull/6 (🌟 MerkleDAG Impl interop)
  • [x] Documentation https://github.com/vijayee/js-ipfs-merkle-dag/pull/7
• api documentation
  • [x] Work on API Docs with @RichardLitt
  • [x] Finish with @RichardLitt https://github.com/ipfs/http-api-spec/pull/17
• js-ipfs-api
  • [x] docs update https://github.com/ipfs/js-ipfs-api/pull/191
  • [x] CR+M https://github.com/ipfs/js-ipfs-api/pull/177
  • [x] https://github.com/ipfs/js-ipfs-api/pull/191
  • [x] https://github.com/ipfs/js-ipfs-api/pull/192
  • [x] https://github.com/ipfs/js-ipfs-api/pull/193
  • [x] https://github.com/ipfs/js-ipfs-api/pull/194
  • [x] CR+M https://github.com/ipfs/js-ipfs-api/pull/196
• libp2p
  • utp
  • [x] ucat with utp-native
• npm on ipfs
  • [x] @whyrusleeping just fixed it :D Debug and solve with @whyrusleeping latests issues on files API to get it completely ready
• extra
  • [x] R https://github.com/ipfs/pm/pull/82
  • [x] R https://github.com/ipfs/awesome-ipfs/pull/40
  • [x] CR https://github.com/whyrusleeping/js-mafmt/pull/1
  • [x] R https://github.com/ipfs/awesome-ipfs/pull/42
  • [x] CR+M https://github.com/ipfs/js-ipfsd-ctl/pull/47
  • [x] R https://github.com/ipfs/community/pull/96
  • [x] R https://github.com/ipfs/webui/pull/217/files
  • [x] check everyday https://github.com/issues?utf8=%E2%9C%93&q=is%3Aopen+org%3Aipfs+label%3A%22needs+review%22
  • [x] CR+M https://github.com/ipfs/pm/pull/85

[dignifiedquire]

@dignifiedquire Sync Update

Work

Webui

• [~] Extract the globe and https://github.com/ipfs/connections-globe
• [x] Finish and merge "remove jquery" :star:
• [x] Finish and merge "ES6/7 transition"
• [x] Remove local storage
• [~] New design (lots of bad drafts, nothing good yet)
• [~] 0.4.0 File Browser
• [x] Update hjs-webpack to 7.0.0
• [x] CR https://github.com/ipfs/webui/pull/219

Distributions

• [x] Ship it :star:
• [ ] Review blog post (waiting on @richardlitt to write it)
• [x] Fix download hierarchy 1 2
• [x] Add better links
• [x] Add about section
• [x] Answering feedback https://github.com/ipfs/distributions/issues/45
• [x] Review feedback issues from @jbenet

JS IPFS API

• [x] Cleanup dependency PRs on ipfs/js-ipfs-api
• [x] CR
  • [x] https://github.com/ipfs/js-ipfs-api/pull/192
  • [x] https://github.com/ipfs/js-ipfs-api/pull/193
• [x] Update to 0.3.11 :star:
• [x] Open Issue review and PR review
• [x] Add tests for ls

JS IPFSD CTL

• [x] Update to 0.3.11 :star:

Misc

• [x] CR https://github.com/ipfs/community/issues/94
• [x] CR https://github.com/ipfs/weekly/pull/14
• [x] CR https://github.com/ipfs/js-ipfs-repo/pull/26

Learn

• [ ] Go
• [~] Learn about Redux

Teach

• [x] introduce the dark magic of bluebird to @RichardLitt

[jbenet]

@jbenet TODO

• [ ] IPLD
  • [ ] finalize IPLD pathing issue -- didnt finish it but made examples repo which helps. i think i've settled on one pathing option.
  • [ ] finalize IPLD protobuf compat issue
  • [x] Made https://github.com/ipfs/ipld-examples to see pathing uses
• [ ] finish transcribing seminar talk into blog post
• [x] multihash base discussion: https://github.com/jbenet/multihash/issues/14
• [x] surfaced notes for POST https://github.com/ipfs/POST/issues/6
• [x] bugfix / solve gateway downtime
• Review
  • [x] CR + M https://github.com/jbenet/multihash/pull/20
  • [x] CR + M https://github.com/jbenet/go-multiaddr/pull/16
  • [x] CR + M https://github.com/ipfs/go-ipfs/pull/2208
  • [x] CR + M https://github.com/ipfs/go-ipfs/pull/2021
  • [x] CR + M https://github.com/ipfs/go-ipfs/pull/2141
  • [x] CR + M https://github.com/ipfs/go-ipfs/pull/2232
  • [x] CR + M some website PRs
  • [x] Review + comment on dist.ipfs.io things
  • [x] Review + comment on firefox gateway things

[jbenet]

worth highlighting in weekly IMO:

• :star: the gateway stuff (now serving both 0.3.x and 0.4.x)
• :star: shipping dist.ipfs.io
• :star: standalone dht binary
• :star: articjs visit + talks

[ghost]

I forgot to post the notes from the infrastructure hangout on january 18th:

dmca

• denylists in go-ipfs
• double-hashing
• infra PR regarding blocking
• add notice to refs-denylists-dmca
• notice.md wording
• what to redact in a notice
  • what does github do?
• end-to-end dmca readme

ipv6

• neccessary change in bootstrap shipped
  • => go!

dev040

• a little go tool that sits between nginx and two ipfs daemons' gateways
  • proxies each request to both of the backends (v03x + v04x), simultaneously
  • we can't do this serially, because we can't reasonably tell when a timeout happens
  • the only thing we can reliably say is when a request is successful
  • requests and responses should not get buffered (RAM usage!)
  • first successful response is sent to the client
  • after successful response, request/response to other backend is aborted/discarded
  • first error is discarded
  • second error is sent to the client
• pinbot

fc00

• peer discovery e.g. via geoip or AS-number
• have a big great meeting in Q2
  • with cjd, ansuz, ehmry, (kubuxu, mildred)
  • maybe around squatconf
• routing schemes
• swarm should be called router or so