search

ipfs / team-mgmt

IPFS Team Planning, Management & Coordination threads
https://github.com/ipfs/team-mgmt/issues
Other
267 stars 97 forks source link

Sprint Nov 23 #56

Closed daviddias closed 8 years ago

daviddias commented 8 years ago

Sprint Nov 23

Sprint Goals

Schedule

Please take notes in a separate pad, if you can, and link it here.

Endeavour Lead Pad Permanent Link
Apps on IPFS Juan Benet and Friedel Ziegelmayer https://public.etherpad-mozilla.org/p/nov-23-apps-on-ipfs
infrastructure Lars Gierth https://public.etherpad-mozilla.org/p/nov-23-infrastructure
libp2p David Dias https://public.etherpad-mozilla.org/p/nov-23-libp2p0-and-js-ipfs https://github.com/ipfs/pm/issues/56#issuecomment-159072713
js-ipfs David Dias https://public.etherpad-mozilla.org/p/nov-23-libp2p0-and-js-ipfs https://github.com/ipfs/pm/issues/56#issuecomment-159072713
go-ipfs Jeromy Johnson https://public.etherpad-mozilla.org/p/nov-23-go-ipfs
testing + ci Jeromy Johnson https://public.etherpad-mozilla.org/p/nov-23-testing-ci
protocol + specs Juan Benet https://public.etherpad-mozilla.org/p/nov-23-protocol-specs
bitswap ml Juan Benet https://public.etherpad-mozilla.org/p/nov-23-bitswap-ml
Data Structures Juan Benet https://public.etherpad-mozilla.org/p/nov-23-data-structures

Please add the Agenda to the Pad before the endeavour sprint starts.

Sprint Deliverables

dignifiedquire commented 8 years ago

IPFS Apps

Lead: Friedel Ziegelmayer

Agenda

IPFS (HTTP) API (SPEC)

Participants

Notes

IPFS (HTTP) API (SPEC)

Questions

@dignifiedquire: Who has implemented the original version? Is thera short breakdown on how this came to be?

@jbenet: The goal for the http API was for it to be able to be layered over any transport. Some people may want to do non-http transports, like UNIX main sockets. Sometimes yuo don't even have HTTP over the wire. Another example woudl be websockets; ie, piping all requests through amux over the daemon, etc. The goal was to design an API that was simply patterened so that it could be layered. We just implemented the HTTP one first. The commands library supports reading and writing with JSON on HTTP and also over the CL. That means all of the regular ways of specifying commands and arguments / options need to be have a ways to be transported to the command in question. So, flags, pathing, query strings, etc.

@dignifiedquire: Ok. There are some main issues. Richard, I know you've been trying to get some actual stuff done using http commands. We also have the js-ipfs-api docs, because that is kind of a documentation. So, today, how does the API as it is now implemented? Ie, returning JSON, chunks, newline-delimitered, etc.

@jbenet: Yeah, we don't even have a thing where that is documented. Another thing is that the core API - what API means - also needs to be defined. API could be a process around the daemon, or actually an API for the libraries. What would be good would be to have the core as close as possible to the daemon, so that you coudl follow the same for all. We're a little bit far from that, but we're close.

@dignifiedquire: So the goal wouldbe to have a spec for the API commands and then a version of that which would structure how the API works.

@dignifiedquire: @diasdavid said earlier that it's not just about documenting, but we also need a Spec that the implementation follows. Ideally, a spec that we could generate a test feed from would be good. Rather than having to guess and think and hope, which is the current scope.

@jbenet: Are there any good versions of that?

@dignifiedquire: That's why I've been looking at apiary. They have two way support. If you write a spec in this format, there are tools for generating mock servers.

@jbenet: One constraint is that the spec will have to be more general than the HTTP. It should be idiomatic, but that should be optional. @RichardLitt: We need to make sure that the main source of truth is the HTTP spec, so that each implementation should talk to that.

@jbenet: one possibility is to make a spec for the core API, and then just go from there. ...<@RichardLitt was talking>

@dignifiedquire: So, HTTP is just a subset of what we need. So, we should have a high level, and then use tooling to convert it into Swagger, or whatever.

@jbenet: https://github.com/ipfs/specs/blob/master/modules/modules.go. This shows off how go might be used as an interface langauge.

@jbenet: another approach is enumerate all of the functions in a file, and then enum. all of the ways that interfacing components may do that. We have the ability to: positional arguments, flags, mention files, and that's when we start hitting things which are less defined. When do we need streaming? When do we need APIs such that we are going to be able to stream the output out, like with huge dirs where you need to stream the output as you stream the input. @dignifiedquire: yeah, we'll need that.

@jbenet: Goals!

(https://github.com/ipfs/api/issues/4)

From that, we can figure out which language is best. (Go may not be best, because of streaming interfaces.). From there:

So, start with the core, and then go from there. init, for instance, isn't in the core. But maybe it should be. Maybe there's an argument for one api - client API, and library apis.

@dignifiedquire: I think it would be good to have them all in the same spec. Let's just make a huge list of markdown in ipfs/api right now. Also,

Talk in the https://github.com/ipfs/api/issues please!

Possible Solutions

Would be interesting if it was RESTful rather than RPC, but probably not worth the effort to design a whole new API. – ion

Thanks everyone and sorry @diasdavid you couldn't be here :stuck_out_tongue:

whyrusleeping commented 8 years ago

go-ipfs

Lead: @whyrusleeping

Agenda

Participants

potentially interested:

daviddias commented 8 years ago

libp2p and js-ipfs

Lead: David Dias

Agenda

You have 30 minutes for this agenda, 5 minutes before the meeting ends, consider moving the remaining items to a github discussion thread so that all the other sprint meetings can start in time.

Participants (write your name on the list)

Notes

@jbenet: I pulled the libp2p slides out of my qconf talk https://ipfs.io/ipfs/QmRthja4NnZEwXcUNgTKpX86dtGMQxstMbhgnTzmVhrW4D/libp2p-part/ipfs-018.libp2p-part.pdf

should this be in the agenda, or is it for the github issue:

js-ipfs

js-ipfs-api

RichardLitt commented 8 years ago

My Todos:

dignifiedquire commented 8 years ago

TODO(@dignifiedquire)

I am not allowed to commit any code unrelated to spec writing and tooling around that.

harlantwood commented 8 years ago

Testing + CI

Lead: @jbenet??? anyobdy else?

Agenda

Participants

@harlantwood, @lgierth, @jbenet

Notes

ci-status

Random topics

harlantwood commented 8 years ago

@harlantwood

jbenet commented 8 years ago

Frozen from: https://public.etherpad-mozilla.org/p/nov-23-data-structures

Data Structures

Lead: @jbenet

Agenda

Participants

Notes

Does a POST have some sort of application ID (if there are multiple versions of an application)?

@fazo96 -- ipfs-boards:

@nginnever -- SwarmChan:

Keychain:

Other Questions -- a set of other datastructures we can parametrize or stub-out. don't need to solve them in POST.

Important requirements:

IPLD

Large-scale processing

yacy

Things to do:

Other organizations to involve

TODOs:

stay close to reality: use the apps already being built to play with datastructs and make sure these things are actually good + fun to program with! if not, wont work.

masylum commented 8 years ago

TODO @masylum

rht commented 8 years ago
whyrusleeping commented 8 years ago

TODO:

jbenet commented 8 years ago

@jbenet:

I'll add some more later today/tomorrow.

daviddias commented 8 years ago

If you feel your task list isn't puffed enough, you can always add this one:

:D

daviddias commented 8 years ago

On the continuous endeavour to improve the libp2p spec, I've summited some PR and categorised issues on the ipfs/specs repo, which would be great if people interested could review :) https://github.com/ipfs/specs/labels/libp2p

jbenet commented 8 years ago

@jbenet's update

Hey everyone, not sure i can make irc sync this morning. Am pretty sick, and still recovering. I'll try to-- but stayed up late catching up with some stuff. (this week was hosed for me-- between family time, getting sick, and org todos, i didnt have much time for my dev todos)

dignifiedquire commented 8 years ago

@jbenet get well soon 😷

dignifiedquire commented 8 years ago

@dignifiedquire update

daviddias commented 8 years ago

@jbenet get better soon!

SPRINT UPDATE

RichardLitt commented 8 years ago

Officially this week:

Extra

whyrusleeping commented 8 years ago
ghost commented 8 years ago

My week was kinda dominated by the NPM import (a.k.a. registry-mirror) and its performance issues. On the other hand got a couple of smaller things done. Also got p2p/discovery/cjdns.go to work, (it's lovely,) just need to add tests.

ghost commented 8 years ago

Sorry for my lack of presence :( not cool to be absent without any notice

rht commented 8 years ago

Merging https://github.com/ipfs/go-ipfs/pull/1964 & https://github.com/ipfs/go-ipfs/pull/1973 are needed to unclog distributed archival of content larger than 50 GB (e.g. wikipedia), otherwise it takes ages for those who attempted.

RichardLitt commented 8 years ago

thanks @lgierth: no worries. :)