New Tutorial: libp2p coding challenges with interaction btw multiple nodes

[terichadbourne]

Once we enable libp2p validation under the hood, we hope to create our first libp2p tutorial.

Some discussion about content has already happened within the libp2p validation issue and on related calls, but this issue can be used as a place to clarify the proposed outline and track progress when we start drafting content.

[zebateira]

Instead of going with a code challenge tutorial, we can first create an introductory tutorial as multiple choice quizzes from some talks we've given in the past:

1. "The Life of a libp2p Connection" by Jacob

• Goes through a lot of the basic concepts to understand what is libp2p and how it tries to solve a lot of the problems in networking
  2. "Solving distributed networking problems with libp2p" by Jacobheun at IPFS Camp
• Goes more in depth on some components of the libp2p
• Talk slides: https://github.com/ipfs/camp/tree/master/CORE_AND_ELECTIVE_COURSES/CORE_COURSE_B
  3. "libp2p: Modular peer-to-peer networking stack" by Raulat at CrossLink Taipei 2019
  4. "Introduction to libp2p" by David
• Introduction to libp2p from "why we need it", "how we are doing it and what is done so far" and "what's the plan for the future".

I'll use the first talks from Jacob as a base and then I'll incorporate some pieces from the other ones.

[terichadbourne]

Yesterday we published our first libp2p tutorial, a multiple-choice, code-free introduction to libp2p. Huge thanks to @zebateira for taking the lead on it!

@zebateira is actively working on the in-memory transport and additional under-the-hood needs for libp2p code validation that will allow us to build coding challenges in the future in which multiple nodes can interact. Please see issue #229 for continued updates on that effort.

Moving forward, we'll use this issue as the place to reflect the status of our first code-based libp2p tutorial, to be built by @yusefnapora. He'll be adding an outline here for our joint review and then will start building the lesson content of the tutorial as @zebateira wraps up the in-memory transport. Once the transport is ready, @yusefnapora will be able to build the code challenges, for which we already have a few ideas bouncing around in #229.

Super excited about the progress we're making on this!

[yusefnapora]

Here's what I have so far for the tutorial outline. @terichadbourne & @zebateira please let me know what you think. There are still a lot of details to work out, of course.

Structure for libp2p hands-on tutorial

Lesson 1 - The libp2p Node & peer IDs

• Introduce the tutorial and describe what we'll acomplish / set expectations
• Introduce the libp2p Node object
  • Not focused on configuration yet, this is about describing what it's capable of
• Introduce Peer IDs
  • show examples & explain that peer ids are multihashes (link to definition) of public keys.
• Demonstrate how to get the peer id from a running node
• Maybe: Q/A challenge:
  • What is a Peer ID?

• Coding challenge:

  • return the peer id of a pre-configured libp2p node

  function getPeerId(libp2pNode) {
  const peerId = // fill in the blank!
  
  return peerId;
  }

Lesson 2 - Dialing another peer

• Describe how we open connections to other peers by "dialing" them
  • Explain that there are two dial methods, dial and dialProtocol
  • we focus on dial for now, next lesson covers dialProtocol
  • We can emphasize that we dial by peer ID, not address. Explain that libp2p tries to find the right address and keep track of it for us.
  • for now, gloss over how peer addresses are tracked / discovered.
• Explain that peers will be "listening" and waiting for us to dial them
  • Don't show the mechanics of listening yet (registering protocol handler, etc)

• Coding challenge

  • Given a libp2p node and the peer id of another node, dial them using their peer id

  function dialPeer(libp2pNode, peerId) {
  
  // dial the peer at peerId
  // e.g.: const conn = await libp2pNode.dial(peerId)
  }
  

  • implementation note:
  • we can pre-populate the peerstore of each node with the in-memory address of the other peers, so we can dial using just peer ids without needing peer discovery
  • for validation, we just want to verify that the user was able to connect to the other peer

Lesson 3 - Protocols

• Explain how the dial method used in the last lesson gives you a connection that can carry many streams.
  • Define multiplexing as the term for this behavior & link to concept docs.
• Explain Protocol IDs and how they relate to dialing / listening
• Show an example of calling conn.newStream on the connection object from previous lession
• Also show how to use libp2p.dialProtocol to avoid the connection object entirely
• Introduce a simple example protocol
  • e.g. ping, echo, random fortune cookie quote etc.
  • Should be dead simple, e.g. just writes a message and closes the stream
• Show an example of using dialProtocol to connect to a peer using the protocol ID for our example protocol

• Coding challenge

  • given a libp2p node, a remote peer id, and the protocol ID string, open a new stream to the peer

  function dialPeerProtocol(libp2pNode, peerId) {
  const protocolId = "/protoschool/example-proto"
  
  // e.g.: const { stream, protocol } = libp2pNode.dialProtocol(peerId, protocolId)
  }

Lesson 4 - Implementing a protocol

• Demonstrate how to implement the listening side of the protocol we dialed in previous lesson.
• Explain what a protocol handler function is and how to wire one up to the libp2p node.
• Show how data can be written to / read from a stream.

• Coding challenge:

  • attach a protocol handler to the libp2p node that sends a message to peers when they connect

  • question: should we send plain text (newline delimited?), JSON objects, etc?

  • TODO: flesh out details & add coding challenge

Lesson 5 - Configuration

• Show the user how to create their own libp2p node by walking them through the configuration

• This lesson is at the end because much of the configuration is confusing unless you know what it's for

  • e.g. telling a user to configure a stream multiplexer before they know what multiplexing is

• Walk through each of the most common configuration options

  • addresses
  • we should have a quick definition of multiaddrs that links to docs
  • transports
  • security
  • multiplexing

• Explain how some options will be different in a production context

  • especially the transport. if we want them to construct a functional node, we need to explain what the in-memory transport is and why they shouldn't use it in a real app.

• Coding challenge:

  • TBD... some options:
  • start with a partially filled in config and have the user fill in the blanks. they succeed if they can start a libp2p node using the config
  • start with a fully filled in real config, but ask the user to modify it in some way. to validate, we don't try starting a node, just check the config for correctness.
    • advantage here is that we could ask them to configure things that won't actually work in the browser, e.g. "make a configuration that enables the TCP transport for node JS" or something. Might be confusing though

[terichadbourne]

@yusefnapora Thanks so much for the outline. Looking great!

I'll leave it to @zebateira to comment on the content itself, but just jumping in to say that in some cases I suspect you have too much content per lesson. The thought process I'd encourage here is: "What's the one thing I'm teaching and then testing in this lesson?"

For your current lesson 1, for example, I suspect that these bullets could be pulled out into 1 or 2 prior lessons to teach about the libp2p node (perhaps with a multiple choice quiz) separate from the peer IDs:

Introduce the tutorial and describe what we'll acomplish / set expectations

Introduce the libp2p Node object

Not focused on configuration yet, this is about describing what it's capable of

I don't have enough technical context to judge if there are other spots here where you could split a lesson into two, but would encourage you to consider it where it works.

Also I saw one spot where it looked like you were considering putting a quiz and a coding challenge in the same lesson. You can only do one format per lesson, so breaking the content apart into the chunks you want to test with each method would be required here.

[yusefnapora]

That's great feedback, thanks @terichadbourne. Lesson one could easily be split into two, and there are probably a few other places that are overloaded and could be split up. I'll work on that today and try to add more details to the bits that are still fuzzy.

[zebateira]

Love the outline so far! Really great simple initial concepts walk-through from the start. I was a little skeptical about the idea of leaving the configuration to the end, but I think I really like the outline as it progresses, good job!

I think Teri is right that this outline pushes a lot of content into each lesson.

Here is the initial tutorial structure for the content you presented better suited for our usual lesson length:

Lesson 1: The libp2p node

- Introduce the tutorial and describe what we'll accomplish / set expectations - Introduce the libp2p Node object - Not focused on configuration yet, this is about describing what it's capable of - Maybe QA: what is the libp2p node?

Lesson 2: Peer IDs

- Introduce Peer IDs - show examples & explain that peer ids are multihashes (link to definition) of public keys. - QA challenge: What is a Peer ID?

Lesson 3: Get Peer ID

- Demonstrate how to get the peer id from a running node - Coding challenge: return the peer id of a pre-configured libp2p node ```js function getPeerId(libp2pNode) { const peerId = // fill in the blank! return peerId; } ```

Lesson 4: Dialing another peer

- Describe how we open connections to other peers by "dialing" them - Explain that there are two dial methods, dial and dialProtocol - we focus on dial for now, next lesson covers dialProtocol - We can emphasize that we dial by peer ID, not address. Explain that libp2p tries to find the right address and keep track of it for us. - for now, gloss over how peer addresses are tracked / discovered.

Hope this makes sense, so you can restructure it from here.

Thank you! 🙏

[yusefnapora]

That sounds great @zebateira - maybe it's best to save the "how to build a protocol" stuff for a separate follow-up tutorial.

That will let us dig in a little deeper and make a (slightly) more involved example. I was thinking maybe a knock-knock joke protocol, since it has a nice "call and response" structure like a real protocol 😄

[zebateira]

maybe it's best to save the "how to build a protocol" stuff for a separate follow-up tutorial.

@yusefnapora I think that makes sense as well 👍

That will let us dig in a little deeper and make a (slightly) more involved example. I was thinking maybe a knock-knock joke protocol, since it has a nice "call and response" structure like a real protocol 😄

Good idea! Adding fun to the tutes works really well 💃

[DantewithZYX]

For upcoming meetups, I would really love to see lessons on DHTs, Content Routing and discovery algos in use and relative uses cases, ie pub/sub, gossipsub, graph-sync. These topics seem to bring the earlier concepts full circle.