docs: Helia providing content

SgtPooki commented 1 month ago

We had a user reach out to us on slack about their content not being provided in the DHT. They were unaware that Helia does not provide content by default.

This is a different experience than Kubo, and we have no documentation mentioning this requirement.

What can we do:

update docs to mention what expectations should be
update Helia 101 example with providing steps
Automatically provide when content is added
Clarify when a user should understand "provide" to be completed
- What does it mean that the promise from .provide resolves?
- Should users be checking for how many FINAL_PEER events they get?

SgtPooki commented 3 weeks ago

gr33nT commented 1 week ago

Hey! I started building a project with Helia to learn about p2p and ipfs...

Is this still an issue?

My project is basically complete except for one problem:
Using the check tool, I can find my CID's if I also include the peer id of my helia node, but not with the CID alone..

Successfully connected to multiaddr
Found multiaddr advertised in the DHT
X Could not find the multihash in the DHT or IPNI. <- issue here
The peer responded it has the CID

I'm having a hard time finding any documentation on having a helia node advertise CIDs on the DHT. If anyone is willing to point me in the right direction or show me how to do this, I would be very grateful.

SgtPooki commented 5 days ago

@gr33nT Hey, did you provide your content explicitly? currently, Helia does not do this by default.

This issue is tracking the work required to document this more clearly so folks don't run into the issue you did :)

We are still not providing by default, and won't for the foresee-able future, but it's something we're open to discussing

gr33nT commented 4 days ago

@SgtPooki Thanks for your response! So, I was NOT explicitly providing and that would explain everything 😅. Adding helia.routing.provide(fileCid); seems to work, i just have to straighten out my network config to be a little more reliable. Solving this has lead to a some more questions, however.

I would suspect the entry in DHT expires every so often and I would need to re-provide files? Is this done automatically, or does re-providing also need to be explicit? Is the information needed to do this persistent in the datastore?

also, if for whatever reason, I have files already in the blockstore but not provided, is there an easy way to loop through all the CID's and provide them again? Say, when the server starts?

I created my node like this

  const blockstore = new FsBlockstore(process.env[`BLOCKSTORE_PATH_${env}`]);
  const datastore = new FsDatastore(process.env[`DATASTORE_PATH_${env}`]);
  const agentVersion = `Helia ${libp2pInfo.name}/${libp2pInfo.version} UserAgent=${process.version}`;
  const libp2p = await createLibp2p({
    privateKey: libp2pKey,
    addresses: {
      listen: [
        `/ip4/0.0.0.0/tcp/${process.env[`LIBP2P_PORT_${env}`]}`,
        '/p2p-circuit',
        '/webrtc',
      ],
      announce: [
        `/ip4/${process.env[`LIBP2P_URL_${env}`]}/tcp/${process.env[`LIBP2P_PORT_${env}`]}`,
      ],
    },
    transports: [
      tcp(),
      circuitRelayTransport(),
      webRTC(),
      webRTCDirect(),
      webSockets(),
    ],
    connectionEncrypters: [
      noise(),
      tls(),
    ],
    streamMuxers: [
      yamux(),
    ],
    peerDiscovery: [
      mdns(),
      bootstrap({
        list: [
          '/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN',
          '/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb',
          '/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt',
          '/dnsaddr/va1.bootstrap.libp2p.io/p2p/12D3KooWKnDdG3iXw9eTFijk3EWSunZcFi54Zka4wmtqtt6rPxc8',
          '/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ',
        ],
      }),
    ],
    services: {
      autoNAT: autoNAT(),
      dcutr: dcutr(),
      aminoDHT: kadDHT({
        protocol: '/ipfs/kad/1.0.0',
        peerInfoMapper: removePrivateAddressesMapper,
        validators: {
          ipns: ipnsValidator,
        },
        selectors: {
          ipns: ipnsSelector,
        },
      }),
      identify: identify({
        agentVersion,
      }),
      identifyPush: identifyPush({
        agentVersion,
      }),
      ping: ping(),
      delegatedRouting: () => createDelegatedRoutingV1HttpApiClient('https://delegated-ipfs.dev/routing/v1'),
      relay: circuitRelayServer(),
      upnp: uPnPNAT(),
    },
  });

  return createHelia({
    datastore,
    blockstore,
    libp2p,
  });
}

const helia = await createNode();
console.log(helia.libp2p.getMultiaddrs());
const fs = unixfs(helia);

I've been adding content by doing: const fileCid = await fs.addBytes(buf); and now, helia.routing.provide(fileCid);

I apologize if these questions seem basic or if Im missing something obvious in documentation. Any help or pointers would be greatly appreciated

ipfs / helia

docs: Helia providing content #657