ipfs / helia

An implementation of IPFS in TypeScript
https://helia.io
Apache License 2.0
986 stars 106 forks source link

feat: create `@helia/http` library without bitswap/libp2p deps #289

Closed SgtPooki closed 10 months ago

SgtPooki commented 1 year ago

Helia consumers should be able to use Helia without depending on the p2p network. Ideally, users could use Helia modularly, with trustless-http-gateway (or other source of getting blocks) and validate those blocks, using only the IPFS portion of Helia.

Current

It doesn't seem like we can "remove" libp2p entirely, but we can disable it:

const helia = await createHelia({
  blockBrokers: [
    trustlessGateway()
  ],
  libp2p: {
    start: false,
    connectionManager: {
      minConnections: 0,
    },
  }
})

Future (with @helia/http)

No runtime js-libp2p deps are pulled in with this module

import type { Helia } from '@helia/interface'
import { createHttpOnlyHelia } from '@helia/http' // export name to be decided. should probably be `createHelia`
import { unixfs } from '@helia/unixfs'

const helia: Helia<null> = createHttpOnlyHelia() 

const fs = unixfs(helia)

// ...
### Tasks
- [ ] https://github.com/ipfs/helia/issues/341
- [x] create @helia/http package
- [x] @helia/http should accept an interface of only blockstore+blockBrokers that should work like Helia's networked blockstore.
- [ ] Add `@helia/http` usage example to readme
achingbrain commented 1 year ago

Maybe we should accept false for libp2p for this?

const helia = await createHelia({
  blockBrokers: [
    trustlessGateway()
  ],
  libp2p: false
})
achingbrain commented 1 year ago

Thinking about this a bit more, if there's no libp2p/bitswap then you're pulling in a whole lot of extra deps for zero benefit and I wonder if this is even Helia any more.

It might be another implementation of the Helia interface? This keeps it compatible with @helia/unixfs etc.

SgtPooki commented 1 year ago

I think Helia can be Helia even without libp2p (people don't want to do p2p stuff, but they do want to do content-addressing stuff), but I agree it feels odd.

It might be another implementation of the Helia interface? This keeps it compatible with @helia/unixfs etc.

@achingbrain are you thinking of going with something like

import {type HeliaRaw} from '@helia/interface`
import {createHelia} from 'helia'

const helia: HeliaRaw = await createHelia({
  blockBrokers: [
    trustlessGateway()
  ],
  libp2p: false
})

so modules like @helia/unixfs can indicate whether they support Helia without libp2p or not?

achingbrain commented 1 year ago

modules like @helia/unixfs can indicate whether they support Helia without libp2p or not?

Yes! The factory for @helia/unixfs only requires a blockstore for this exact reason.

So it's more like:

import type { Helia } from '@helia/interface'
import { createHeliaThatOnlyUsesGateways } from '@helia/http'
import { unixfs } from '@helia/unixfs'

const helia = createHeliaThatOnlyUsesGateways()
const fs = unixfs(helia)

// ...do unixfs stuff

The Helia type is already there so you don't need HeliaRaw and it's generic for the libp2p type so you can make it unknown or perhaps we'd remove it entirely.

SgtPooki commented 1 year ago

Got it, so we can create @helia/http and import that instead of helia, and that would handle the appropriate configuration of the Helia interface. I think that makes sense.. and putting it in a separate package would mean we can ensure other deps (libp2p/bitswap/others) are not ~pulled in~ exported.

Sounds good to me, going to update this title to reflect this? but I still imagine this is low-priority.

SgtPooki commented 12 months ago

I've added what I think to be the general action items for this into the issue description

BigLep commented 12 months ago

Thanks @SgtPooki - sounds good. Lets also make sure there is accompanying information too. There should probably be a Helia readme item for something like "want an even lighterweight Helia that still has the goodness of CIDs and verifiable retrieval? see..."

meandavejustice commented 12 months ago

Instead of modifying Helia interface we could extend and omit the libp2p bits

interface HeliaHTTP extends Omit<Helia, 'start'|'stop'|'libp2p'> {}
meandavejustice commented 11 months ago

@SgtPooki I have a branch where I'm working on this. Can you assign to me?

2color commented 11 months ago

Currently @meandavejustice is working on this in https://github.com/meandavejustice/helia-http/

SgtPooki commented 10 months ago

This is now being covered by https://github.com/ipfs/helia/pull/372