RangerMauve / dat-store

A CLI tool for backing up hyperdrive datasets
GNU Affero General Public License v3.0
59 stars 14 forks source link

SDK pull a hypercore from dat-store? #34

Open DougAnderson444 opened 4 years ago

DougAnderson444 commented 4 years ago

Hi,

Does dat-store also pin hypercores easily?

I'm able to add hypercore url to the dat-store, but not seeing them appear as a peer with the Hypercore discovery extension. Wondering if the dat-store supports hypercore discovery, or just hyperdrive.

I'm thinking if I can get the SDK to pull a hypercore from dat-store, then I could more easily replicate the hypertrie without all the complex discovery,peer,pipe,ws,stream messiness.

Thoughts?

  // hypercore(storage, [key], [options])
  const feed = new Hypercore(rawStorage, myPinnedHypercoreURL, {
    persist: true,
    extensions: ['discovery']
  })

  feed.on('peer-add', (peer) => {
    console.log('Got a peer!')   //never fires for pinned hypercore :(

   // read the pinned hypercore
  // make a hypertrie from the downloaded hypercore 

  })
})
RangerMauve commented 4 years ago

It doesn't support hypercores yet, I don't think, but I'd be happy to take a PR for supporting it. We might want to modify the Pinning service DEP with changes to specify when we're pinning a hypercore.

DougAnderson444 commented 4 years ago

As much as I'd love to do a PR for supporting hypercore, that is likely over my head at this point (but I'll try), I'm still trying to figure out how to get the hypercores to replicate. I've been relying on Dat and the SDK to do all that magic for me.

RangerMauve commented 4 years ago

Hmm, I'll have to look into it later this month. It shouldn't be too hard to whip up a server that replicates a set of hypercores and have a simple API for adding more to it though. 🤔