search

qri-io / rfcs

Request For Comments (RFCs) documenting changes to Qri
MIT License
12 stars 6 forks source link

RFC request: cafs module for starlib #26

Open ramfox opened 6 years ago

ramfox commented 6 years ago

referencing this comment by @stuart from the Revise Transform Process RFC:

if you are going to include an IPFS module as well for pulling in other distributed datasets? Having http is a must to interact with the existing web but it seems weird not to be able to grab a distributed dataset in the downloads phase by IPFS hash or IPFSNS, or another named Qri dataset. If you are encouraging people to use Qri as their main way of sharing data, it seems restrictive to not allow them to use those datasets in the transform of another Qri dataset. As I say probably for another RFC but wanted to mention this here.

This makes sense, and I don't think we have a way of doing this right now, besides maybe calling to the the ipfs gateway using http, which seems silly (even if it works for now)

edit: see @b5 's comment for why the change from ipfs module to cafs module:

As for grabbing raw IPFS data within starlark, absolutely planned, in a few different instantiations. The first module like to rfc-then-ship is a module called cafs that provides an API for doing native interaction with our cafs interface. I have no idea if we'll ever pull this off, but by calling it cafs we leave the door open to something like cafs.fetch('/ipfs/QmFoo...') and maybe cafs.fetch('dat://4cbe2f...'), which would fetch via if the underlying cafs instance had a DAT connection (or some way to resolve dat links). cafs can currently resolve IPNS links, so that should come for free when we ship this cafs module.

All this would happen in the download step of a transform. If other IPFS tech like IPLD stabilizes we can ship 'em as separate Qri modules for starlark.