Paratii-Video / paratii-db

Mongodb mirror of the Paratii's video contracts, used for indexing, querying and aggregating video info.
GNU General Public License v3.0
4 stars 2 forks source link

Integrate a cache level based on IPFS, makes sense? #83

Open geckoslair opened 6 years ago

geckoslair commented 6 years ago

Something like:

  1. portal -> generic request -> db (could be a search on video API, a video play page etc. )
  2. db -> response -> portal: DB saves the entire payload on IPFS and answers to the portal just with the ipfsHash of the payload and an expiring date, something like:
{
query: "the query from the portal",
ipfsPayload: "the ipfs hash of the payload",
expiresIn: "a date that represent the payload validity"
}
jellegerbrandy commented 6 years ago

It is a great idea, very much in the ipfs spirit.

The problem is only that ipfs is much more slower than http - so transfering the data over ipfs would just slow it down. So that is not good (but perhaps we can send the data themselves together with the response)

What is interesting is that such a thing could help with decentralizing also the paratii-db - i.e. there would be the central paratii-db service that seeds ipfs hashes of query results, but even if the service goes away, clients could still get the query results from peers. (to work like that we would have to invent more details, but it is intriguing)

felipegaucho commented 6 years ago

To see if I get the idea right: the DB would normally respond over http, but still upload to IPFS a hash of the response payload and expiry date for the query? And then nodes could query for peers that have cached the response for a particular query they want to repeat, besides going directly to the DB over http?

This may be relevant here https://github.com/ipfs/notes/issues/161

geckoslair commented 6 years ago

Yes this is the idea, I've discussed with @jellegerbrandy and it comes out that we need also to know what is the query/key in order to get the right ipfs cached payload, that means that we need to register on a contract, called Index.sol or whatever, something like:

{
 query : '{keyword: "cat"}', -> the query
 ipfsPayload: 'hash to the ipfs payload',
 ttl: 'a time that tells how long this payload is valid'.
}

this will bring the db to a more decentralized state.