search

filecoin-project / lassie

A minimal universal retrieval client library for IPFS and Filecoin
Other
111 stars 17 forks source link

Please add support for HTTP trustless gateway retrievals. #485

Closed agmap closed 2 months ago

agmap commented 2 months ago

Please add support for HTTP trustless gateway retrievals. There are so many CIDs which have no providers but can be fetched through trustless gateway.

rvagg commented 2 months ago

http trustless gateway was developed in conjunction with its client implementation here in lassie, this is done and available

agmap commented 2 months ago

How to specify this using the CLI?

rvagg commented 2 months ago

it's on by default, if the indexer knows a provider makes blocks available via http then it'll try it, you can be explicit and just allow http with --protocols http and thereby exclude bitswap (and graphsync). Or you can be explicit with your provider and use --provider http://..., even using http://ipfs.io should work as a provider if you want to fetch something that's available via the IPFS gateway since it speaks http trustless too.

agmap commented 2 months ago

@rvagg Sometimes having CIDs where lassie finds no providers/candidates on indexer (cid.contact) or DHT. Now I have recognized I can retreive the CID through ipfs http trustless gateway. --provider http://trustless-gateway.link Unfortunately, this switches off the default fetching routine (bitswap,graphsync,http).

  1. is it possible that the default fetching routine remains enabled and if no providers are found, Lassie tries to fetch the CID from the --provider in the order I added in the CLI?
  2. Or lassie also automatically seaching on --provider http://trustless-gateway.link by default?
rvagg commented 2 months ago

no, --provider overrides the indexer lookup step because you're essentially giving it the answer it would try and get from there if you want to use it as a fall-back, you might have to come up with some kind of wrapper, or even query the indexer yourself first to find out what lassie might find when it does that alternatively if you want to try and contribute a feature here you could, perhaps a mode where --provider can feed into the indexer-supplied list of providers instead of overriding it

agmap commented 2 months ago

I would like to add this function, but unfortunately I am not able to do so.

agmap commented 2 months ago

How to add the DHT as index --ipni http://delegated-ipfs.dev is not working.