dht.findprovs does not work correctly

[mitar]

When I call dht.findprovs with one hash, I would expect that I get the same output back as running ipfs dht findprovs for go implementation. But it is not the same.

First, if I do documented:

http://localhost:5001/api/v0/dht/findprovs?arg=<key>&verbose=false

The output is a bunch of JSON strings one after another, of various types. It seems like a dump of ALL providers my instance knows about, not just about the key I provided. A very different output than go version in the terminal.

As a consequence, the same happens also through this API.

My versions:

agentVersion: go-ipfs/0.4.5/ protocolVersion: ipfs/0.1.0 js-ipfs-api: 12.1.7 (master branch at 9b8fed040dfe57c8a084f516b18d1eac44cf853f)

[dignifiedquire]

@whyrusleeping @Kubuxu could you shed some light on why this difference exists?

[daviddias]

@mitar is this still an issue?

[Kubuxu]

Yes, it will still be an issue. JSON output includes all notifications that are being executed, CLI just filters them and outputs. js-ipfs-api should probably include consts defining DHT notification types.

[vasco-santos]

After a deep analysis of this problem, I managed to understand what is received using the js-ipfs-api. Shortly, go-ipfs, and more precisely, go-libp2p-routing uses routing notifications for exchanging routing information among peers, such as the findprovs.

Unfortunately, in the js-ipfs context, findprovs core component is still not being used, along with all dht core features. So, I went through all the js-libp2p modules, in order to find the implementation of the notifications, but I couldn't manage to find them. Is this not implemented yet? Or is there a different approach?

Going deep into this issue, using a GO node with the js-ipfs-api. When executing dht.findprovs(<key>), a bunch of JSON strings are received, which correspond to all the notifications received by the peer, as implemented here. Comparing the obtained output with the CLI response, I understood that the CLI shows the notifications with "Type": 4. Accordingly, notification types are defined in go-libp2p-routing#L16 and "Type": 4 consists of a Provider notification.

After this analysis, the way to uniform the output result is to filter the notifications received based on the notification type. However, in the JS implementation of ipfs, as well as libp2p, I couldn't find any reference to those notifications. Therefore, I would like to know what are the plans regarding this theme, since it may not make sense to filter those notifications by type if the JS implementation will not embrace this approach @diasdavid .

NOTE: It is also important pointing out (maybe for new contributors like me) that when using js-ipfs-api with a go-ipfs node, it is necessary to set Access-Control-Allow-Origin in order to use the API. This was a blocker to me, and I needed some time to understand how I could do this. Luckily, there is an easy way to achieve this referenced in https://github.com/ipfs/ipfs/issues/75

[olizilla]

NOTE: It is also important pointing out (maybe for new contributors like me) that when using js-ipfs-api with a go-ipfs node, it is necessary to set Access-Control-Allow-Origin in order to use the API.

https://github.com/ipfs/js-ipfs-api#cors documents the CORS setup. Perhaps it should be mentioned sooner in the doc.

[victorb]

Perhaps it should be mentioned sooner in the doc

@olizilla I'm unsure. It's mentioned in the readme as many people hit that issue, but it's also already pretty far up + is not a IPFS issue or specific to how go-ipfs works, it's simply how the web works and does not happen in all scenarios, only when you do cross-origin requests. it is necessary to set Access-Control-Allow-Origin in order to use the API is only true when doing cross-origin requests, which is not necessarily true always.

[vasco-santos]

Yes, I agree that it is clear enough. I didn't get it in the README.md because I only got the problem using a GO peer. As I read README.md when I started using it with js-ipfs I did not recall the CORS section.

@VictorBjelkholm It is strange that in the same conditions that I used both implementations, it only occurred in GO. And that is why I think we may add additional information in the README.

[daveseco7]

Hi @diasdavid @alanshaw @vasco-santos, is the approach of filter out the results other then provider, the best approach here? I would be interested in submitting a PR with that approach. Further more on this subject, shouldn't the return of a call to ipfs.dht.findprovs(_hash_) be an simple array of Strings containing only the hashes of the providers ? As occurs in the CLI.

[vaibhav6233m]

I am using thejs-ipfsd-ctl and js-ipfs-api inside one of my electron application.

As per @vasco-santos comment,

Going deep into this issue, using a GO node with the js-ipfs-api. When executing dht.findprovs(), a bunch of JSON strings are received, which correspond to all the notifications received by the peer, as implemented here. Comparing the obtained output with the CLI response, I understood that the CLI shows the notifications with "Type": 4. Accordingly, notification types are defined in go-libp2p-routing#L16 and "Type": 4 consists of a Provider notification.

I came to know that the actual providers for the given hash are all "Type": 4. All other records should be some routing notifications for exchanging routing information among peers. This info could be really helpful for the people who wants to use dht functions in their projects.

Refer Screenshot:

[whizzzkid]

js-ipfs is being deprecated in favor of Helia. You can https://github.com/ipfs/js-ipfs/issues/4336 and read the migration guide.

Please feel to reopen with any comments before 2023-06-05. We will do a final pass on reopened issues afterward (see https://github.com/ipfs/js-ipfs/issues/4336).