bug: kubo doesn't announce itself as provider

SgtPooki commented 2 weeks ago

Checklist

[X] This is a bug report, not a question. Ask questions on discuss.ipfs.tech.
[X] I have searched on the issue tracker for my bug.
[X] I am running the latest kubo version or have an issue updating.

Installation method

ipfs-desktop

Version

0.28.0 
https://github.com/SgtPooki/repro-kubo-not-announcing-self-as-provider/blob/5fa8b9cfad9b7f5b0a991d7efc58062f824591d8/package.json#L17

Config

config is via ipfsd-ctl in https://github.com/SgtPooki/repro-kubo-not-announcing-self-as-provider/blob/5fa8b9cfad9b7f5b0a991d7efc58062f824591d8/index.js#L16-L40

  {
    kuboRpcModule: kuboRpcClient,
    ipfsBin: kuboPath(),
    // test: true,
    ipfsOptions: {
      config: {
        Addresses: {
          Swarm: [
            '/ip4/0.0.0.0/tcp/4001',
            '/ip4/0.0.0.0/tcp/4002/ws'
          ],
          Gateway: '/ip4/127.0.0.1/tcp/8180'
        },
        Gateway: {
          NoFetch: true,
          ExposeRoutingAPI: true,
          HTTPHeaders: {
            'Access-Control-Allow-Origin': ['*'],
            'Access-Control-Allow-Methods': ['GET', 'POST', 'PUT', 'OPTIONS']
          }
        }
      }
    },
    args: ['--enable-pubsub-experiment', '--enable-namesys-pubsub']
  }

Description

Repro at https://github.com/SgtPooki/repro-kubo-not-announcing-self-as-provider/

Basically, if I enable ExposeRoutingAPI, and NoFetch, I should at least get the local kubo node to return itself as a router.

Another question here: is there any scenario where respecting NoFetch makes sense with routing/v1 api? i.e. not querying the network for providers, only returning providers we know about? If locally configured providers makes sense at all, do we need a second config option, or can we overload NoFetch?

cc @hacdias @lidel

lidel commented 2 weeks ago

Not returning self as provider on /routing/v1 indeed ended up looking like a bug. This is because /routing/v1 in Kubo as a thin wrapper around Kubo's ow n routing system, which does not return self in results. Including it makes sense.

@hacdias you may remember the details, how involved it would be to check local blockstore in parallel to kubo routing system, and inject self if we have the block?

ps. NoFetch does not apply to routing info and /routing/v1 results, it only applies to data (fetch of new blocks can't be triggered via Gateway).

aschmahmann commented 2 weeks ago

This is because /routing/v1 in Kubo as a thin wrapper around Kubo's ow n routing system, which does not return self in results. Including it makes sense.

🤷, I can see either way here. Sure, the node can do a check like blockstore.Has(cid) and append that result to the output of the delegated routing call but the result is likely more confusion. Are there any use cases for this outside of a testing environment?

I can see a few potential issues from doing this:

Confusion due to inconsistency
- If the local routing-v1 returns different results from ipfs routing that could confuse people
  - If the inconsistency with ipfs routing is resolved by having it also return itself then how will users debug if they've advertised their data or not?
- If the results of routing-v1 requests from different peers with the same config returns different values that could cause some confusion around if data is advertised or not
If/when we get something like this https://github.com/ipfs/specs/pull/388 the "I have stuff, but it's in no routing system" wouldn't be found unless you made up a new routing system name like ask-me or just asked for everything without enumerating the routers

That being said if others really feel this should be here I won't object (although if it causes more work around https://github.com/ipfs/specs/pull/388 I'll advocate for dropping this change).

achingbrain commented 1 week ago

Something else to consider here maybe is that if the goal is to have a PeerInfo for the current node appear in it's own routing output if that node has the block being searched for, and that PeerInfo is to contain a HTTP multiaddr, we should be able to override the host/port in that multiaddr.

I.e. browsers will expect a DNS+HTTPS address and will error if it ends up being IP+HTTP.

aschmahmann commented 1 week ago

we should be able to override the host/port in that multiaddr. ...browsers will expect a DNS+HTTPS address and will error if it ends up being IP+HTTP.

Why does the latter imply the former? You can advertise DNS-based multiaddrs and you shouldn't need anything special in the local routing-v1 implementation to do so since you'd want other node's routing-v1 implementations and local queries to also find your DNS address.

If there's some kubo issue with over-resolution of DNS addresses that seems like a separate bug to file and fix.

ipfs / kubo