search

ipfs / js-ipfs

IPFS implementation in JavaScript
https://js.ipfs.tech
Other
7.44k stars 1.25k forks source link

Datastore Config doesn't populate in worker #4096

Closed piboistudios closed 2 years ago

piboistudios commented 2 years ago

Severity:

Critical

Description:

I simply run an IPFS node from within a Haraka plugin (which runs as a node cluster worker) and get all sorts of errors.

The first error comes as:

[CRIT] [-] [core] TypeError: `[object Object]` is not an Option Object
[CRIT] [-] [core]     at Object.module.exports (/home/topboy/web3mail/haraka/node_modules/merge-options/index.js:163:10)

Where, for some reason the options object is anything but a "plain object" (as told by is-plain-obj), so then I just hack at merge-options and skip the check that throws this error...

I then get this error:

 [CRIT] [-] [core] TypeError: this._datastore.get is not a function
[CRIT] [-] [core]     at PersistentStore.load (/home/topboy/Haraka/node_modules/libp2p/src/peer-store/store.js:73:39)
[CRIT] [-] [core]     at PersistentProtoBook.add (/home/topboy/Haraka/node_modules/libp2p/src/peer-store/proto-book.js:128:40)
[CRIT] [-] [core]     at async Libp2p.handle (/home/topboy/Haraka/node_modules/libp2p/src/index.js:633:5)

I then tried to use ipfsd-ctl to spawn a Go IPFSD node and get this error:

[CRIT] [-] [core] Error: Initializing daemon...
[CRIT] [-] [core] go-ipfs version: 0.12.2
[CRIT] [-] [core] Repo version: 12
[CRIT] [-] [core] System version: amd64/linux
[CRIT] [-] [core] Golang version: go1.16.15
[CRIT] [-] [core] Swarm listening on /ip4/127.0.0.1/udp/4001/quic
[CRIT] [-] [core] Swarm listening on /ip4/192.168.1.248/udp/4001/quic
[CRIT] [-] [core] Swarm listening on /ip6/::1/tcp/4001
[CRIT] [-] [core] Swarm listening on /ip6/::1/udp/4001/quic
[CRIT] [-] [core] Swarm listening on /p2p-circuit
[CRIT] [-] [core] Swarm announcing /ip4/127.0.0.1/udp/4001/quic
[CRIT] [-] [core] Swarm announcing /ip4/192.168.1.248/udp/4001/quic
[CRIT] [-] [core] Swarm announcing /ip6/::1/tcp/4001
[CRIT] [-] [core] Swarm announcing /ip6/::1/udp/4001/quic
[CRIT] [-] [core]  
[CRIT] [-] [core] 
[CRIT] [-] [core]  2022/05/15 09:10:35 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.
[CRIT] [-] [core] Error: serveHTTPApi: manet.Listen(/ip4/127.0.0.1/tcp/5001) failed: listen tcp4 127.0.0.1:5001: bind: address already in use 
[CRIT] [-] [core] 
[CRIT] [-] [core]  Command failed with exit code 1: /home/topboy/web3mail/haraka/node_modules/go-ipfs/go-ipfs/ipfs daemon
[CRIT] [-] [core] 2022/05/15 09:10:35 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.
[CRIT] [-] [core] Error: serveHTTPApi: manet.Listen(/ip4/127.0.0.1/tcp/5001) failed: listen tcp4 127.0.0.1:5001: bind: address already in use
[CRIT] [-] [core] Initializing daemon...
[CRIT] [-] [core] go-ipfs version: 0.12.2
[CRIT] [-] [core] Repo version: 12
[CRIT] [-] [core] System version: amd64/linux
[CRIT] [-] [core] Golang version: go1.16.15
[CRIT] [-] [core] Swarm listening on /ip4/127.0.0.1/udp/4001/quic
[CRIT] [-] [core] Swarm listening on /ip4/172.17.0.1/udp/4001/quic
[CRIT] [-] [core] Swarm listening on /ip4/192.168.1.248/udp/4001/quic
[CRIT] [-] [core]  
[CRIT] [-] [core] 
[CRIT] [-] [core] 
[CRIT] [-] [core]     at makeError (/home/topboy/web3mail/haraka/node_modules/execa/lib/error.js:60:11)
[CRIT] [-] [core]     at handlePromise (/home/topboy/web3mail/haraka/node_modules/execa/index.js:118:26)
[CRIT] [-] [core]     at processTicksAndRejections (node:internal/process/task_queues:96:5)

(its also interesting to note that despite passing options to the Go IPFSD node explicitly turning off API, Gateway and RPC... it is still trying to open an API on 5001, which I already have open for another node..)

Steps to reproduce the error:

I am not 100% sure of the circumstances under which this occurs... I would guess the simplest way to reproduce the issue is to spin up a node subprocess starting an IPFS node, but I'm not sure if that's exactly what Haraka is doing that is causing things to be... different.. other than using cluster workers; as this setup works perfectly fine anywhere else.

My IPFS config looks like this:

{
    repo: REPO_NAME,
    config: {
      Addresses: {
        "Swarm": [
        ],
        API: '',
        RPC: '',
        Gateway: '',
        Delegates: []
      },

      Bootstrap: [
 //... bootstrap peers
      ],
      "Datastore": {
        "Spec": {
          "type": "mount",
          "mounts": [
            {
              "mountpoint": "/blocks",
              "type": "measure",
              "prefix": "flatfs.datastore",
              "child": {
                "type": "flatfs",
                "path": "blocks",
                "sync": true,
                "shardFunc": "/repo/flatfs/shard/v1/next-to-last/2"
              }
            },
            {
              "mountpoint": "/",
              "type": "measure",
              "prefix": "leveldb.datastore",
              "child": {
                "type": "levelds",
                "path": "datastore",
                "compression": "none"
              }
            }
          ]
        }
      },
      "Keychain": {
//      ...
      }
    }
  };

I ended up explicitly adding the Keychain and Datastore options to attempt to resolve; no avail

piboistudios commented 2 years ago

I'm going to attempt to start a separate ipfsd-ctl server process and create remote factories from that... I have never had problems using the http client from cluster workers, but I need the full functionality of a node...

EDIT: Creating a remote factory from an ipfsd server returns an ipfsd instance with a null api 😀

EDIT EDIT: I eventually got the ipfsd instance to spawn. I am only able to get OrbitDB to replicate in one direction.. If I open an existing orbit store in the subprocess via the OrbitDB instance listening on an ipfs-client (connected to the ipfsd instance I finally got running, the records replicate into the process, but any records I put into the DB inside the process do not replicate outside of it)

EDIT EDIT EDIT: Eventually, I get a pubsub socket hang-up when using the client:

2022-05-15T16:55:05.523Z [ERROR] pubsub: FetchError: request to http://127.0.0.1:37829/api/v0/pubsub/peers?arg=uL29yYml0ZGIvemRwdUFxVnhtU2d6djZWNkhYQ2VmaHo1alNKd3p2V3BOSDVhYUhKaUx1WFhWTDhFbi93ZWIzbWFpbC5oYXJha2EubWFpbHN0b3Jl failed, reason: socket hang up
piboistudios commented 2 years ago

Well... I was able to finally get an ipfsd-ctl remote node spawned in the Haraka subprocess.

This should probably be documented in ipfsd-ctl (although, perhaps it is obvious to anyone that understands how ipfsd-ctl works internally...), but you can't spawn a remote node without an HTTP API...

BigLep commented 2 years ago

@piboistudios : this is a good point. Glad you got unblocked. We'd welcome a PR in ipfsd-ctl to clarify this.