bacalhau-project / bacalhau

Compute over Data framework for public, transparent, and optionally verifiable computation
https://docs.bacalhau.org
Apache License 2.0
671 stars 87 forks source link

Hard to understand how to set external IP for node #3557

Open aronchick opened 6 months ago

aronchick commented 6 months ago

Here is my current IP output:

And when I start, it provisions the following IPs

2024/03/02 21:02:49 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Receive-Buffer-Size for details.
SDK 2024/03/02 21:02:50 WARN falling back to IMDSv1: operation error ec2imds: getToken, http response error StatusCode: 405, request to EC2 IMDS failed
SDK 2024/03/02 21:02:50 WARN falling back to IMDSv1: operation error ec2imds: getToken, http response error StatusCode: 405, request to EC2 IMDS failed

To connect another node to this one, run the following command in your shell:
bacalhau serve --node-type=compute --network=libp2p --peer=/ip4/10.128.0.2/tcp/1235/p2p/QmZ5qaJ8Zo4GtcCJrpFuS9gA7VkoWN4WDtroKEZ5jtTapZ --private-internal-ipfs --ipfs-swarm-addrs=/ip4/127.0.0.1/tcp/41595/p2p/QmdhxicDpPqb9L5yPLF1qWxJMiVfMBMPXSpxTDfje2opQV

To connect to this node from the client, run the following commands in your shell:
export BACALHAU_NODE_CLIENTAPI_HOST=0.0.0.0
export BACALHAU_NODE_CLIENTAPI_PORT=1234
export BACALHAU_NODE_NETWORK_TYPE=libp2p
export BACALHAU_NODE_LIBP2P_PEERCONNECT=/ip4/10.128.0.2/tcp/1235/p2p/QmZ5qaJ8Zo4GtcCJrpFuS9gA7VkoWN4WDtroKEZ5jtTapZ
export BACALHAU_NODE_IPFS_SWARMADDRESSES=/ip4/127.0.0.1/tcp/41595/p2p/QmdhxicDpPqb9L5yPLF1qWxJMiVfMBMPXSpxTDfje2opQV

A copy of these variables have been written to: /root/.bacalhau/bacalhau.run
2024/03/02 21:03:10 Unsolicited response received on idle HTTP channel starting with "\x13/multistream/1.0.0\n"; err=<nil>

This VM has an external IP, but it's not provisioned to the nic0 itself.

This is gettable with this:

curl ifconfig.me

Two features here:

wdbaruni commented 5 months ago

We can't accurately figure out the public ip of a node that a node is actually listening to and can accept requests on. We can do best effort that is correct most of the time, but not always.

I think this is a case where doing less or nothing is better than trying to do something that ends up confusing many users. In this case, we should stop generating the connect string and let the user figure out the ip address of the requester node that compute nodes can access. Whether that is a public or a private ip.

wdbaruni commented 2 months ago

@frrist is this fix easy to include in our config refactoring story?

frrist commented 2 months ago

In this case, we should stop generating the connect string and let the user figure out the ip address of the requester node that compute nodes can access. Whether that is a public or a private ip.

If I am reading this right, the fix is to stop showing:

To connect another node to this one, run the following command in your shell:
bacalhau serve --node-type=compute --network=libp2p --peer=/ip4/10.128.0.2/tcp/1235/p2p/QmZ5qaJ8Zo4GtcCJrpFuS9gA7VkoWN4WDtroKEZ5jtTapZ --private-internal-ipfs --ipfs-swarm-addrs=/ip4/127.0.0.1/tcp/41595/p2p/QmdhxicDpPqb9L5yPLF1qWxJMiVfMBMPXSpxTDfje2opQV

To connect to this node from the client, run the following commands in your shell:
export BACALHAU_NODE_CLIENTAPI_HOST=0.0.0.0
export BACALHAU_NODE_CLIENTAPI_PORT=1234
export BACALHAU_NODE_NETWORK_TYPE=libp2p
export BACALHAU_NODE_LIBP2P_PEERCONNECT=/ip4/10.128.0.2/tcp/1235/p2p/QmZ5qaJ8Zo4GtcCJrpFuS9gA7VkoWN4WDtroKEZ5jtTapZ
export BACALHAU_NODE_IPFS_SWARMADDRESSES=/ip4/127.0.0.1/tcp/41595/p2p/QmdhxicDpPqb9L5yPLF1qWxJMiVfMBMPXSpxTDfje2opQV

A copy of these variables have been written to: /root/.bacalhau/bacalhau.run

When users start a node because we don't know if their intention is connect to it from the same machine or a different machine. And if they are on a different machine, the more common case, these instructions will be incorrect.

I don't think we need to change anything in the configuration here. We should still serve on 0.0.0.0 by default, and users are free to change this value to their liking via the --host flag on serve