scylladb / care-pet

Care Pet IoT ScyllaDB example
https://care-pet.docs.scylladb.com/
Apache License 2.0
47 stars 28 forks source link

Bug/build with rust #129

Closed timkoopmans closed 1 year ago

timkoopmans commented 1 year ago

Working through the rust example I noticed that the documentation refers to exposed ports on a container's private network. When executing code for rust in the context of the host, this network is not reachable by default. This in turn raises the following error:

Error: IO Error: No connections in the pool

I can see other people have encountered this issue e.g. https://github.com/scylladb/care-pet/issues/123

I think a better solution would be to publish the port of the first node to the host via docker compose i.e. https://github.com/scylladb/care-pet/compare/master...timkoopmans:care-pet:bug/build-with-rust?expand=1#diff-ceae43596ee25d0b21baa54737bcf1338964c780c6f6a12a420e4ea6ef4caba1R13-R14

Then you can typically access that port via localhost | 0.0.0.0 however to play it safe, I updated the doc to obtain the host IP itself (sometimes this binding is different depending on host/os etc).

Also updated a couple of typos etc in the rust doc.

If this approach is OK I can update the other tool examples (Go, PHP, etc)

tarzanek commented 1 year ago

if you expose just single node, client won't be able to get direct connection to others and its internal load balancer will be broken

ideal thing would be to run the client on same network as nodes (inside some dummy container in same internal net as scylla nodes?)

timkoopmans commented 1 year ago

I assumed the documentation is referring to a rust build/binary on the host, not within a separate build container.

This PR does not change container networking. When you run docker compose up, a network called rust_default is created. The nodes can still communicate with each other. Can confirm that with netcat e.g.

❯ docker exec -it carepet-scylla2 sh
# # connect to scylla1 from scylla2 for CQL
#  nc -zv carepet-scylla1 9042
Connection to carepet-scylla1 9042 port [tcp/*] succeeded!
# # connect to scylla1 from scylla2 for RPC
# nc -zv carepet-scylla1 7000
Connection to carepet-scylla1 7000 port [tcp/*] succeeded!

Can also confirm operation via nodetool status UN e.g.

❯ docker exec -it carepet-scylla1 nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address     Load       Tokens       Owns    Host ID                               Rack
UN  172.19.0.3  1.18 MB    256          ?       a192680c-4b00-46c6-a6bd-0122245fa9d4  rack1
UN  172.19.0.2  1.21 MB    256          ?       ccb6c676-78dc-40af-8637-67baff42e146  rack1
UN  172.19.0.4  212 KB     256          ?       17727d18-314e-4bea-ac75-13f362e8058a  rack1

This PR publishes the container port 9042 on host port 9042 for the first node so that a CQL connection can be made from a rust binary also on the host.

timkoopmans commented 1 year ago

Should also add I assumed we were only connecting to a single known node via https://github.com/timkoopmans/care-pet/blob/4b429ba6906f8ec54e8761097ff5e71e89c40cb5/rust/src/db/mod.rs#L40 in this example. That said if we wanted to provide multiple hosts we could publish 9042 on each of the nodes in docker compose.