search

scylladb / scylla-operator

The Kubernetes Operator for ScyllaDB
https://operator.docs.scylladb.com/
Apache License 2.0
332 stars 162 forks source link

Unable to Connect Scylladb From Scala, Scylladb Nodes are Running in Kubernetes #1885

Closed zaryabRiasat closed 2 months ago

zaryabRiasat commented 5 months ago

I’m trying to run Scylladb Nodes in k8s and access them from Scala (Phantom Library).

Kubernetes

Firstly I’ve followed commands from official scylladb documentation to create scylladb cluster.

$ minikube start --cpus=6
$ eval $(minikube docker-env)
$ git clone https://github.com/scylladb/scylla-operator.git
$ cd scylla-operator
$ kubectl apply -f examples/common/cert-manager.yaml
$ kubectl wait --for condition=established crd/certificates.cert-manager.io crd/issuers.cert-manager.io
$ kubectl -n cert-manager rollout status deployment.apps/cert-manager-webhook
$ kubectl apply -f deploy/operator.yaml
$ kubectl wait --for condition=established crd/scyllaclusters.scylla.scylladb.com
$ kubectl -n scylla-operator rollout status deployment.apps/scylla-operator
$ kubectl create -f examples/generic/cluster.yaml

After all above commands scylladb cluster got ready, I’ve checked using some commands:

$ kubectl -n scylla get pods
NAME                                    READY   STATUS    RESTARTS   AGE
simple-cluster-us-east-1-us-east-1a-0   2/2     Running   0          90m
simple-cluster-us-east-1-us-east-1a-1   2/2     Running   0          80m
simple-cluster-us-east-1-us-east-1a-2   0/2     Pending   0          77m

$ kubectl -n scylla get svc
NAME                                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                                                                          AGE
simple-cluster-client                   ClusterIP   10.98.139.252   <none>        7000/TCP,7001/TCP,9042/TCP,9142/TCP,19042/TCP,19142/TCP,7199/TCP,10001/TCP,9180/TCP,5090/TCP,9100/TCP,9160/TCP   90m
simple-cluster-us-east-1-us-east-1a-0   ClusterIP   10.98.37.120    <none>        7000/TCP,7001/TCP,9042/TCP,9142/TCP,19042/TCP,19142/TCP,7199/TCP,10001/TCP,9180/TCP,5090/TCP,9100/TCP,9160/TCP   90m
simple-cluster-us-east-1-us-east-1a-1   ClusterIP   10.105.171.34   <none>        7000/TCP,7001/TCP,9042/TCP,9142/TCP,19042/TCP,19142/TCP,7199/TCP,10001/TCP,9180/TCP,5090/TCP,9100/TCP,9160/TCP   90m
simple-cluster-us-east-1-us-east-1a-2   ClusterIP   10.110.32.166   <none>        7000/TCP,7001/TCP,9042/TCP,9142/TCP,19042/TCP,19142/TCP,7199/TCP,10001/TCP,9180/TCP,5090/TCP,9100/TCP,9160/TCP   90m

$ kubectl exec -n scylla -it simple-cluster-us-east-1-us-east-1a-0 -- nodetool status
Defaulted container "scylla" out of: scylla, scylla-manager-agent, sidecar-injection (init)
Datacenter: us-east-1
=====================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address        Load       Tokens       Owns    Host ID                               Rack
UN  10.98.37.120   108.99 KB  256          ?       a1d76285-964f-48b1-b87d-2e2fafa96877  us-east-1a
UN  10.105.171.34  201.48 KB  256          ?       c2204fec-b713-4fd5-ba47-d8b72505ac12  us-east-1a

Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless

After setting above scylladb cluster, now I need to access it via my scala code.

Scala

schema.scala:

abstract class MyDataSchema extends Table[MyDataSchema, MyData] {

  override def tableName: String = "mydata"

  object myColumn extends OptionalStringColumn with PartitionKey {
    override def name: String = "myColumn"
  }
}

connection.scala:


class ScyllaDBConnection extends ScyllaDBConnectionTrait {

  override def connectionProvider: CassandraConnection = {
    val applicationConf: Config = ConfigFactory.load("application.conf")
    val keyspace: String = applicationConf.getString("keyspace.keyspaceName")
    val connection: CassandraConnection =
      ContactPoints(List("10.98.139.252")) //10.98.139.252 is IP of Service
        .keySpace(keyspace)
    connection
  }

}

database.scala:


class MyDB(override val connector: CassandraConnection)
  extends Database[MyDB](connector) {
  object MyData extends MyDataSchema with Connector
}

Now the problem is whenever I run this, it gives an error:

All host(s) tried for query failed (tried: /10.98.139.252 (com.datastax.driver.core.exceptions.TransportException: [/10.98.139.252] Cannot connect))

I’ve tried different things in:

connection.scala:

class ScyllaDBConnection extends ScyllaDBConnectionTrait {

  override def connectionProvider: CassandraConnection = {
    val applicationConf: Config = ConfigFactory.load("application.conf")
    val keyspace: String = applicationConf.getString("keyspace.keyspaceName")
    val connection: CassandraConnection =
      ContactPoints(List("10.98.139.252")) //10.98.139.252 is IP of Service
        .keySpace(keyspace)
    connection
  }

}

I’ve tried to give pod IP, service IP, scylladb node IP and service name to connect ContactPoints but nothing worked for me.

Guide me how can I connect my above scala service with Scylladb Cluster which is running on Kubernetes.

tnozicka commented 5 months ago

Please attach the must-gather archive.

https://github.com/scylladb/scylla-operator/blob/f3561383a75b394bc7688363767db1a38c5bf308/.github/ISSUE_TEMPLATE/bug-report.yaml?plain=1#L57

https://operator.docs.scylladb.com/stable/support/must-gather.html

/priority awaiting-more-evidence

scylla-operator-bot[bot] commented 2 months ago

The Scylla Operator project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

/lifecycle stale