thegridman / coherence-k8s-example

0 stars 1 forks source link

= Coherence Spring Boot Kubernetes Example

This repository is an example of running a simple Spring Boot Coherence application in Kubernetes.

There are a number of parts to this example, each of the modules is a separate Maven projects. The reason for this is that in the real world each of those modules could be built by a separate development team. Each module may have a different development and deployment lifecycle.

== Spring Boot CacheStore Implementation

This example demonstrates how to write a Spring Boot CacheStore implementation. The SpringJpaCacheStore class is generic and takes any Spring CrudRepository as its constructor parameter. It then uses this to perform the required cache store operations.

There is a Spring JPA repository class for Student named StudentRepository. This is just an interface that extends the Spring CrudRepository. There is no need to write any more code than this interface.

[source,java]

@Repository("students") public interface StudentRepository extends CrudRepository<Student, StudentId> { }

NOTE: The StudentRepository class has been annotated with @Repository("students") to make it a Spring bean with the name students. This name is important as it will be used by the CacheStore factory class to look up the bean in the Spring context. This example used the convention that the Coherence cache name will match the corresponding repository bean name.

=== CacheStore Factory There is a factory class which will be used by Coherence to create CacheStore instances. The CacheStoreFactory class is generic, so no additional code will be needed if more entity classes get added to the project. The factory will look-up in the Spring context a CrudRepository bean with the same name as the cache which is being created.

The Coherence cache configuration file configures the cache scheme to use the CacheStoreFactory to create CacheStore instances, as shown below.

[source,xml]

db-scheme StorageService unlimited-scheme com.oracle.coherence.examples.storage.CacheStoreFactory createCacheStore java.lang.String {cache-name} com.tangosol.net.BackingMapManagerContext {manager-context} true

=== Adding More Domain Classes & CacheStores

To add more domain classes to the project all that needs to be implemented is the domain classes themselves, annotated with JPA and Coherence POF annotations and a corresponding repository class.

For example, suppose we wanted to add a Course entity to our Student database, we create the CourseId and Course classes:

[source,java]

@Embeddable @PortableType(id = 1003) public class CourseId implements Serializable {

@Portable
private String classId;

public CourseId() {
}

public CourseId(String classId) {
    this.classId = classId;
}

public String getClassId() {
    return classId;
}

public void setClassId(String classId) {
    this.classId = classId;
}

@Override
public boolean equals(Object o) {
    if (this == o) {
        return true;
    }
    if (o == null || getClass() != o.getClass()) {
        return false;
    }
    CourseId courseId = (CourseId) o;
    return Objects.equals(classId, courseId.classId);
}

@Override
public int hashCode() {
    return Objects.hash(classId);
}

}

[source,java]

@Entity @PortableType(id = POF_TYPE_COURSE) public class Course {

@EmbeddedId
@Portable
private CourseId id;

@Portable
private String name;

public Course() {
}

public Course(CourseId id, String name) {
    this.id = id;
    this.name = name;
}

public CourseId getId() {
    return id;
}

public void setId(CourseId id) {
    this.id = id;
}

public String getName() {
    return name;
}

public void setName(String name) {
    this.name = name;
}

}

For the CacheStore to work all we need is a repository class for the Course entity like this: [source,java]

@Repository("courses") public interface CourseRepository extends CrudRepository<Course, CourseId> { }

Now when a cache named courses is created the CacheStoreFactory will look up the repository bean named courses and use it to create the cache store.

== Size Limiting the Caches

The cache configuration does not contain any size limits.

The distributed scheme for the caches is below. [source,xml]

db-scheme StorageService storage-local-scheme com.oracle.coherence.examples.storage.CacheStoreFactory createCacheStore java.lang.String {cache-name} com.tangosol.net.BackingMapManagerContext {manager-context} true

In the scheme above in its <internal-cache-scheme> the <local-scheme> refers to a local-scheme named storage-local-scheme which is defined further down in the cache configuration file and looks like this:

[source,xml]

storage-local-scheme

The storage-local-scheme is a plain local-scheme with no other configuration, so it has no size limit or eviction policy.

To set a size limit we can set the high-units like this: [source,xml]

storage-local-scheme 1000

This means that the caches will be limited to 1000 entries each. When a cache reaches 1000 entries a number of entries will be evicted down to the low-units value. In the example above we have not set low-units, so the default is 80% of the high units. So in this case when the cache size reaches 1000 it will be reduced down to 800.

You could set the low-units to a different value like this: [source,xml]

storage-local-scheme 1000 500

Now when the cache reaches 1000 entries, 500 will be evicted.

=== Size Limiting Using Size

The high-units can be set as a bytes value by setting the unit-calculator value of the local-scheme to BINARY:

[source,xml]

storage-local-scheme BINARY 10MB 5MB

Now the high-units is set to 10 mega-bytes, and the low-units is 5 mega-bytes. When the amount of data in the cache reaches 10MB, regardless of how many entries, then entries will be evicted until the size gets down to 5MB or less.

There is lots of information on this in the Coherence documentation: https://docs.oracle.com/en/middleware/standalone/coherence/14.1.1.0/develop-applications/cache-configurations-example.html#GUID-3A02103F-0DBF-47B2-A940-9EDD928E45C5

== Build the Examples

The examples are Maven projects and should be build with the following commands in this order:

Domain Classes [source,bash]

./mvnw clean install -DskipTests -f domain-model/

Storage Service This is a Spring Boot application so needs the addition of spring-boot:build-image to get the Spring Boot plugin to build the image. [source,bash]

./mvnw clean install spring-boot:build-image -DskipTests -f storage-service/

REST Microservice This is a Spring Boot application so needs the addition of spring-boot:build-image to get the Spring Boot plugin to build the image.

./mvnw clean install spring-boot:build-image -DskipTests -f rest-service/

== Running the Example

This example requires an Oracle database to connect to. One way to do this for testing is use the Oracle DB Docker image on Oracle Container Registry, which is simple to start and throw away afterward.

The following database details will be required:

The user must have tables in schema like the ones below:

[Source,sql]

CREATE USER College IDENTIFIED BY Coherence2020;

GRANT CONNECT, RESOURCE, DBA TO College;

DROP TABLE COLLEGE.ADDRESS; DROP TABLE COLLEGE.STUDENT;

CREATE TABLE COLLEGE.ADDRESS ( ROLL VARCHAR2(255 char) NOT NULL PRIMARY KEY, LINE_ONE VARCHAR2(255 char), LINE_TWO VARCHAR2(255 char), CITY VARCHAR2(255 char), POSTAL_CODE VARCHAR2(255 char), COUNTRY VARCHAR2(255 char) );

CREATE TABLE COLLEGE.STUDENT ( ROLL VARCHAR2(255 char) NOT NULL PRIMARY KEY, FIRST_NAME VARCHAR2(255 char), LAST_NAME VARCHAR2(255 char), CLASS_NAME VARCHAR2(255 char), ADDRESS_ROLL VARCHAR2(255 char) CONSTRAINT STUDENT_ADDRESS REFERENCES COLLEGE.ADDRESS );

The different modules can be run in Docker or in Kubernetes.

=== Running in Docker

First start the Coherence storage application. Choose whether you want to run the plain JDBC storage application or the Spring Boot storage application, ONLY RUN ONE OF THEM.

==== Run the Coherence Storage Application

Run the command below after changing the environment variables to the values required for your database.

==== Run the Spring Boot Storage Application [Source,bash]

docker run -it --rm -p 20000:20000 \ -e ORACLE_DB_HOST=192.168.1.33 -e ORACLE_DB_PORT=31521 \ -e ORACLE_DB_SID=ORCLPDB1 -e ORACLE_DB_USER=College \ -e ORACLE_DB_PASSWORD=Coherence2020 --name storage coherence-example-spring-storage:1.0.0-SNAPSHOT

=== Run the REST Microservice

Now run the Spring Boot REST microservice. This is a Coherence Extend client application that will connect to the storage application. When we started the storage application we exposed the Coherence Extend port 20000 to port 20000 on the host. We can configure the REST service to connect to Extend on the local machines IP address.

Run the command below after changing the extend.host system property value to the IP address of the local machine.

[Source,bash]

docker run -it --rm -p 8080:8080 -e JAVA_TOOL_OPTIONS='-Dextend.host=192.168.1.33' rest-service:1.0.0-SNAPSHOT

=== Try the REST Service

We can use curl to execute REST commands. The REST service exposed the container port 8080 to port 8080 on the local machine so we can connect to that port.

==== First Get a Non-Existent Student

[Source,bash]

curl -i -w '\n' -X GET http://127.0.0.1:8080/student/foo

which should return a 404 error something like this [Source,bash]

HTTP/1.1 404 Content-Type: application/json Transfer-Encoding: chunked Date: Thu, 10 Sep 2020 14:00:30 GMT

{"timestamp":"2020-09-10T14:00:30.416+00:00","status":404,"error":"Not Found","message":"","path":"/student/foo"}

==== Add a New Student

We can do a POST command to add a Student. The API requires the request body to be a json representation of the Student.

For example: [Source,json]

{ "firstName":"Aamir", "lastName":"Khan", "className":"drama", "address": { "lineOne":"Freeda Apartments", "lineTwo":"Carter Road, Bandra West", "city":"Mumbai", "postalCode":"123456", "country":"India" } }

[Source,bash]

curl -i -w '\n' -X POST http://127.0.0.1:8080/student \ -H "Content-Type: application/json" \ -d '{"firstName":"Aamir","lastName":"Khan","className":"drama","address":{"lineOne":"Freeda Apartments","lineTwo":"Carter Road, Bandra West","city":"Mumbai","postalCode":"123456","country":"India"}}'

Which should return something like this: [Source,bash]

HTTP/1.1 201 Content-Type: application/json Transfer-Encoding: chunked Date: Thu, 10 Sep 2020 14:08:45 GMT

{"firstName":"Aamir","lastName":"Khan","className":"drama","address":{"lineOne":"Freeda Apartments","lineTwo":"Carter Road, Bandra West","city":"Mumbai","postalCode":"123456","country":"India","evolvableHolder":{"typeIds":[],"empty":true}},"rollNumber":"3161f377-e98c-4a19-8992-05329699088f","evolvableHolder":{"typeIds":[],"empty":true}}

The json returned will show the roll number that has been created as the Student identifier, in this case 3161f377-e98c-4a19-8992-05329699088f.

==== Get an Existing Student

Now we have added a Student we can execute a GET for that Student using the roll number.

[Source,bash]

curl -i -w '\n' -X GET http://127.0.0.1:8080/student/3161f377-e98c-4a19-8992-05329699088f

Which this time should return a 200 status and the json representation of the Student. [Source,bash]

HTTP/1.1 200 Content-Type: application/json Transfer-Encoding: chunked Date: Thu, 10 Sep 2020 14:10:51 GMT

{"firstName":"Aamir","lastName":"Khan","className":"drama","address":{"lineOne":"Freeda Apartments","lineTwo":"Carter Road, Bandra West","city":"Mumbai","postalCode":"123456","country":"India","evolvableHolder":{"typeIds":[],"empty":true}},"rollNumber":"3161f377-e98c-4a19-8992-05329699088f","evolvableHolder":{"typeIds":[],"empty":true}}

=== Running in Kubernetes

To run in Kubernetes you still require an Oracle Database as with the Docker example. Again, a simple solution is to run the Oracle DB image in k8s.

As before there are two choices for storage, the plain JDBC storage or the Spring Boot JPA storage, CHOOSE ONLY ONE.

First make sure the Coherence Operator has been installed, as this will be required to run the storage cluster.

NOTE: The operator must be at least version 3.1.0 to support Spring Boot Buildpacks images.

==== Start the Coherence Storage Cluster

The following yaml can be used to create the Spring Boot storage cluster using the image built from this project. [Source,yaml] .k8s/storage-cluster.yaml

apiVersion: coherence.oracle.com/v1 kind: Coherence metadata: name: student-storage spec: annotations: openshift.io/scc: anyuid image: storage-service:1.0.0-SNAPSHOT # <1> application: type: spring # <2> env:

<1> The application image built by the Spring Boot plugin has been specified as the image name <2> It is important tell the Coherence Operator that the application is a Spring Boot application by setting the `spec.application.type` field to `spring`. The yaml above is in the ./k8s/storage-cluster.yaml file. Create the storage cluster with the following command: [Source,bash] ---- kubectl create -f ./k8s/spring-storage-cluster.yaml ---- ==== Start the Student REST Microservice The Student Microservice is not a Coherence cluster member application, it is an Extend client, so it is not managed by the Coherence Operator. It can be deployed into Kubernetes using any suitable method that Kubernetes has for running Spring Boot images. This example will use a Kubernetes `Deployment`. The Coherence Operator will have created a K8s Service to expose the Extend proxy, this service will be called `student-storage-extend`. The DNS name created in Kubernetes for this will be `student-storage-extend..svc` where `` is the name of the namespace the storage cluster was created in. We can use this to set the `extend.host` System property when we run the REST service below. In this example we assume that the storage cluster is in a namespace called `sbi` so the Extend service name is `student-storage-extend.sbi.svc`. To start the Spring Boot REST Service use the following yaml: [Source,yaml] .k8s/rest-service.yaml ---- apiVersion: apps/v1 kind: Deployment metadata: name: students-application labels: app: students spec: selector: matchLabels: app: students strategy: type: Recreate template: metadata: labels: app: students spec: containers: - name: students image: rest-service:1.0.0-SNAPSHOT env: - name: JAVA_TOOL_OPTIONS value: "-Dextend.host=student-storage-extend.sbi.svc" ports: - name: rest containerPort: 8080 --- apiVersion: v1 kind: Service metadata: name: students spec: type: NodePort selector: app: students ports: - name: rest protocol: TCP port: 8080 targetPort: rest nodePort: 30080 ---- The yaml above will create a Deployment to run the application and a Service to expose the REST endpoint. This example uses a Service with a type of NodePort, which works well locally in Docker. If you want to expose the port externally change the Service type from `NodePort` to `LoadBalancer`. Create the REST service with the following command: [Source,bash] ---- kubectl create -f ./k8s/rest-service.yaml ---- When the service starts the REST endpoint will be reachable on port 30080 on the node or load balancer. The same curl commands can now be executed against this host and port.