// Copyright (c) 2020, 2024 IBM Corporation and others. // Licensed under Creative Commons Attribution-NoDerivatives // 4.0 International (CC BY-ND 4.0) // https://creativecommons.org/licenses/by-nd/4.0/ // // Contributors: // IBM Corporation // :projectid: microprofile-reactive-messaging :page-layout: guide-multipane :page-duration: 20 minutes :page-releasedate: 2020-04-22 :page-majorupdateddate: 2024-04-04 :page-guide-category: microprofile :page-essential: false :page-description: Learn how to use MicroProfile Reactive Messaging to implement an application with a reactive architecture. :guide-author: Open Liberty :page-tags: ['microprofile', 'jakarta-ee'] :page-related-guides: ['reactive-service-testing'] :page-permalink: /guides/{projectid} :imagesdir: /img/guide/{projectid} :page-seo-title: Creating asynchronous Java microservices using MicroProfile Reactive Messaging :page-seo-description: A getting started reactive programming tutorial with examples on how to send and receive messages between asynchronous Java microservices using Eclipse MicroProfile Reactive Messaging and Apache Kafka. :common-includes: https://raw.githubusercontent.com/OpenLiberty/guides-common/prod :source-highlighter: prettify = Creating reactive Java microservices
[.hidden] NOTE: This repository contains the guide documentation source. To view the guide in published form, view it on the https://openliberty.io/guides/{projectid}.html[Open Liberty website].
Learn how to write reactive Java microservices using MicroProfile Reactive Messaging.
== What you'll learn
You will learn how to build reactive microservices that can send requests to other microservices, and asynchronously receive and process the responses. You will use an external messaging system to handle the asynchronous messages that are sent and received between the microservices as streams of events. MicroProfile Reactive Messaging makes it easy to write and configure your application to send, receive, and process the events efficiently.
Asynchronous messaging between microservices
Asynchronous communication between microservices can be used to build reactive and responsive applications. By decoupling the requests sent by a microservice from the responses that it receives, the microservice is not blocked from performing other tasks while waiting for the requested data to become available. Imagine asynchronous communication as a restaurant. A waiter might come to your table and take your order. While you are waiting for your food to be prepared, that waiter serves other tables and takes their orders too. When your food is ready, the waiter brings your food to the table and then continues to serve the other tables. If the waiter were to operate synchronously, they must take your order and then wait until they deliver your food before serving any other tables. In microservices, a request call from a REST client to another microservice can be time-consuming because the network might be slow, or the other service might be overwhelmed with requests and can’t respond quickly. But in an asynchronous system, the microservice sends a request to another microservice and continues to send other calls and to receive and process other responses until it receives a response to the original request.
What is MicroProfile Reactive Messaging?
MicroProfile Reactive Messaging provides an easy way to asynchronously send, receive, and process messages that are received as continuous streams of events. You simply annotate application beans' methods and Open Liberty converts the annotated methods to reactive streams-compatible publishers, subscribers, and processors and connects them up to each other. MicroProfile Reactive Messaging provides a Connector API so that your methods can be connected to external messaging systems that produce and consume the streams of events, such as https://kafka.apache.org/[Apache Kafka^].
The application in this guide consists of two microservices, system and inventory. Every 15 seconds, the system microservice calculates and publishes an event that contains its current average system load. The inventory microservice subscribes to that information so that it can keep an updated list of all the systems and their current system loads. The current inventory of systems can be accessed via the /systems REST endpoint. You'll create the system and inventory microservices using MicroProfile Reactive Messaging.
image::reactive-messaging-system-inventory.png[Reactive system inventory,align="center"]
// ================================================================================================= // Prerequisites // ================================================================================================= == Additional prerequisites
You need to have Docker installed. For installation instructions, refer to the official https://docs.docker.com/get-docker/[Docker documentation^]. You will build and run the microservices in Docker containers. An installation of Apache Kafka is provided in another Docker container.
// ================================================================================================= // Getting started // ================================================================================================= [role='command'] include::{common-includes}/gitclone.adoc[]
== Creating the producer in the system microservice
Navigate to the start directory to begin.
// cloud hosted instructions
ifdef::cloud-hosted[]
cd /home/project/guide-microprofile-reactive-messaging/startendif::[]
The system microservice is the producer of the messages that are published to the Kafka messaging system as a stream of events. Every 15 seconds, the system microservice publishes an event that contains its calculation of the average system load (its CPU usage) for the last minute.
SystemService class.system/src/main/java/io/openliberty/guides/system/SystemService.javaThe SystemService class contains a Publisher method that is called [hotspot=sendSystemLoad file=0]sendSystemLoad(), which calculates and returns the average system load. The [hotspot=publishSystemLoad file=0]@Outgoing annotation on the [hotspot=sendSystemLoad file=0]sendSystemLoad() method indicates that the method publishes its calculation as a message on a topic in the Kafka messaging system. The [hotspot=flowableInterval file=0]Flowable.interval() method from rxJava is used to set the frequency of how often the system service publishes the calculation to the event stream.
The messages are transported between the service and the Kafka messaging system through a channel called [hotspot=systemLoad file=1]systemLoad. The name of the channel to use is set in the [hotspot=publishSystemLoad file=0]@Outgoing("systemLoad") annotation. Later in the guide, you will configure the service so that any messages sent by the system service through the [hotspot=systemLoad file=1]systemLoad channel are published on a topic called [hotspot=topic1 file=1]system.load, as shown in the following diagram:
image::reactive-messaging-system-inventory-publisher.png[Reactive system publisher,align="center"]
== Creating the consumer in the inventory microservice
The inventory microservice records in its inventory the average system load information that it received from potentially multiple instances of the system service.
InventoryResource class.inventory/src/main/java/io/openliberty/guides/inventory/InventoryResource.javaThe inventory microservice receives the message from the system microservice over the [hotspot=systemLoad file=0]@Incoming("systemLoad") channel. The properties of this channel are defined in the [hotspot=systemLoad file=1]microprofile-config.properties file. The inventory microservice is also a RESTful service that is served at the [hotspot=inventoryEndPoint file=0]/inventory endpoint.
The InventoryResource class contains a method called [hotspot=updateStatus file=0]updateStatus(), which receives the message that contains the average system load and updates its existing inventory of systems and their average system load. The [hotspot=systemLoad file=0]@Incoming("systemLoad") annotation on the [hotspot=updateStatus file=0]updateStatus() method indicates that the method retrieves the average system load information by connecting to the channel called [hotspot=systemLoad file=0]systemLoad. Later in the guide, you will configure the service so that any messages sent by the system service through the systemLoad channel are retrieved from a topic called [hotspot=topic1 file=1]system.load, as shown in the following diagram:
image::reactive-messaging-system-inventory-detail.png[Reactive system inventory detail,align="center"]
== Configuring the MicroProfile Reactive Messaging connectors for Kafka
The system and inventory services exchange messages with the external messaging system through a channel. The MicroProfile Reactive Messaging Connector API makes it easy to connect each service to the channel. You just need to add configuration keys in a properties file for each of the services. These configuration keys define properties such as the name of the channel and the topic in the Kafka messaging system. Open Liberty includes the liberty-kafka connector for sending and receiving messages from Apache Kafka.
The system and inventory microservices each have a MicroProfile Config properties file to define the properties of their outgoing and incoming streams.
system/src/main/resources/META-INF/microprofile-config.propertiesThe [hotspot=kafkaConfig file=0]mp.messaging.connector.liberty-kafka.bootstrap.servers property configures the hostname and port for connecting to the Kafka server. The system microservice uses an outgoing connector to send messages through the [hotspot=systemLoad file=0]systemLoad channel to the [hotspot=topic1 file=0]system.load topic in the Kafka message broker so that the inventory microservices can consume the messages. The [hotspot=serializer1 file=0]key.serializer and [hotspot=serializerVal1 file=0]value.serializer properties characterize how to serialize the messages. The SystemLoadSerializer class implements the logic for turning a SystemLoad object into JSON and is configured as the [hotspot=serializerVal1 file=0]value.serializer.
The inventory microservice uses a similar microprofile-config.properties configuration to define its required incoming stream.
inventory/src/main/resources/META-INF/microprofile-config.propertiesThe inventory microservice uses an incoming connector to receive messages through the [hotspot=systemLoad file=1]systemLoad channel. The messages were published by the system microservice to the [hotspot=topic1 file=1]system.load topic in the Kafka message broker. The [hotspot=deserializer1 file=1]key.deserializer and [hotspot=deserializerVal1 file=1]value.deserializer properties define how to deserialize the messages. The SystemLoadDeserializer class implements the logic for turning JSON into a SystemLoad object and is configured as the [hotspot=deserializerVal1 file=1]value.deserializer. The [hotspot=group1 file=1]group.id property defines a unique name for the consumer group. A consumer group is a collection of consumers who share a common identifier for the group. You can also view a consumer group as the various machines that ingest from the Kafka topics. All of these properties are required by the https://kafka.apache.org/documentation/#producerconfigs[Apache Kafka Producer Configs^] and https://kafka.apache.org/documentation/#consumerconfigs[Apache Kafka Consumer Configs^].
== Configuring Liberty
To run the services, the Open Liberty on which each service runs needs to be correctly configured. Relevant features, including the https://openliberty.io/docs/ref/feature/#mpReactiveMessaging-3.0.html[MicroProfile Reactive Messaging feature^], must be enabled for the system and inventory services.
system/src/main/liberty/config/server.xmlThe [hotspot file=1]server.xml file is already configured for the inventory microservice.
== Building and running the application
Build the system and inventory microservices using Maven and then run them in Docker containers.
system/pom.xml
The [hotspot file=0]pom.xml file lists the [hotspot=reactiveMessaging file=0]microprofile-reactive-messaging-api, [hotspot=kafka file=0]kafka-clients, and [hotspot=rxjava file=0]rxjava dependencies.
The [hotspot=reactiveMessaging file=0]microprofile-reactive-messaging-api dependency is needed to enable the use of MicroProfile Reactive Messaging API. The [hotspot=kafka file=0]kafka-clients dependency is added because the application needs a Kafka client to connect to the Kafka broker. The [hotspot=rxjava file=0]rxjava dependency is used for creating events at regular intervals.
Start your Docker environment. Dockerfiles are provided for you to use.
To build the application, run the Maven install and package goals from the command line in the start directory:
[role='command']
mvn -pl models install
mvn packageRun the following commands to containerize the microservices:
[role='command']
docker build -t system:1.0-SNAPSHOT system/.
docker build -t inventory:1.0-SNAPSHOT inventory/.Next, use the provided script to start the application in Docker containers. The script creates a network for the containers to communicate with each other. It also creates containers for Kafka and the microservices in the project. For simplicity, the script starts one instance of the system service.
include::{common-includes}/os-tabs.adoc[]
[role='command']
.\scripts\startContainers.bat--
[role='command']
./scripts/startContainers.sh--
== Testing the application
The application might take some time to become available. After the application is up and running, you can access it by making a GET request to the /systems endpoint of the inventory service.
Visit the http://localhost:9085/health[^] URL to confirm that the inventory microservice is up and running.
When both the liveness and readiness health checks are up, go to the http://localhost:9085/inventory/systems[^] URL to access the inventory microservice.
You see the CPU systemLoad property for all the systems:
You can revisit the http://localhost:9085/inventory/systems[^] URL after a while, and you will notice the CPU systemLoad property for the systems changed.
You can use the \http://localhost:9085/inventory/systems/{hostname} URL to see the CPU systemLoad property for one particular system.
In the following example, the 30bec2b63a96 value is the hostname. If you go to the \http://localhost:9085/inventory/systems/30bec2b63a96 URL, you can see the CPU systemLoad property only for the 30bec2b63a96 hostname:
== Tearing down the environment
Run the following script to stop the application:
include::{common-includes}/os-tabs.adoc[]
[role='command']
.\scripts\stopContainers.bat--
[role='command']
./scripts/stopContainers.sh--
== Great work! You're done!
You just developed a reactive Java application using MicroProfile Reactive Messaging, Open Liberty, and Kafka.
include::{common-includes}/attribution.adoc[subs="attributes"]