Add examples for events emitted by mainflux services when using nats JetStream and rabbitmq streams as the event source. This will help in documenting the events and also help in understanding how the events are emitted and how to view them.
Event Sourcing
Mainflux uses Event Sourcing. This means that some of the data is stored in the form of events. This allows mainflux to easily integrate with other systems and to have a clear audit trail of all the changes that happened in the system. With event sourcing, data changes are saved in domain events. These events are held in the chronological order of occurrence. These events can be replayed to get to the current state of the data. For example, when a user registers, an event is emitted to the event store. This event can be replayed to get the current state of the user. This is useful when you want to integrate with other systems. You can just replay the events to get the current state of the data.
Every function call in the things and users service emits an event. Some function calls in the mqtt service emit events but not all. Bootstrap service emits and consumes events from the things service. For example, assume you have connected a thing to a channel and have created a bootstrap configuration for that thing. When you delete the thing, the bootstrap configuration for that thing is also deleted. This is done by listening to the thing.deleted event in the bootstrap service. When this event is received, the bootstrap configuration for that thing is deleted.
These events are then stored in an event source which can either be redis streams, nats JetStream or rabbitmq messages. By default, the events are stored in Nats JetStream. With build tags, you can change the event source to either Redis streams or rabbitmq messages. The build tags are redis and rabbitmq respectively.
Nats JetStream
Using this PR as the reference, we can see how to use Nats JetStream as the event source.
This will create 1 user, 1 thing, 1 channel and 1 group. It will do CRUD operation on the entities and will also connect the thing to the channel. This will generate a lot of events together with mqtt events as it will publish messages to the channel.
To simulate bootstrap events, you can use the UI. Do CRUD operations on the bootstrap configurations and you will see the events in the nats logs. During the update provide actual certificates to simulate the bootstrap.updated event.
Once the events are generated, you can use the nats-cli to view the events.
nats stream view
Record the events to the documentation following this approach under the events page you can see the events that were generated. Do the same for Nats.
Redis Streams
This has already been implemented in master branch. You can use the same approach to document the events and make sure they are aligned. Use MF_ES_STORE_TYPE=reids when building mainflux and running it. You can use the redis-cli to view the events or redis insight.
RabbitMQ Messages
This has not been implemented yet. You can use the same approach to document the events. Use MF_ES_STORE_TYPE=rabbitmq when building mainflux and running it. You can use the rabbitmq-cli to view the events or rabbitmq management.
Requirements
[ ] Add examples for event sourcing using nats JetStream.
[ ] Add examples for event sourcing using rabbitmq messages.
[ ] Add documentation for how to view the events when using different event sources.
Description
Add examples for events emitted by mainflux services when using
nats JetStream
andrabbitmq streams
as the event source. This will help in documenting the events and also help in understanding how the events are emitted and how to view them.Event Sourcing
Mainflux uses Event Sourcing. This means that some of the data is stored in the form of events. This allows mainflux to easily integrate with other systems and to have a clear audit trail of all the changes that happened in the system. With event sourcing, data changes are saved in domain events. These events are held in the chronological order of occurrence. These events can be replayed to get to the current state of the data. For example, when a user registers, an event is emitted to the event store. This event can be replayed to get the current state of the user. This is useful when you want to integrate with other systems. You can just replay the events to get the current state of the data.
Every function call in the
things
andusers
service emits an event. Some function calls in themqtt
service emit events but not all.Bootstrap
service emits and consumes events from thethings
service. For example, assume you have connected a thing to a channel and have created a bootstrap configuration for that thing. When you delete the thing, the bootstrap configuration for that thing is also deleted. This is done by listening to thething.deleted
event in thebootstrap
service. When this event is received, the bootstrap configuration for that thing is deleted.These events are then stored in an event source which can either be
redis streams
,nats JetStream
orrabbitmq messages
. By default, the events are stored in Nats JetStream. With build tags, you can change the event source to eitherRedis streams
orrabbitmq messages
. The build tags areredis
andrabbitmq
respectively.Nats JetStream
Using this PR as the reference, we can see how to use Nats JetStream as the event source.
Build mainflux
Run mainflux
Simulate events using the e2e tool
This will create 1 user, 1 thing, 1 channel and 1 group. It will do CRUD operation on the entities and will also connect the thing to the channel. This will generate a lot of events together with
mqtt
events as it will publish messages to the channel.To simulate bootstrap events, you can use the UI. Do CRUD operations on the bootstrap configurations and you will see the events in the
nats
logs. During the update provide actual certificates to simulate thebootstrap.updated
event.Once the events are generated, you can use the nats-cli to view the events.
Record the events to the documentation following this approach under the events page you can see the events that were generated. Do the same for Nats.
Redis Streams
This has already been implemented in master branch. You can use the same approach to document the events and make sure they are aligned. Use
MF_ES_STORE_TYPE=reids
when building mainflux and running it. You can use the redis-cli to view the events or redis insight.RabbitMQ Messages
This has not been implemented yet. You can use the same approach to document the events. Use
MF_ES_STORE_TYPE=rabbitmq
when building mainflux and running it. You can use the rabbitmq-cli to view the events or rabbitmq management.Requirements
nats JetStream
.rabbitmq messages
.Resources