cap-js-community / event-queue

An event queue that enables secure multi-tenant enabled transactional processing of asynchronous events, featuring instant event processing with Redis Pub/Sub and load distribution across all application instances.
https://cap-js-community.github.io/event-queue/
Apache License 2.0
11 stars 1 forks source link

Some questions regarding how to use this lib #201

Open jiangxin0503 opened 1 week ago

jiangxin0503 commented 1 week ago

Hi Colleagues

  1. I want to initialize the event-queue in coding level, like in sever.js
    eventQueue.initialize({
    configFilePath: "./srv/eventConfig.yml",
    });

    But there is error when init image

Then the coding level initialization is not successed, because it will anyway return saying that init is done,

image

Question: Does this mean there is only ONE chance to init the event-queue? but we have case like: we use cds-shim, in cds-shim, there is event initialized, but we also want to initialized by our own event but in case the config file from cds.eventQueue. configFilePath is initialized, the cds-shim's event will not register

Does this support multiple event init or register

  1. How to implement the case for publishing and processing of events in different microservices Like publishing event from service A, and consumption in service B it needs to init the event for both sevice A and B, but A will set isEventQueueActive false, and B for true, correct ?

Any other points needs to be taken care ?

Many thanks for your help.

Regards, Xin

soccermax commented 1 week ago

Hi Xin,

We discussed the issue with the shim and decided to offer an option to skip the initialization of the event queue in the shim. This ensures that the event queue is not initialized twice. If you skip the initialization in the shim, you can initialize the module in your project. However, ensure that you incorporate the events defined in the shim into your own event queue configuration.

Regarding your second question: The better approach is to use registerAsEventProcessor: false in Service A. The isEventQueueActive option is more suitable for temporarily disabling the event queue, for example, during a deployment. Do you already use Redis? If events are published in separate applications, it is recommended to use the same Redis instance for both applications. This significantly speeds up the processing time for newly published events. Without Redis, app A won't be immediately notified about new events, and they will be processed during the next periodic poll (depending on your configuration).

I hope this answers your questions. If not, feel free to reach out.

Best regards, Max

jiangxin0503 commented 1 week ago

Hi @soccermax ,

Thank you for the information; it's really helpful. Please find below my additional questions:

  1. If the shim provides the option, that would be great.
  2. I am facing an issue where I want to initialize the event queue at the code level in server.js. However, there is an error caused by the default initialization at https://github.com/cap-js-community/event-queue/blob/main/cds-plugin.js#L9. image

Bcause when i install the lib, the cds env for event-queue is

image

Even i do not config event-queue in my package.json

  1. The option registerAsEventProcessor looks good, especially since we are already using Redis. However, is the option registerAsEventProcessor intended for all events within one app? If registerAsEventProcessor: false is configured in App A, does this mean that all events will only be published in App A and cannot be consumed in App A? We have a scenario where:
    • In App A, one event needs to be published but consumed in App B.
    • Another event in App A needs to be both published and consumed within App A. Can this scenario be achieved?

Thank you. Xin

soccermax commented 1 week ago

Hi Xin,

there is a new shim version (cds-shim: 0.5.48) with a new env option (process.env.EVENT_QUEUE_CUSTOM_INIT). You can use this option to init the event-queue your self, but as said copy over the shim events into your own config file.

Regarding the empty event-queue config it seems that cds changed there behaviour regarding this or this happens in combination with shim. Let me test this. But if you init the event-queue yourself you can simply do it via plugin. But nevertheless this should also work via coding, I'll check this.

Regarding App A and B, this case is not implemented yet. But this is not a big thing, if I find some time next week I'll start/do it. I would imagine something like this. What do you think?

events:
  - type: Email
    subType: Task
    impl: ./srv/util-service/mail-service/business/EventQueueNotificationProcessor
    load: 1
    appNames: 
      - app-a

  - type: Fiori
    subType: Task
    impl: ./srv/util-service/mail-service/business/EventQueueNotificationProcessor
    load: 1
    appNames: 
      - app-b
jiangxin0503 commented 5 days ago

Hi Max, Thanks for your quick action.

For the case App A and App B, the config looks good. So does this mean once i publish the message, it should specify the consumer/processor ? like

await publishEvent(tx, {
  type: "Email",
  subType: "Task",
  appNames: [app-a],  // then only app a can process the event ?
  payload: JSON.stringify({
    recipients: ["alice@wonder.land"],
  }),
});

Thanks. Regards, Xin

soccermax commented 4 days ago

No the idea is that you specify this in the configuration like in the example above. So this would be in the config yml:

events:
  - type: Email
    subType: Task
    impl: ./srv/util-service/mail-service/business/EventQueueNotificationProcessor
    load: 1
    appNames: 
      - app-a

  - type: Fiori
    subType: Task
    impl: ./srv/util-service/mail-service/business/EventQueueNotificationProcessor
    load: 1
    appNames: 
      - app-b

Would this work for you?

jiangxin0503 commented 4 days ago

Hi Max, OK, I think I got your point now. Please kindly correct me if i am wrong:
like the config, the Email task configs as app-a, it means the process of Email Task will only be processed in app-a.

Then no matter the Email Task event is pulished in app-a , app-b, or even app-c, the process of this task will only happen in app-a. Then i think it works. Many Thanks. Regards, Xin

soccermax commented 4 days ago

yes exactly. I'll implement that in the near future, Thanks for the input