New Backend

[kstan79]

I know that current backend for prototyping and not design for production use.

Since bpmn-server is pure backend engine I recommend to create new backend using nestjs. it offer benefit which included:

1. More robust design
2. bigger ecosystem and we able to use lot of nestjs plug-in like sso authentication, swagger-ui, middleware
3. We have better way to manage complex workflow requirement
4. Use openapi we no need maintain client library. It mean we can accept frontend request from Java,python, .net without headache

Others bpmn project like kogito, they did similar and are they use quarkus which can allow developer use plenty of quarkus plugin.

I use it sometimes but I can't stay with them cause it is too rigit

If you agree we may work on it

[bpmn-ts]

Hi Sounds great Please go ahead by providing more description and design information Thanks

Sent from my iPhone

On Sep 21, 2023, at 6:44 AM, Ks Tan @.***> wrote:



I know that current backend for prototyping and not design for production use.

Since bpmn-server is pure backend engine I recommend to create new backend using nestjs. it offer benefit which included:

1. More robust design 2 bigger ecosystem and able to use lot of nestjs plug-in like sso authentication, swagger-ui, middleware
2. We have better way to manage complex workflow requirement

So for bpmn project like kogito they are using quarkus which can allow developer use plenty of quarkus plugin

If you agree we may work on it

— Reply to this email directly, view it on GitHubhttps://github.com/ralphhanna/bpmn-server/issues/118, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQL5NQZ3I7MOPO6TBU7TCIDX3QK5ZANCNFSM6AAAAAA5BKO7NE. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[kstan79]

We may start from easier way like:

1. Create an kickstart git repo like "bpmn-nest-template"
2. Inside content latest nestjs project and appropriate route/service controller
3. Put some bpmn sample, delegates inside
4. Write read me for how to kick start:
   • add more process definition
   • change configuration via .env
   • try api via swagger
   • turn on api key or sso jwt token (like keycloak)
   • how to build for production
   • and etc

One comfortable we may create cli executor and user install it via "npm install -g bpmn-server-cli"

Then setup and maintain the project using

• bpmn-server-cli init my-proj
• bpm-server-cli start/stop
• bpmn-server-cli add-deligate uploads3 emailuser push-notification
• etc

[ralphhanna]

Great Please go ahead with a separate branch

I will be done with a full release in couple of days Thanks

Sent from my iPhone

On Sep 21, 2023, at 6:15 PM, Ks Tan @.***> wrote:



We may start from easier way like:

1. Create an kickstart git repo like "bpmn-nest-template"
2. Inside content latest nestjs project and appropriate route/service controller
3. Put some bpmn sample, delegates inside

4. Write read me for how to kick start:

   • add more process definition
   • change configuration via .env
   • try api via swagger
   • turn on api key or sso jwt token (like keycloak)
   • how to build for production
   • and etc

One comfortable we may create cli executor and user install it via "npm install -g bpmn-server-cli"

Then setup and maintain the project using

• bpmn-server-cli init my-proj
• bpm-server-cli start/stop
• bpmn-server-cli add-deligate uploads3 emailuser push-notification
• etc

— Reply to this email directly, view it on GitHubhttps://github.com/ralphhanna/bpmn-server/issues/118#issuecomment-1730370456, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AC2XVKDC56NNLKV5TZ4CCD3X3S37HANCNFSM6AAAAAA5BKO7NE. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[ingpconci]

I tried to create a loopback extension to use with the loopback backend framework https://github.com/ingpconci/loopback4-bpmn-server I know the loopback project is not updated like Nestjs, but the project is hosted on the OpenJS foundation https://openjsf.org/blog/2022/04/25/node-js-framework-loopback-graduates-to-at-large-project-at-openjs-foundation/ What do you think about it?

[kstan79]

Sounds good, I will spend sometime check loop back is it good for me,

[ralphhanna]

Can we think of Bpmn-server as a microservice that can be used in either framework?

[kstan79]

IMHO bpmn act as purely backend microservices, it doesn't effect much as long as it comply openapi api standard. All languages and framework can tan to openapi easily.

I not recommend bpmn server embed as library to support 2 or more framework cause it add burden to development for testing and documentation. Let's pick one and stick with it until perfect, than decide again.

In typescript I only have experience in nuxt and nestjs, never try loopback framework so I can't give comment about loopback good or bad yet.

Nest has build plenty of features I think suitable for industry grade application, bpmn-server adopt by industry most of the time so I feel we park inside good for us

[ralphhanna]

Hi I have looked at https://github.com/ingpconci/loopback4-bpmn-server and it seems to be a full port to loopback4 which will force everyone to use loopback4.

I am hoping that only the bpmn-client will be ported to lb4 with is minimal code while keeping bpmn-server as pure nodeJS

Thanks, Let us keep the discussion going

[ingpconci]

Hi I use the loopback as a client for bpmn-server, because I want to use the loopback API framework to control authentication, filtering and connecting to the relational database PostgreSQL. I imported the library, created the server instance and used it in the controllers. I don't modify the code of bpmn-server and I change only the 4 customization files for the library

The main effort is focused on defining a flexible system user task assignment; to facilitate the acceptance of BPMN workflow, I use this approach based on association between Lane in diagram and Use Role

In the extension, I put the controller API endpoints to support the Role management. In the picture there is the Html interface.

I would like to put in the loopback bpmn-server extension an API endpoint that returns the information about the User Task Form. At the moment, I define the BPMN process with camunda modeler and use the camunda form definition.

`

{ "components": [ { "text": "<h2>(1.1) New nonconformity</h2>", "label": "Text view", ….. } ` In my Html App, I have developed a form engine that uses this information in the model definition to build the Form, but in the future I would like to facilitate the construction of the form at server side. The function must provide a form representation that permit to build a non Html form (e.g. a standalone C++ app form engine) 

[bpmn-ts]

Thanks very much for the explanation, that looks great, the only challenge I found is how to update bpmn-server?

Is there a way to use npm instead of copying the code into the library folder?

Please let me know if you need any changes from my end.

Thanks

On Fri, Sep 29, 2023 at 4:17 AM ingpconci @.***> wrote:

Hi I use the loopback as a client for bpmn-server, because I want to use the loopback API framework to control authentication, filtering and connecting to the relational database PostgreSQL. I imported the library, created the server instance and used it in the controllers. I don't modify the code of bpmn-server and I change only the 4 customization files for the library

[image: image] https://user-images.githubusercontent.com/8488830/271501524-d3391dbc-077c-40e6-929c-bc952b5efcae.png

The main effort is focused on defining a flexible system user task assignment; to facilitate the acceptance of BPMN workflow, I use this approach based on association between Lane in diagram and Use Role

[image: image] https://user-images.githubusercontent.com/8488830/271503879-75e83f02-dc30-4399-b692-c44f1d430cf2.png

In the extension, I put the controller API endpoints to support the Role management. In the picture there is the Html interface. [image: image] https://user-images.githubusercontent.com/8488830/271505081-3d894bfe-bd67-4569-8e53-876209e76ed4.png

I would like to put in the loopback bpmn-server extension an API endpoint that returns the information about the User Task Form. At the moment, I define the BPMN process with camunda modeler and use the camunda form definition.

[image: image] https://user-images.githubusercontent.com/8488830/271508462-7bd58c18-5a20-497c-9c04-949dc0d08d7f.png

{ "components": [ { "text": "<h2>(1.1) New nonconformity</h2>", "label": "Text view", ….. } In my Html App, I have developed a form engine that uses this information in the model definition to build the Form, but in the future I would like to facilitate the construction of the form at server side. The function must provide a form representation that permit to build a non Html form (e.g. a standalone C++ app form engine) [image: image] — Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you commented.Message ID: ***@***.***>

[kstan79]

After view back your project seems u did extension for loopback only. Previously misunderstand and thought you host whole loopback application in repo.

@bpmn-ts you may need to decide is loopback a good framework and qualify as firstclass citizen for bpmn or not. if yes we may update documentation according that approach. Else, i will spend times to do in nestjs.

@ingpconci can you make your effort become npm package so it more portable? I'm excited actually because your may solve a big issue which can make bpmn-server use in production

[ingpconci]

I clone the project locally

git clone https://github.com/ralphhanna/bpmn-server.git

Then compile and pack

> npm run clean
> npm run build
> npm pack
...
npm notice === Tarball Details ===
npm notice name:          bpmn-server
npm notice version:       1.3.28
npm notice filename:      bpmn-server-1.3.28.tgz
npm notice package size:  10.2 MB
npm notice unpacked size: 49.3 MB
npm notice shasum:        7c52e0ac88d9b340516c8b484db5d7700af73c26
npm notice integrity:     sha512-DqC9SRCt3HlzT[...]mrIoswkvWnNjw==
npm notice total files:   851
npm notice
bpmn-server-1.3.28.tgz

After that I insert in the package.json

 "dependencies": {
    "bpmn-moddle": "^8.0.1",
    "bpmn-server": "file:bpmn-server-1.3.28.tgz",
    "camunda-bpmn-moddle": "^7.0.1",
    "tslib": "^2.0.0"
  },

[ingpconci]

To use in the main loopback application, I import these libraries into the application.ts

    "loopback-connector-postgresql": "^6.0.8",
    "loopback4-tenant-table-filter": "file:loopback4-tenant-table-filter-1.0.8.tgz",
    "bpmn-server": "file:bpmn-server-1.3.28.tgz",
    "loopback4-bpmn-server": "file:loopback4-bpmn-server-1.0.0.tgz"

and insert the initialization code

    //--------------------------------------------------------------------------
    // BpmnServerComponent
    //--------------------------------------------------------------------------
    const definitionsPath = path.join(__dirname, '../processes/')
    console.log('AppServerApplication:startup: bpmnProcessDefinitionDirectory=', definitionsPath);
    const componentOptions: Loopback4BpmnServerComponentOptions = {
      bpmnProcessDefinitionDirectory: definitionsPath
    }
    this.configure(Loopback4BpmnServerComponentBindings.COMPONENT).to(componentOptions);
    this.component(Loopback4BpmnServerComponent);

Remember that the extension uses a multitenant authorization extension https://github.com/ingpconci/loopback4-tenant-table-filter that need to be configured before the BpmnServerComponent

    //--------------------------------------------------------------------------
    // Multitenant Authorization from loopback4-tenant-table-filter
    // Authorization provider
    //--------------------------------------------------------------------------
    const authorizationOptions: AuthorizationOptions = {
      precedence: AuthorizationDecision.DENY,
      defaultDecision: AuthorizationDecision.DENY,
    };
    this.configure(AuthorizationBindings.COMPONENT).to(authorizationOptions);
    this.component(AuthorizationComponent);
    // bind the authorizer provider
    this.bind('authorizationProviders.multitenant-authorizer-provider')
      .toProvider(MultitenantAuthorizerProvider)
      .tag(AuthorizationTags.AUTHORIZER);

The loopback4-tenant-table-filter permit to have a unique DB but different isolated organization For example, the bpmnProcessInstance table contains records of different tenant

When I start the process I set this instance variable in the process with the tenantId of the user

      inputData: {
        tenantId: this.authorizationService.currentTenantId,
        startProcessUserId: this.currentUser.id
      }
    const startEngineResult = await this.bpmnServer.engine.start(processName, inputData);

The bpmn server engine use this information for all operation in the DB in the custom datastore that uses the bpmnProcessInstanceRepository.createNoTenantIdFilter

  // save instance to DB
  static seq = 0;
  private async saveInstance(instance, items) {
    //  this.logger.log("Saving...");
    console.log('CustomDataStore.saveInstance:');
......
......
      //-----------------------------------------------------------
      //Get tenantId from inputData
      //-----------------------------------------------------------
      const currentTenantId = instance.data.tenantId;
      bpmnProcessInstance.tenantId = currentTenantId;

      const result = await this.server.configuration.database.loopbackRepositories.bpmnProcessInstanceRepository.createNoTenantIdFilter(bpmnProcessInstance);
      debug('saveInstance: new instance inserted! result=', result);

[ralphhanna]

I believe the full App (Web UI, Server) can use any framework. but I would rather have bpmn-server not rely on any framework, because that would be restrictive.

In other words I like that bpmn-server to remain on pure nodeJS and only offer ExpressJS for demo purposes only?

So the question is what features in bpmn-server can be enhanced using these frameworks?

Any thoughts?

[kstan79]

I don't think it is good ideal to maintain as independent express due to below reason

1. Pure express framework far behind production use, there is a lot of thing need to do and until up to production level to use. Such as cluster, cache, security, queue system and etc which need high experience developer assemble it in express. Use 3rd party framework provide reliable documentation and step to do that.

2. I believe you wish to remain independent with good reason, but I think that make project getting far from provide easy to use bpmn solution. my thought is bpmn his kind of low code solution and everything include setup/database shall implement low code concept as well. I feel you pick mongodb is wise decision cause it also match low code philosophy concept.

Current project quality, completeness, reliability yet to achieve perfect. In this situation flexible is kind of no-no and drag the progress. Why don't Make everything static at the time being until near perfect, then decide again in future?

[kstan79]

Anyhow, it is my personal perspective, which I believe people wish to find robust bpmn framework rather than bpmn library.

I'm ok with using someone bpmn framework, as long as reliable and suitable

[ralphhanna]

Not sure I understand your recommendation, can you please clarify what do you recommend, keep in mind Express is only used for demo purposes

[kstan79]

1. Many kind users, some is programmer and some is not.
2. Event though the specific user is programmer, he might not expert in typescript and express
3. The user may feel interested in this project and wish to implement in their organisation
4. They may not have capability to walk through complex setup process, and they wish to have something just work fine for production
5. Most of the time, their concern is how to put bpmn inside, how to chain this microservice to their existing webapp. They not care what framework it is, they care is reliable and well documented step. If possible it can install and forget.

Technically most of time the users want something prepackaged for production use, included me. For production package, it need:

1. quick start
2. Documentation for implementation, configuration base on common needs:
3. Authentication/authorisation
4. Database setup
5. Integration with existing frontend web app
6. Some test case, some video guides or walk through

Some of above requirement will need the project to settle down best practice of implement bpmn, in this case flexibility may not a good ideal. We may rather put that as advance topic like how to use others framework (if it is necessary)

[ingpconci]

When I chose the bpmn-server library, I checked other libraries and full frameworks like Camunda Products. The Camunda product meets all of your needs, but it is not open source. I agree with @ralphhanna. It's better to keep the core as a pure Node.js library. Now there is an Express client for demo purposes to verify the behavior of the library, but I agree that it should not be used for production. My approach was to use the library in the main application service based on loopback. Here are the advantages of my approach:

• The BPMN data is saved in the same database as the main application server.
• We don't have to manage any other services or Java applications on the server. However, there is one downside:
• The main application cannot be scaled because it is currently not possible to have two instances of the engine that share process instance data.

I agree that a reference framework could benefit the development of the bpmn-server library. I am also open to discussing the possibility of making my main loopback application service, along with the HTML angular client, open source. This could be useful for managing and interacting with the bpmn server.

[ralphhanna]

@ingpconci Thanks very much for you comments and contribution.

I definitely appreciate any additional contribution to enhance the Client UI.

As for the server, would a package like Memcached help solve the scalability issue?

Thanks again