Headless bpmn executor

[linonetwo]

I'd like to use bpmn server in tiddlywiki, as discussed in https://github.com/bpmnServer/bpmn-server/issues/145#issuecomment-1852365807 some time ago, I hope this package can have a pure bpmn executor, that does not depend on mongodb and http server, so I can run the bpmn file purely in Typescript, and get result in Typescript, no HTTP request, also save the internal state to files or in-memory database, no mongodb requires.

Do you think this is possible?

[ralphhanna]

Yes, bpmn-server does not require HTTP but requires a database, you can replace mongoDB with in-memory database by implementing your own implementation of IDataStore

interface IDataStore {
    dbConfiguration: any;
    db: any;
    logger: any;
    locker: any;
    save(instance:any,options:any): Promise<void>;
    loadInstance(instanceId: any,options:any): Promise<{
        instance: any;
        items: any[];
    }>;
    findItem(query: any): Promise<IItemData>;
    findInstance(query: any, options: any): Promise<IInstanceData>;
    findInstances(query: any, option: 'summary' | 'full'|any): Promise<IInstanceData[]>;
    findItems(query: any): Promise<IItemData[]>;
    deleteInstances(query?: any): Promise<void>;
    install(); 
    archive(query);
}

I can help you create a custom install, once you have selected a db and completed the db driver to implement the above interface.

[linonetwo]

I'm working on this today, can you explain or add JSDoc comment to methods, so I can understand what I should do in

2. install
3. archive

and what is difference between instance and item? (maybe it is already in the doc)

And what is the type of query?

I find your mongodb Datastore may return null on loadInstance, so its return type should contains null.

[linonetwo]

I tried my best to write one based on current doc https://github.com/tiddly-gittly/workflow/blob/master/src/workflow-engine/data/tw-datastore.ts

But I need more of you help, thanks!

[ralphhanna]

Sure, can you provide instructions on how to install and debug the code.

• 'installis invoked on first time package is installed throughnpm run setup`
• 'archive` is called to move the instance to a secondary storage
• instance is the entire workflow case, while item is the for each node

[linonetwo]

I managed to make a basic setup. I haven't write UI part, so it is currently only run on server side.

pnpm i
# or npm i will work too
pnpm run dev
# then the server will start, but nothing will write back to fs
# pnpm run dev:wiki  will write changes back to fs

An instance is successfully created here https://github.com/tiddly-gittly/workflow/blob/4f0e831a8862f17d554e3ce6da1ceac36b951a81/src/workflow-engine/startup/engine/engine.ts#L30 You can only play with this instance for this moment. For debugging, I only use console.log ...

And it load the example bpmn https://github.com/tiddly-gittly/workflow/blob/master/wiki/tiddlers/log-console.bpmn

using a custom datastore https://github.com/tiddly-gittly/workflow/blob/master/src/workflow-engine/data/twModelDataStore.ts

and save intermediate state using https://github.com/tiddly-gittly/workflow/blob/master/src/workflow-engine/data/twDatastore.ts

[linonetwo]

My TODO:

• [ ] learn how to use bpmnApi instance to load bpmn file, (and check it is using my custom modelDatastore)
• [ ] write some service for it to call, like log-console
• [ ] see if it save state back to wiki
• [ ] PR the patches\bpmn-server@2.2.8.patch back to this repo
• [ ] refactor mongodb and fs related code to another package, so this package is truely headless and can runs on browser

[ralphhanna]

Here is what I am working on for you so far:

1. Moved Mongodb to a separate package (bpmn-server-mongo)
2. new bpmn-server has no dependency on mongodb
3. new bpmn-server has a default JSONDataStore to save to flat files
4. new package bpmn-node is available for testing bpmn-server without MongoDB

Will let you know when all done.

[ralphhanna]

btw, have you considered running MongoDB remotely ?

[linonetwo]

No, I'm trying to make a local-first workflow app that can run as a single HTML file, this will be very light weight, so more people will get to know and use BPMN.