Add Feature to define OpenAPI spec POST endpoint that will create BPMN process instances and resume instances using trapped signals

[dubnemo]

Is your feature request related to a problem? Please describe.

We need a way to expose an API-first standards-based REST endpoint using OpenAPI specification 3.0 endpoint so that middleware tools like Mulesoft, Fuse, etc., could POST a ProductionOrder to a Camunda runtime. Camunda would immediately create a BPMN instance in a synchronous manner on receipt of the HTTP POST.

I was looking for a feature in the Modeler on either the start task or the receive message task to accept this JSON payload as an input variable, and be able to use JSON path to extract data from the JSON payload, and display in a human task. $variableName.productionOrder.identifier to get the identifier of the production order, $variableName.productionOrder.lastModificationDateTime to get the date of the production order, and so forth

Describe the solution you'd like

The goal is the manufacturing plant would be able to view these Production Orders in the tasklist, and a user task would allow viewing on the production order. To get and set each property to a separate variable is burdensome, and JSON path much more eloquent and likely more performant than a transformation. It would be good to archive the original payload as well.

We have initiatives for small and medium sized businesses that need simpler manufacturing process tool, and Camunda would a great tool at work center level.

I have attached the OAGIS OpenAPI Specification generated using the NIST/OAGi Score tool found in Github. This is an open standard and the Score tool is in open source.

We would need a way to reference the attached YML file during deployment.

There would need to be a way to update the YML in the Modeler, create a variable for the physical artifact, and reference that in the BPMN.xml.

Describe alternatives you've considered

I have been looking at the REST API and do see POST /message, but there is no detail in the documentation. https://docs.camunda.org/manual/latest/reference/rest/message/ It appears to be an idea or work-in-progress.

I am also looking at POST /signal But this appears to be a signal task for a running instance. I need to create an instance.

I have also explored other middleware technologies to front-end Camunda, by hosting this API and handing off the JSON to the process. But herein lies the challenge. Why another tool? Or why should I have to write Java code to 'wrap' camunda?

Production-Order-oagis-id-4abf241d4f2d47cebf6d44382a9fdced(POST).yml.txt

[pinussilvestrus]

Thanks for your feature request. Can you describe/ sketch what your proposed solution for the Camunda Modeler could look like? Applying our issue template would also help.

[dubnemo]

@pinussilvestrus sorry, I blew that text away, which I thought I covered most of the ask. I can try to sketch something up.

[dubnemo]

@pinussilvestrus I reformatted the original ask per your template, adding some text. The screen shots for the new type and new listener are self-explanatory.

Here are BPMN diagrams showing our goals:

End-to-end manufacturing process (overly simplistic for now, showing flexible manufacturing process). The receives ProductionOrder JSON, calls sub-processes POST /Operation and receiving WIPStatus when complete.

Workcenter level template This accepts POST /Operation JSON and sends POST /WIPStatus when complete or downtime.

[dubnemo]

Consider adding JSON type

[dubnemo]

Consider adding HTTP listener

[pinussilvestrus]

Thanks for describing your use Case. It's still hard for me to understand your dedicated feature request for the Camunda Modeler. Your use case sounds very special and I can't see a standard feature the Modeler can offer so far.

Can you briefly describe the Feature you're thinking Off, and where to implement in the Modeler?

[pinussilvestrus]

Did you also consider asking in the forum?

[dubnemo]

We have used Oracle BPM (OBPM) in the past, and it offered capabilities to call a SOAP web service and now REST API that would automatically generate a process instance. We also had OBPM monitor JMS queues for messages. Then we would map from the message payload to to ADF forms.

I will also look/ ask in the forum.

[dubnemo]

@pinussilvestrus
I am seeing a number of form posts related to this, with unanswered questions, e.g., https://forum.camunda.org/t/input-parameter-variable-type-json/10022 The above is EXACTLY my request, but with the response did not answer the need.

https://forum.camunda.org/t/how-can-use-json-variable-in-camunda-modeler/20528 The above is unanswered

It seems that people are having to write Java, Javascript to get and set individual variables using the Spin Dataformat. That should be transparent to the user and a single variable should be set to the JSON payload instance. One should not have to parse the file and set individual properties then reference those new properties in a form or BPMN expression. JSON path navigation should be the approach.

The HTTP listener goal is to provide 1) a new way to provide a input path to set the above JSON type based on the HTTP request body for content-type application/json, application/oagis+json (future ICANN entry) or custom vnd content-types, and 2) provide a way for the runtime deployed in Tomcat, or what container used, to route by the OpenAPI operation and resource path to the proper BPMN process. This feature would also provide means to receive catch event notifications from other systems if the business process requires data in an asynchronous manner from an ERP or other systems.

[volkergersabeck]

Hi @dubnemo ,

Thanks for sharing all the details.

Have you considered using our REST API to trigger a new process instance (using a default start event) and adding the JSON that you mentioned as a process variable? https://docs.camunda.org/manual/latest/reference/rest/process-definition/post-start-process-instance/

Then you could use a simple script task as the first task of your process that reads the JSON and creates further variables from it using setVariable(). https://docs.camunda.org/manual/latest/user-guide/data-formats/json/#scripting-integration

Please let me know if this would solve your problem or whether there is anything on top that you would require as a new feature of the Camunda Modeler.

Best, Volker

p.s.: Please find attached the Postman config for the REST call that triggers a new process instance and the bpmn file with a script task reading the JSON input from the start event. json-as-variable-example.zip

[dubnemo]

@volkergersabeck While I appreciate your goal to get me 'operational', that is not the theme of this enhancement request.
1) it does not address the import of OpenAPI 3 spec for a specific variable (UI to add type) 2) the copy of data from the JSON payload to a new variable (setVariable()) increases memory utilization, therefore less performant, and is redundant. The intent is to be able to access the data directly from an UI form using JSON path from the initial variable set. Some invoice payloads may have one hundred or so data elements (e.g., Peppol and the Federal Reserve's Business Payments Coalition here in the United States). 3) be able to expose my own endpoint (UI change add the HTTP listener) with a standards-based JSON, in this case OAGIS from the Open Applications Group. Looking at the post-start-process-instance documentation, this would like be typed as object, and the client would have to pass as an escaped string which in my opinion, should not be required based on the approach I am suggesting.
"For variables of type Object, the serialized value has to be submitted as a String value." 4) Last, this could be expanded to trap 'signal' events, by calling these APIs. Your suggestion focuses on the create the process instance, not to resume the process instance based on a trapped event via API. I have been looking for API calls for this, but the documentation seems to be incomplete (work in progress). These signals could come from delay response manufacturing operations processes, from other systems or programmable logic contollers (PLCs).

While it seems that SPIN does handle JSON Path, https://docs.camunda.org/manual/latest/reference/spin/json/03-querying-json/ String value = JSON(json).jsonPath("$.child[0].name").stringValue(); this should be able to be used directly from a UI form text control. I am not seeing that capability at this time. I have been reviewing a number of references, e.g., https://blog.camunda.com/post/2018/08/modeler-plugin-embedded-form-generator/ but only see list of parameters which I would have to copy from my input payload using a similar script as you have provided (thank you). I am reviewing documentation for scripting within embedded forms: https://docs.camunda.org/manual/7.13/reference/embedded-forms/ https://docs.camunda.org/manual/7.13/reference/embedded-forms/json-data/

In summary, this is less of a 'problem' but a means to provide better efficiencies from design to deployment of the process model, and to provide better interoperability with other systems especially with 'business collaborations' modeled in BPMN where message flows and signals are predominantly used.

Thanks.

Scott Nieman Expert Enterprise Architect Land O'Lakes

[nikku]

Thanks @dubnemo again for sharing your feedback and clarifying your use case!

While it seems that SPIN does handle JSON Path, https://docs.camunda.org/manual/latest/reference/spin/json/03-querying-json/ String value = JSON(json).jsonPath("$.child[0].name").stringValue(); this should be able to be used directly from a UI form text control. I am not seeing that capability at this time.

You are right that working with JSON documents is limited at this point and could be improved in the future.

If what you are searching for such improved capabilities right now, I strongly suggest you to have a look into Zeebe, our next generation process engine. Over there, we use JSON only as variables with FEEL for scripting / extracting data (cf. Zeebe variables and Zeebe Expression Language.

If you are searching for a more convenient way to interact with Open API specified endpoints I ask you to share your use case with others via our forums. If a significant number of our users require such capabilities we may account for it on our roadmap.

I'll close this issue for now, as I do not see that we can tackle this specific scenario in the near future.

[dubnemo]

Since Swagger Editor can take an OpenAPI 3.x specification and generator Spring Boot code, it may be worth the effort to leverage the models folder content to reference the JSON payload as variable. I believe code from older Java libraries that display XML trees can be refactored to display JSON trees. JSON path references can be used to 'set' UI parameters, API parameters in GET calls, and data transformations to DB tables, and so forth.

So perhaps there are two requests 1) a way to set JSON root or other sibling paths as a variable and 2) provide guidance to users of OpenAPI a rapid means to take an API definition and rapidly incorporate as UIs or other aspects of workflow.

BTW: The ProductionOrder, Operation, and WIPStatus are OAGIS standard core components, expressed in OpenAPI 3.0.2 or JSON Schema using the NIST/ OAGi Score tool.
The tool is located at https://oagiscore.net/.
Access info at: https://oagiscore.org/

I will look at Zeebe as well.