Minimal new deployment mechanism for the Spring Boot Starter

[ThorbenLindhauer]

This issue was imported from JIRA:

Field	Value
JIRA Link	CAM-9902
Reporter	@ThorbenLindhauer
Has restricted visibility comments	true

---

• replaces SpringProcessEngineConfgiuration auto-deployment mechanism
• supports the currently exposed application properties
• resource scanning: classpath resources get environment-independent names
• the deployment should be made before any other @PostConstruct callbacks are made for custom beans (those might already reference the deployed resources, e.g. start a process instance)
• @EnableProcessApplication works just as before

Links:

• is depended on by https://jira.camunda.com/browse/CAM-9906
• is depended on by https://jira.camunda.com/browse/CAM-9452
• is depended on by https://jira.camunda.com/browse/CAM-9903
• is depended on by https://jira.camunda.com/browse/CAM-9904
• is depended on by https://jira.camunda.com/browse/CAM-9905
• is depended on by https://jira.camunda.com/browse/CAM-9907
• is depended on by https://jira.camunda.com/browse/CAM-9909
• is depended on by https://jira.camunda.com/browse/SUPPORT-6297
• is depended on by https://jira.camunda.com/browse/CAM-9908
• is related to https://jira.camunda.com/browse/CAM-8109
• is related to https://jira.camunda.com/browse/CAM-10623

[ThorbenLindhauer]

This comment was imported from JIRA and written by user @ThorbenLindhauer

---

Findings and Ideas for "resource scanning: classpath resources get environment-independent names":

It is hard to develop a solution that determines relative resource paths automatically. Spring resource scanning provides us with absolute paths to the resources. It is not obvious how to automatically determine the environment-dependent parts of. Example: Path to resource returned by Spring is /foo/bar/my-app/lib/processes-1.0.0.jar/classes/org/camunda/foo.bpmn. The resulting resource name should be org/camunda/foo.bpmn.

With processes.xml, this file is the marker that all resources are relative to. Removing processes.xml, we do not have that reference point anymore.

Approaches:

• Have a marker file that determines the resource root path (e.g. named camunda-resource-root). Should be in a well-defined location (e.g. in META-INF similar to processes.xml) to avoid that we scan all folders of all elements on the classpath. This may require that this file must have content in order to point to the actual resource root.
• Set up a convention that resources must be located in specific packages on the classpath. This package can be defined via the application properties. Deployment then first determines the resource root, then determines the resources to deploy and matches resource roots and resources to define the relative name where possible. If the user defines the root package as the resource root, then we have to match all resource roots of all libraries with the resources which could be expensive.
• An entry in the manifest of a JAR that determines the resource root. Requires users to define this, which is not easy for everyone and you do not have a manifest in all circumstances (e.g. unit testing in eclipse)
• stick with processes.xml

[ThorbenLindhauer]

This comment was imported from JIRA and written by user @koevskinikola

---

The final decision was not to implement a new deployment mechanism for the Spring Boot Starter project so solve CAM-9452.

Explanations

The goal

The original idea was to consolidate the two deployment mechanisms available in the Starter (Process Application (PA) and non-PA deployment) into a single (new) deployment mechanism. The new mechanism was supposed to offer the advantages of a Process Application deployment like:

• multiple deployments,
• multi-tenant deployments,
• process definition scanning,
• delete upon undeploy,
• explicitly list resources

But offer them in a spring-boot-like manner, i.e. through the existing .properties, .yml configuration files. In addition, we hoped to use the benefits of the Spring resource-loading mechanism.

The requirements

A requirement was that the changes wouldn't break existing Process Applications, or at least would require a minimal adjustment to use the new deployment mechanism. A big part of this was that resourceNames needed to contain relative paths, so that the deployment doesn't depend on the environment. This is made difficult by the Spring resource-loading mechanism, because it provides absoulute resource paths.

The Process Application/Archive concept/requirements

1. Portable
2. Self-contained

Both characteristics depend on the processes.xml file and its contents. Process archives can be declared in both the main application and in JAR files included as libraries of the application.

1. To maintain portability, a Process Application must contain a processes.xml file. The file is used to locate the resource root path and find all the available resources on this path. The resource root path is also used to generate the relative path in resource names.
2. To be self-contained, a Process Application needs the configuration information provided through the content of the processes.xml file. An empty file may be provided, which results in using the default configuration values

Using these properties, a user can change the location of the project without the Process Application breaking.

The proposed solutions

Provide the Process Application configuration properties as (custom) Spring Boot properties. Detect the root resource path through:

• Use an empty marker file (like processes.xml) to detect the root resource path;
• Introduce a directory convention that can be user defined. Resource files must be located in these directories. The root resource path is detected by searching for these resource path directories.

1. Both approaches manage to solve the portability property, i.e. detect the root resource path and provide relative paths for resource names. However, they don't provide a fully compatible solution.
   • Using a marker file removes the possibility to easily define multiple deployments. An inefficient solution, like creating a separate deployment for every available marker, is possible, but what about multi-tenant deployments?
   • Using a directory convention makes things a bit easier. Multiple deployments (and multi-tenant deployments) can be separated in different directories, then those directories specified as root resource directories_. Shared resources can be put into a _shared root resource directory made available to all deployments. However, this requires to create a colmplicated resource file structure which will become impracticle when multiple Process Applications/Archives are used.
2. Both approaches violate the self-contained property of a Process Application. The configuration information is not available in the Process Application archive and will always need to be explicitly defined in the Spring Boot configuration files. Moving the Process Application to a different environment will also require to explicitly copy the properties from the Spring Boot properties file.
3. Both approaches negate the benefits of using the Spring resource-loading mechanism.

We also considered solutions where we automatically try to detect the root resource path, e.g. based on the presence of a Spring Boot application class or similar (as for example tried in https://github.com/camunda/camunda-bpm-spring-boot-starter/pull/296). We could not make that work, i.e. there are no classes, properties or other resources that are always present in a Spring Boot application and that can be used to determine the root path of the application (and its libraries).

The result

In the end, both solutions introduce more complexity to maintain compatibility, but don't provide a lot of advantages to justify this. Furthermore, we currently have little information on how the Starter project is used, which makes making assumptions in order to simplify/remove features difficult.

[ThorbenLindhauer]

This comment was imported from JIRA and written by user @ThorbenLindhauer

---

More options for the problem that we cannot tell at which point in a resource name the classpath begins:

• We can change the logic that determines the name of a deployment resource such that
  • The default name is the classpath name
  • For every deployment resource (or pattern), the users can specify a prefix path; this path will then be prepended to the resource name when making the deployment
  • Users can provide a bean of a certain type that allows to set this programmatically

Configuration example (related to CAM-9905):

camunda.bpm.deployments<0>.resources<0>.pattern = classpath:foo.bpmn
camunda.bpm.deployments<1>.resources<0>.pattern = classpath:bar.bpmn
camunda.bpm.deployments<1>.resources<0>.prefix = deployment1
camunda.bpm.deployments<2>.resources<0>.pattern = classpath:baz/***/**.bpmn
camunda.bpm.deployments<2>.resources<0>.prefix = prefix1
camunda.bpm.deployments<2>.resources<1>.pattern = classpath:bla/***/**.bpmn
camunda.bpm.deployments<2>.resources<1>.prefix = prefix2

Deployment 0 will have a resource with name foo.bpmn. Deployment 1 will have a resource with name deployment1/bar.bpmn. Deployment 2 will have a resource with name prefix1/file1.bpmn and another resource with name prefix2/file2.bpmn (note that the scanned path is not part of the resource name).

This would be a practical solution to solve the portability problem with only Spring Boot configuration properties. In individual cases, it may not be trivial to migrate existing solutions (with processes.xml or current auto-deployment mechanism) to the new deployment way.