search

camunda-community-hub / camunda-7-to-8-migration

A collection of tools to support migration from Camunda Platform 7 to Camunda Platform 8
Apache License 2.0
32 stars 14 forks source link

The CLI converter stops converting in case a process contains an Umlaut #156

Closed ingorichtsmeier closed 1 year ago

ingorichtsmeier commented 1 year ago

Describe the bug

If one process model deployed to an engine and contains an Umlaut and gets converted with the CLI converter, you will get an exception and no results.

Steps to reproduce

  1. Model a process with an Umlaut
  2. Deploy it to the process engine
  3. Convert it with the CLI converter
  4. Find the output: 
    org.camunda.bpm.model.xml.ModelParseException: SAXException while parsing input stream
    at org.camunda.bpm.model.xml.impl.util.DomUtil.parseInputStream(DomUtil.java:245)
    at org.camunda.bpm.model.xml.impl.parser.AbstractModelParser.parseModelFromStream(AbstractModelParser.java:131)
    at org.camunda.bpm.model.bpmn.impl.BpmnParser.parseModelFromStream(BpmnParser.java:65)
    at org.camunda.bpm.model.bpmn.Bpmn.doReadModelFromInputStream(Bpmn.java:250)
    at org.camunda.bpm.model.bpmn.Bpmn.readModelFromStream(Bpmn.java:132)
    at org.camunda.community.migration.converter.cli.ConvertEngineCommand.lambda$modelInstances$1(ConvertEngineCommand.java:59)
    at java.util.stream.Collectors.lambda$toMap$58(Unknown Source)
    at java.util.stream.ReduceOps$3ReducingSink.accept(Unknown Source)
    at java.util.HashMap$EntrySpliterator.forEachRemaining(Unknown Source)
    at java.util.stream.AbstractPipeline.copyInto(Unknown Source)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)
    at java.util.stream.AbstractPipeline.evaluate(Unknown Source)
    at java.util.stream.ReferencePipeline.collect(Unknown Source)
    at org.camunda.community.migration.converter.cli.ConvertEngineCommand.modelInstances(ConvertEngineCommand.java:56)
    at org.camunda.community.migration.converter.cli.AbstractConvertCommand.call(AbstractConvertCommand.java:57)
    at org.camunda.community.migration.converter.cli.AbstractConvertCommand.call(AbstractConvertCommand.java:18)
    at picocli.CommandLine.executeUserObject(CommandLine.java:2041)
    at picocli.CommandLine.access$1500(CommandLine.java:148)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
    at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
    at picocli.CommandLine.execute(CommandLine.java:2170)
    at org.camunda.community.migration.converter.cli.ConvertCommand.main(ConvertCommand.java:17)
Caused by: org.xml.sax.SAXException: Fatal Error: URI=null Line=40: Ungültiges Byte 2 von 3-Byte-UTF-8-Sequenz.
    at org.camunda.bpm.model.xml.impl.util.DomUtil$DomErrorHandler.fatalError(DomUtil.java:207)
    at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(Unknown Source)
...

Expected Behaviour

The process model gets converted

Library version

egal

Camunda version

egal

Camunda Distribution

Tomcat

Other Camunda distribution

No response

jonathanlukas commented 1 year ago

The problem does not lie in our codebase at first glance, we should check how camunda-bpm-platform handles this.

jonathanlukas commented 1 year ago

I tried out but could not reproduce. Let's talk and log our results here @ingorichtsmeier

ingorichtsmeier commented 1 year ago

I could reproduce it with the following steps.

  1. Start my windows laptop
  2. Start SpringToolSuite 4.17 (maybe any other eclipse or even IDE helps as well)
  3. Check the workspace setting for file encoding: Windows -> Preferences -> General -> Workspace. My setting is Default: (windows-1252).
  4. Create a new project in the workspace.
  5. Double-check the file encoding on the project: Right click -> Properties -> Resources -> Text file encoding. My setting for Text file encoding is Other: windows-1252.
  6. Open the Camunda modeler and create a new BPMN Diagram with Umlaute.
  7. Save it in the project.
  8. Deploy the model from the Modeler to a running Camunda-7-Platform.
  9. Convert all process models with the command line converter: 
    java -jar backend-diagram-converter-cli-0.4.3.jar engine http://localhost:8081/engine-rest
  10. Check the output and see the stacktrace.
  11. To double-check, delete the deployment from the cockpit.
  12. Convert again like in 9.
  13. All is good, the example diagrams got converted.
ingorichtsmeier commented 1 year ago

Another finding:

If I start the program from the IDE against my engine, everything is fine.

If I start it from the command line against my engine, I get the exception.

Maybe the XML encoding is influenced by system settings that are overridden in the maven build here: https://github.com/camunda-community-hub/camunda-7-to-8-migration/blob/main/pom.xml#L15-L18 (just a rough guess)

jonathanlukas commented 1 year ago

The build should stick to UTF-8 encoding, however, we could see if the HTTP clients response encoding/decoding can be set depending on the platform encoding.

Here is the entrypoint for this approach: https://github.com/camunda-community-hub/camunda-7-to-8-migration/blob/fe947d342f1da74ff26210ae97dea13bb29c90b0/backend-diagram-converter/cli/src/main/java/org/camunda/community/migration/converter/cli/ProcessEngineClient.java#L46

ingorichtsmeier commented 1 year ago

These are the characters from the BpmnXml:

09:53:02.629 [main] INFO  o.c.c.m.c.cli.ProcessEngineClient - 228:ä
09:53:02.629 [main] INFO  o.c.c.m.c.cli.ProcessEngineClient - 246:ö
09:53:02.629 [main] INFO  o.c.c.m.c.cli.ProcessEngineClient - 252:ü

09:59:24.546 [main] INFO  o.c.c.m.c.cli.ProcessEngineClient - 196:Ä
09:59:24.546 [main] INFO  o.c.c.m.c.cli.ProcessEngineClient - 214:Ö
09:59:24.546 [main] INFO  o.c.c.m.c.cli.ProcessEngineClient - 220:Ü
09:59:24.546 [main] INFO  o.c.c.m.c.cli.ProcessEngineClient - 44:,
09:59:24.546 [main] INFO  o.c.c.m.c.cli.ProcessEngineClient - 32: 
09:59:24.546 [main] INFO  o.c.c.m.c.cli.ProcessEngineClient - 223:ß
ingorichtsmeier commented 1 year ago

This solves the problem: 

java -Dfile.encoding=UTF-8 -jar backend-diagram-converter-cli-0.4.3.jar engine http://localhost:8081/engine-rest

It's a Windows issue if you run it on the Windows command line.

ingorichtsmeier commented 1 year ago

Documented here: https://github.com/camunda-community-hub/camunda-7-to-8-migration/tree/main/backend-diagram-converter/cli