[Bug]: Nested JSON creation with multiple Enhanced JSON output transforms

[abhishek-j08]

Apache Hop version?

2.7.0

Java version?

20.0.2

Operating system

Windows

What happened?

I recently migrated from HOP version 2.0.6 to 2.0.7 and see this difference in behavior while creating a nested JSON output. With 2.0.6, same pipeline give result without any newline characters or backslashes. I tried the sample pipeline json-output-generate-nested-structure.hpl in both HOP versions and results are different. With HOP version 2.6.0 : { "results" : [ { "campo1" : "C", "details" : [ { "campo2" : "B", "campo3" : "7" }, { "campo2" : "F", "campo3" : "5" }, { "campo2" : "F", "campo3" : "6" }, { "campo2" : "V", "campo3" : "6" } ] }, { "campo1" : "B", "details" : [ { "campo2" : "C", "campo3" : "1" }, { "campo2" : "C", "campo3" : "2" }, { "campo2" : "D", "campo3" : "4" } ] }, { "campo1" : "A", "details" : [ { "campo2" : "B", "campo3" : "1" }, { "campo2" : "B", "campo3" : "2" } ] } ] }

With HOP version 2.7.0: { "results" : [ { "campo1" : "C", "details" : "[ {\r\n \"campo2\" : \"B\",\r\n \"campo3\" : \"7\"\r\n}, {\r\n \"campo2\" : \"F\",\r\n \"campo3\" : \"5\"\r\n}, {\r\n \"campo2\" : \"F\",\r\n \"campo3\" : \"6\"\r\n}, {\r\n \"campo2\" : \"V\",\r\n \"campo3\" : \"6\"\r\n} ]" }, { "campo1" : "B", "details" : "[ {\r\n \"campo2\" : \"C\",\r\n \"campo3\" : \"1\"\r\n}, {\r\n \"campo2\" : \"C\",\r\n \"campo3\" : \"2\"\r\n}, {\r\n \"campo2\" : \"D\",\r\n \"campo3\" : \"4\"\r\n} ]" }, { "campo1" : "A", "details" : "[ {\r\n \"campo2\" : \"B\",\r\n \"campo3\" : \"1\"\r\n}, {\r\n \"campo2\" : \"B\",\r\n \"campo3\" : \"2\"\r\n} ]" } ] }

Issue Priority

Priority: 2

Issue Component

Component: Transforms

[bamaer]

hi @abhishek-j08, would you mind adding a sample pipeline with a reproduction path?

[abhishek-j08]

Hi @bamaer, this pipeline is provided at path hop\config\projects\samples\transformsjson-output-generate-nested-structure.hpl

[bamaer]

I don't see any difference in behavior between 2.7, 2.6 or 2.5, so definitely not a regression. has anything on your system (java version?) changed that could cause a difference in behavior?

[KBoersch]

I can confirm that issue. I'm experiencing the issue. I didn't change anything in the pipeline or the environment.

In 2.6 I get a clean JSON entry back. In 2.7 it escapes every special character.

In my case Windows version Version 10.0.19045.4046 and Java-Version: 17.0.2

[gjongerh]

I can confirm this issue using the standard mentioned example DOES work openjdk version "11.0.17" 2022-10-18 with HOP 2.4 on MAC Big Sur DOES work openjdk version "11.0.11" 2021-04-20 with HOP 2.6 on MAC 13.6.3 (22G436)

NOT working openjdk version "17.0.2" with HOP 2.7 on Windows 11 Home version 23H2 NOT working openjdk 11.0.17 2022-10-18 with HOP 2.7 on MAC BigSur 11.7.10 NOT working openjdk version "17.0.8.1" 2023-08-24 with HOP 2.7 on MAC BigSur 11.7.10 NOT working openjdk version "11.0.11" 2021-04-20 with HOP 2.7 on MAC 13.6.3 (22G436) NOT working openjdk version "17.0.9" 2023-10-17 with HOP 2.7 on MAC 13.6.3 (22G436)

Is does not seem to be the java version. Different java version and three different systems gave the same result.

This is the faulty result

{"results":[{"campo1":"C","details":"[ {\n  \"campo2\" : \"B\",\n  \"campo3\" : \"7\"\n}, {\n  \"campo2\" : \"F\",\n  \"campo3\" : \"5\"\n}, {\n  \"campo2\" : \"F\",\n  \"campo3\" : \"6\"\n}, {\n  \"campo2\" : \"V\",\n  \"campo3\" : \"6\"\n} ]"},{"campo1":"B","details":"[ {\n  \"campo2\" : \"C\",\n  \"campo3\" : \"1\"\n}, {\n  \"campo2\" : \"C\",\n  \"campo3\" : \"2\"\n}, {\n  \"campo2\" : \"D\",\n  \"campo3\" : \"4\"\n} ]"},{"campo1":"A","details":"[ {\n  \"campo2\" : \"B\",\n  \"campo3\" : \"1\"\n}, {\n  \"campo2\" : \"B\",\n  \"campo3\" : \"2\"\n} ]"}]}

[gjongerh]

I just tested with compiling the current master branch (2.8.0), and on Big Sur 11.7.10 java 11 has the same result The nested structure is handled as String instead of json

[gjongerh]

We just tested with release 2.9.0, and on Windows and has the same result The nested structure is handled as String instead of json

[campen]

This is still an issue with 2.10-SNAPSHOT - this breaks on the samples file: json-output-generate-nested-structure.hpl.zip

[hansva]

.take-issue