search

rcongiu / Hive-JSON-Serde

Read - Write JSON SerDe for Apache Hive.
Other
733 stars 391 forks source link

Fixed two tests in JsonSerDeTest #241

Open Sujishark opened 9 months ago

Sujishark commented 9 months ago

What is the purpose of the change: To address the validation issue created due to the potential variations in String ordering during comparison.

This PR aims to fix two tests:

org.openx.data.jsonserde.JsonSerDeTest.testNestedExplicitNullValue
org.openx.data.jsonserde.JsonSerDeTest.testExplicitNullValue

REASON AND FIX:

The issue arises while serializing the {column: column types} pair, the result returned is not in a constant order. This is because result is a JSONObject which internally uses HashMap and doesn’t maintain order.

https://github.com/rcongiu/Hive-JSON-Serde/blob/7a05e91bf80a781da30cc69e2f5698b78f5b3397/json-serde/src/main/java/org/openx/data/jsonserde/JsonSerDe.java#L265

Due to this, validating the jsonStr with another string doesn’t turn out to return true. The fix is to modify the assert statements in a way to check only the presence of the individual key-value pair and ignore the order.

REPRODUCE: This was found by using the NonDex tool. 

cd Hive-JSON-Serde
mvn install -pl json-serde -am -DskipTests
mvn -pl json-serde edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.openx.data.jsonserde.JsonSerDeTest#testNestedExplicitNullValue