When using jslt function number() and passing string with big decimal for eg "180000000000" returns scientifc/exponential notation

[jugup]

Hello,

We are using following jslt function to convert string values received to numeric value. The function works well with the small numbers for eg "180" is returned as 180 but when we try passing a big number for eg "18000000.00000" gets converted to 1.8E7.

We are using Jackson to read the input and to write the Json Node as string.

Java code:

            // Create mapper for parsing
            ObjectMapper mapper = new ObjectMapper().enable(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS)
                    .setNodeFactory(JsonNodeFactory.withExactBigDecimals(true))
                    .enable(JsonGenerator.Feature.WRITE_BIGDECIMAL_AS_PLAIN);
            String output;
            // Enable features to handle long decimal values rather than converting them in
            // exponential format
            mapper.enable(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS);
            mapper.enable(Feature.WRITE_BIGDECIMAL_AS_PLAIN);

            // Initialize data and mapping
            JsonNode transformedData;
            JsonNode inputJson = mapper.readTree(data);
            transformedData = template.apply(inputJson);
            output = mapper.writeValueAsString(transformedData);

When we try to pass 18000000.00000 as number in input json, we get the expected result as 18000000.00000. But fails when it is passed as string, so we suspect number() function is converting it into exponential notation

[larsga]

I assume it's the last line here,

output = mapper.writeValueAsString(transformedData);

where the number becomes "1.8E7"? If so, this is not a JSLT question, since JSLT is returning a JsonNode with a decimal number in it. The choice to represent it as "1.8E7" in string form is made by Jackson, not JSLT. I don't think it's possible to modify JSLT (or your JSLT code) to change that.

[jugup]

If its jackson, wouldnt {"amount": 18000000.00000} also give out the exponential result, but instead it works fine, but not when {"amount": "18000000.00000"}

and we have our Jackson mapper to not use scientific notation

[larsga]

I agree it's strange that Jackson does not appear to be obeying your settings, but to see why I think this has nothing to do with JSLT, try running the code like this:

            JsonNode transformedData;
            JsonNode inputJson = mapper.readTree("18000000.00000");
            output = mapper.writeValueAsString(inputJson);

You should still get "1.8E7", even with no JSLT involved at all.

[jugup]

Tried the code, and we are getting expected result 18000000.00000

[jugup]

So it does seem that the number() function is converting it into scientific notation

[larsga]

Nope.

This code:

import com.fasterxml.jackson.core.JsonParseException;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.schibsted.spt.data.jslt.Parser;
import com.schibsted.spt.data.jslt.Expression;
import com.schibsted.spt.data.jslt.impl.ExpressionImpl;

import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.node.JsonNodeFactory;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonGenerator.Feature;

public class Test {

  public static void main(String[] args) throws Exception {
    // Create mapper for parsing
    ObjectMapper mapper = new ObjectMapper().enable(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS)
      .setNodeFactory(JsonNodeFactory.withExactBigDecimals(true))
      .enable(JsonGenerator.Feature.WRITE_BIGDECIMAL_AS_PLAIN);

    // Enable features to handle long decimal values rather than
    // converting them in exponential format
    mapper.enable(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS);
    mapper.enable(Feature.WRITE_BIGDECIMAL_AS_PLAIN);

    // Initialize data and mapping
    JsonNode inputJson = mapper.readTree("18000000.00000");
    //JsonNode transformedData = template.apply(inputJson);
    String output = mapper.writeValueAsString(inputJson);
    System.out.println("output: " + output);
  }
}

produces:

MacBook-Pro-3:tmp larsga$ java Test
output: 18000000.00000

[jugup]

Hello @larsga, as you have also tried the code, when you do

    // Initialize data and mapping
    JsonNode inputJson = mapper.readTree("18000000.00000");
    //JsonNode transformedData = template.apply(inputJson);
    String output = mapper.writeValueAsString(inputJson);
    System.out.println("output: " + output);

it outputs 18000000.00000 but initially you mentioned it would return 1.8E7.

I think there is some confusion. Maybe we can go over the problem statement again.

1. While parsing big numbers using jackson and _Feature.WRITE_BIGDECIMAL_ASPLAIN it does not scientific/exponential result, number is in plain text.
2. When use JSLT with number() function and parse the big number, returned out put is in scientific/exponential notation. Which we think is unexpected because without JSLT just with jackson it works fine (as tried by both of us), maybe the function is causing some issues?

[larsga]

You're right, of course, I was too quick there. Sorry.

The test above showed that when JSLT is not used, Jackson gives the correct output. Your claim is that putting the string through the number() function somehow changes that. So let's test that, too.

I get the same result as you: output: 1.8E7. OK, so something is going on in Jackson.

This code:

    // Initialize data and mapping
    JsonNode inputJson = mapper.readTree("18000000.00000");
    System.out.println("" + inputJson + " " + inputJson.getClass());

    inputJson = mapper.readTree("\"18000000.00000\"");
    JsonNode transformedData = template.apply(inputJson);
    String output = mapper.writeValueAsString(transformedData);
    System.out.println("" + transformedData + " " + transformedData.getClass());

produces:

18000000.00000 class com.fasterxml.jackson.databind.node.DecimalNode
1.8E7 class com.fasterxml.jackson.databind.node.DoubleNode

So the problem here is that Jackson is serializing DecimalNode in a different way from DoubleNode.

Note that what you are having problems with is the string representation of the number, but JSLT is returning DoubleNode; that is, a number, not a string. So the problem is still how Jackson converts the number to a string. It's not JSLT that's producing the string, but Jackson.

You've set WRITE_BIGDECIMAL_AS_PLAIN, but this isn't a BIgDecimal, but rather a DoubleNode. If you convert the node to a DecimalNode the problem goes away. Add this line at the end:

    System.out.println("" + DecimalNode.valueOf(transformedData.decimalValue()));

and you get:

18000000.00000 class com.fasterxml.jackson.databind.node.DecimalNode
1.8E7 class com.fasterxml.jackson.databind.node.DoubleNode
1.8E+7

[larsga]

Sorry. Still too quick. I'm in a rush here.

Last line:

    System.out.println(mapper.writeValueAsString(DecimalNode.valueOf(transformedData.decimalValue())));

Output:

18000000.00000 class com.fasterxml.jackson.databind.node.DecimalNode
1.8E7 class com.fasterxml.jackson.databind.node.DoubleNode
18000000

[jugup]

Yeah, was able to reproduce the same outcome. Tried out following seting on mapper to not use DoubleNode

            ObjectMapper mapper = new ObjectMapper().enable(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS)
                    .setNodeFactory(JsonNodeFactory.withExactBigDecimals(true))
                    .configure(JsonGenerator.Feature.WRITE_BIGDECIMAL_AS_PLAIN, true)
                    .configure(DeserializationFeature.USE_BIG_INTEGER_FOR_INTS, true)
                    .configure(SerializationFeature.WRITE_BIGDECIMAL_AS_PLAIN, true);

but still it doesnt work. Do you know any solution which can help us to overcome this issue?

[larsga]

I'm afraid I don't know of any way to change Jackson's behaviour here.

But I do have one question for you. Why do you care? It's a perfectly usable serialization of the number.

[jugup]

Yeah, i agree, it shouldnt have mattered so much, but have one cosumer who is not able to convert it back to actual number, will try to convence that this is how it is.

[larsga]

It doesn't help to tell Jackson not to use DoubleNode, because that's what JSLT uses internally. I didn't want to use BigDecimal internally for performance reasons.

But if your consumer uses a JSON parser that can't read 1.8E7 then ... well ... it's a part of JSON, so their JSON parser is broken.