poidasmith / xlloop

XLLoop Excel Function (UDF) Server
104 stars 49 forks source link

Number precision only 6 digits #13

Open ZJONSSON opened 11 years ago

ZJONSSON commented 11 years ago

I have an annoying problem with numbers coming from excel (both integers and floats). It seems that any number can only have a maximum number of 6 digits. Any additional digits will be rounded or transmitted in scientific notation. Cell formatting does not make any difference (however defining text containing a number causes no problems, but is inconvenient).

Here are some examples (I'm using Excel 2013 and xlloop-0.3.2 with protocol = http)

input: 123456: 
{"request":"XLLoop",
  "version":"0.1.0",
  "name":"print",
   "args":[{"type":1,"num":123456}]
}

input: 1234567
{ "request":"XLLoop",
  "version":"0.1.0",
  "name":"print",
   "args":[{"type":1,"num":1.23457e+006}]
}

input: 12345678
{ "request":"XLLoop",
  "version":"0.1.0",
  "name":"print",
  "args":[{"type":1,"num":1.23457e+007}]
}

input: 12345.6789
{ "request":"XLLoop",
  "version":"0.1.0",
  "name":"print",
  "args":[{"type":1,"num":12345.7}]
}
poidasmith commented 11 years ago

Thanks for the report. I suspect this will be in the layer between excel and json. Let me take a look.

ZJONSSON commented 11 years ago

Thanks, much appreciated!