MikaelGRA / InfluxDB.Client

InfluxDB Client for .NET
MIT License
102 stars 22 forks source link

Problem deserializing very large floating-point values #66

Open SchumacherBrianJ opened 4 years ago

SchumacherBrianJ commented 4 years ago

I have encountered an issue processing very large floating-point values. All of the data being written to InfluxDB are 32-bit floating point values with large exponents. When the values are greater than 1.0E21 (approximately) everything works fine as InfluxDB will serialize the JSON response using exponential format. However, when the values fall into the 1E17 to 1E20 range, InfluxDB serializes those values using all significant digits followed by trailing zeroes. For example, 2.016152E17 is serialized as 201615200000000000. When InfluxDB.Client (Newtonsoft) attempts to deserialize this value it does so as a System.Numerics.BitInteger for which there is no conversion to float (or most any number) and an exception is thrown. I have worked around the issue temporarily by checking for BigInteger, converting it to a string and then converting the string. This is all done in ResultFactory.cs as shown below (excuse the obnoxious comment slashes - put them to highlight change). Another possibility might be to capture the exception during the final SetValue and then create the string

              if( property != null )
              {
                 if( value != null )
                 {
                    if( property.Key == InfluxConstants.TimeColumn )
                    {
                       property.SetValue( dataPoint, timestampParser.ToTimestamp( options.Precision, value ) );
                    }
                    else if( property.IsDateTime )
                    {
                       property.SetValue( dataPoint, DateTime.Parse( (string)value, CultureInfo.InvariantCulture, OnlyUtcStyles ) );
                    }
                    else if( property.IsDateTimeOffset )
                    {
                       property.SetValue( dataPoint, DateTimeOffset.Parse( (string)value, CultureInfo.InvariantCulture ) );
                    }
                    else if( property.IsEnum )
                    {
                       property.SetValue( dataPoint, property.GetEnumValue( value ) );
                    }
                    else
                    {
                       if( value.GetType() == property.Type )
                       {
                          property.SetValue( dataPoint, value );
                       }
                      /////////////////////////////////////////////////
                      //Add check for BigInteger deserialization
                      /////////////////////////////////////////////////
                       else if (value.GetType() == typeof(System.Numerics.BigInteger))
                       {
                          string stringValue = value.ToString();
                          property.SetValue(dataPoint, Convert.ChangeType(stringValue, property.Type, CultureInfo.InvariantCulture));
                       }
                       else if ( value is string stringValue )
                       {
                          if( !string.IsNullOrEmpty( stringValue ) )
                          {
                             property.SetValue( dataPoint, Convert.ChangeType( stringValue, property.Type, CultureInfo.InvariantCulture ) );
                          }
                       }
                       else
                       {
                         ////////////////////////////////////////////////////////////////////////////////////////
                         //Or catch exception for invalid conversion and attemp conversion from string
                         ////////////////////////////////////////////////////////////////////////////////////////
                          try
                          {
                             property.SetValue( dataPoint, Convert.ChangeType( value, property.Type, CultureInfo.InvariantCulture ) );
                          }
                          catch (InvalidCastException)
                          {
                             string stringValue = value.ToString();
                             property.SetValue(dataPoint, Convert.ChangeType(stringValue, property.Type, CultureInfo.InvariantCulture));
                          }
                       }
                    }
                 }
              }
MikaelGRA commented 4 years ago

Interesting that BigInteger does not implement IConvertible. Would have thought so myself.

Not really sure if there is a better way of handling it than this. (Maybe the type check can be done with value is BigInteger instead to avoid incurring whatever minor reflection cost is involved in this implementation)

I will see if I can implement a fix to properly support these big numbers.

sharmilamuthukrishnan commented 2 years ago

@MikaelGRA , any solution for this issue? We are blocked due to this defect.. Or any other work arounds?