BuzzCutNorman / tap-mssql

Singer Tap for MS SQL built with Meltano Singer SDK.
MIT License
2 stars 9 forks source link

bug: `hd_to_jsonschema_type` minimum and maximum are rounded for `numeric(18,7)` #55

Closed BuzzCutNorman closed 1 year ago

BuzzCutNorman commented 1 year ago

The method hd_to_jsonschema_type is returning minimum and maximum that have been rounded with given a numeric(18,7) column.

{"minimum": -100000000000.0, "maximum": 100000000000.0, "type": ["number"]}

This has to do with how float values are rounded. I have tried a Decimal type but that isn't allowed. Jsonschema validation will fail if I send a str value.

TypeError: '<' not supported between instances of 'float' and 'str' 

This is what I would expect to see if all is working well.

"long": {"minimum": -99999999999.9999999, "maximum": 99999999999.9999999, "type": ["number"]},
BuzzCutNorman commented 1 year ago

This might be too simplistic of solution, but it has covered the few cases I came up with. When the rounding happens since I am using all 9s in the strings float is rounding to a number that always starts with a 1, has 0s, and a decimal point . . I played around and ended adding a second check to see if 1 is in the string version of str(float(maximum_as_number)) .

if "e+" not in str(float(maximum_as_number)) and "1" not in str(float(maximum_as_number)):

When this fails the else is to send over the minimum and maximum in scientific notation.