search

datalust / seq-tickets

Issues, design discussions and feature roadmap for the Seq log server
https://datalust.co/seq
97 stars 5 forks source link

Data logged as `long` datatype not being rendered correctly #1699

Closed shaunsales closed 1 year ago

shaunsales commented 1 year ago

Describe the bug We use SnowflakeId extensively, which by default stores IDs as a long datatype. When we log to Seq the ID number is rendered inconsistently and incorrectly. We have seen this behaviour in another package called Swagger, and it ended up being an issue with the JS framework they were using to render numbers.

Expected behavior All integer datatypes should be rendered consistently, matching the original log output data.

Screenshots In the top line, the PositionId is rendered correctly, however in the detail view, the last digit 2 is replaced with a 0

image
nblumhardt commented 1 year ago

Hi Shaun, thanks for reaching out. This relates to https://github.com/datalust/seq-tickets/issues/216. We'll need to take another look at it.

The main stumbling block is that the browser's JSON serializer doesn't support any numeric type larger than 64-bit floating point, so some 64-bit integral values are represented approximately. We implemented a couple of workarounds via the earlier issue - the wavy-equals marker, and formatted value within the log message - but I can see how that's not ideal in your use case.

On the server side, because Seq is .NET and Rust rather than JavaScript, no rounding occurs and your ids will be stored exactly (so at least at this point the data isn't lost).

It looks like you might be using Serilog on the client side. Do you have a custom type wrapping your ids, or are they just long in your .NET code? While we figure out a better long-term approach in Seq, it's likely that a Serilog enricher which turns your ids into strings for JSON encoding will be the best workaround. I can help put this together if you can share any details of how your ids are represented and logged.

Hope this helps, Nick

shaunsales commented 1 year ago

Thanks Nick - I just has a peruse of #216 and it indeed looks like the same issue we're facing.

I think, as you said, the best solution for now is to simply convert long IDs to strings. However I've also realised that our trading data may suffer the same loss of precision if we're dealing with highly precise decimal values for prices and quantities.

Our datatypes for IDs are just longs and we use decimals for all trade accounting - we do use Serilog, so I'll implement an enricher to convert the at-risk integrals and floating point numbers into strings.

Appreciate the feedback. Thanks, Shaun