[Bug] TripsLayer's vTime has rendering problems when the segments are short with precise timestamps

fungiboletus commented 5 months ago

Description

I had an issue with the TripsLayer: it was mis-rendering the beginning of my trip paths.

It's similar than what is described there:

After debugging for a while, I noticed that the rendering bug was only present when precise timestamps were used, such as the unix epoch in seconds or microseconds. The scale of the numbers didn't really seem to matter. Dividing the timestamp by one million and updating updateTime and trailLength accordingly didn't prevent the rendering bug. Maybe my dataset has too many points, or points too close to each other.

I'm not an expert with deck.gl and shaders, but I feel like it's a precision issue on the vtime there: https://github.com/visgl/deck.gl/blob/a51fc6966994b1fcb90a534baf563b3cd1444f0a/modules/geo-layers/src/trips-layer/trips-layer.ts#L77

In my case, I replaced the calculation by the much simpler following code because my data has enough points:

vTime = instanceTimestamps;

This issue has been useful to me: https://github.com/visgl/deck.gl/issues/6844

Flavors

[ ] Script tag
[X] React
[ ] Python/Jupyter notebook
[ ] MapboxOverlay
[ ] GoogleMapsOverlay
[ ] CartoLayer
[ ] ArcGIS

Expected Behavior

Steps to Reproduce

A trip layer with a simple path.

In my case:

[
[14.959, 68.9856],
[14.954, 68.9902],
[14.95, 68.9948],
[14.946, 68.9994],
[14.939, 69.007],
[14.932, 69.015],
[14.925, 69.023],
[14.921, 69.0275],
[14.916, 69.032],
[14.912, 69.0365]
]

And a set of timestamps that have many digits.

Environment

Framework version: 9.0.16
Browser: Microsoft Edge 125.0
OS: MacOS Sonoma 14.5

Logs

No response

Pessimistress commented 5 months ago

https://deck.gl/docs/api-reference/geo-layers/trips-layer#gettimestamps

Because timestamps are stored as 32-bit floating numbers, raw unix epoch time can not be used. You may test the validity of a timestamp by calling Math.fround(t) to check if there would be any loss of precision.

You said you tried to divide the timestamp values by one million - that does not change the precision required to store them.

A common technique is to remove an arbitrary value from the timestamps, e.g.

const START_TIME = 17e8

new TripsLayer({
  // ...
  currentTime: dateTime - START_TIME,
  getTimestamps: d => d.timestamps.map(t => t - START_TIME)
})

What is important to you is the relative time between the way points, not the relative time to 1970-01-01.

fungiboletus commented 5 months ago

Oh I see now that it was mentioned in the documentation. I missed that:

Because timestamps are stored as 32-bit floating numbers, raw unix epoch time can not be used. You may test the validity of a timestamp by calling Math.fround(t) to check if there would be any loss of precision.

Thanks for the tip about using the relative time. I didn't use a timestamp before and the trips layer was working well.

I got a bit enthusiastic with the trips layer and I display about 320k trips since 2015, and I do need the precision. Starting from 2015-01-01 didn't change much compared to 1970-01-01. I could perhaps have some chunks of times, like every weekly, but that is a bit more complex.

I realise that my use-case is perhaps out of scope. 64-bit floating numbers could perhaps be a nice to have feature, but I don't know about the consequences in terms of performances, or if it's also possible.

For now, I copy pasted the trips layer into my codebase with the much simpler vTime and that works well for me.

By the way, I am very pleased to see that I could feed the trips layer some Float64Array. I receive the data using Arrow IPC with zstd compression, parse it and prepare the array in Rust/WASM, and give to deck.gl.

visgl / deck.gl