snowplow / snowbridge

For replicating streams across clouds, accounts and regions
Other
15 stars 7 forks source link

Underdocumented type for timestamp fields #365

Open jethron opened 2 months ago

jethron commented 2 months ago

When using a Custom JS transform with snowplow_mode = true, the types of timestamp fields like derived_tstamp or collector_tstamp in the incoming event data are JS versions of the time.Time golang objects, but working with these values isn't documented. I was expecting plain strings, numeric timestamps, or native Date objects instead.

The objects appear in JS to be similar to Date in that they have typeof === object, and serialize correctly in JSON to ISO timestamps, (or a different format when converted directly to strings) but are missing all methods that might be expected on a JS Date type, which is slightly confusing.

Both the JSON.stringify(ts) and String(ts) string formats aren't parsable with goja's implementation of Date.parse or new Date, so it's a bit confusing how to actually get a regular Date from these values until you know what the actual type is. (Protip: new Date(ts.UnixMilli()))

In particular I was looking for the unary + operator, which for Dates returns the unix-milliseconds timestamp, but for time.Time objects results in a NaN value, which produces an error encoding message data error when trying to marshal the returned object back to JSON (This error was quite confusing; the JS JSON.stringify() function generally serializes NaN as null so this was a bit annoying to debug).

It would be nice if the non-primitive value types were better documented than "the Data field contains an object representation of the event - with keys as returned by the Snowplow Analytics SDK". At the moment there is no clear indication that you might have to be calling golang methods instead of JS.

colmsnowplow commented 2 months ago

Thanks @jethron

Likely this is some unnoticed quirk of passing data from the Go environment to the JS engine - we can take a look at what's going on here :)

In the meantime, would you like to make a PR to update the docs site?