search

snowplow / snowplow-scala-analytics-sdk

Scala SDK for working with Snowplow enriched events in Spark, AWS Lambda, Flink et al.
https://snowplow.github.io/snowplow-scala-analytics-sdk/
20 stars 14 forks source link

Placeholder for pre-R73 enriched event support #5

Open alexanderdean opened 8 years ago

alexanderdean commented 8 years ago

We would use the count of fields to determine which version of the enriched event we are working with. See also: #2, #4.

Release Column count Additions Removals
R73 131 xxx xxx
alexanderdean commented 8 years ago

Assigning to @ihortom to populate the release table...

ihortom commented 8 years ago

@alexanderdean - here's my view

Release Column count Additions Removals Renamings
>=R73 (v0.8.0) 128 unstruct_event contexts derived_contexts
R71-R72 (v0.7.0) 131 true_tstamp event_vendor event_name event_format event_version event_fingerprint dvce_tstamp renamed to dvce_created_tstamp
R63-R70 (v0.6.0) 125 tr_currency tr_total_base tr_tax_base tr_shipping_base ti_currency ti_price_base base_currency geo_timezone mkt_clickid mkt_network etl_tags dvce_sent_tstamp refr_domain_userid refr_dvce_tstamp derived_contexts domain_sessionid derived_tstamp
<=R62 (v0.5.0) 108
alexanderdean commented 8 years ago

Looks great, though I don't think we ever removed derived_tstamp:

https://github.com/snowplow/snowplow/blob/master/4-storage/redshift-storage/sql/atomic-def.sql#L181

ihortom commented 8 years ago

According to release notes we renamed derived_tstamp to dvce_created_tstamp

alexanderdean commented 8 years ago

Have you got a link to that Ihor? I think we renamed dvce_tstamp to dvce_created_tstamp...

ihortom commented 8 years ago

You are right, amended the table

alexanderdean commented 8 years ago

Looks great, thanks. Actually fewer iterations that I was remembering...

bogaert commented 8 years ago

Is there a reason not to add a renmaings column, in addition to additions and removals? That will make it a little clearer that dvce_tstamp was renamed to, not replaced with, dvce_created_tstamp.

alexanderdean commented 8 years ago

Good idea, let's make that change

ihortom commented 8 years ago

Updated the table