Please add the following time-related custom dimensions in the pipeline before storing in Bigquery

[ngulamai]

Hi We would really appreciate if you could calculate in the pipeline the following time-related custom dimensions and write them in big query

TimeZone --> CD91 LocalTime --> CD92 Day --> CD93 Weekday --> CD94 (example: Monday, Tuesday, etc) Month --> CD95 Year --> CD96

[akolchin-MM]

@ngulamai All other values can be calculated but they depend on the certain value of TimeZone. But we need to discuss and agree on a certain solution for the TimeZone determining.

OWOX solution is using next approach "... according to a time zone of a Google Analytics view selected during streaming setup" Unfortunately, we don't have such explicit setting so we can't determine any certain meaningful TimeZone value for hits.

For now we use the same one hardcoded "Europe/Madrid" time zone in our code for ALL hits and calculate all time related values (date, time, hour, minute, etc.) using this timezon.

While we don't have any better idea we can continue to use this approach and we can implement this certain task by using hardcoded "Europe/Madrid" time zone too.

If can advise some better idea, please let us know.

[ngulamai]

You may calculate time zone with these datapoints ( CD33 -- Country Ad Server CD30 -- City Ad Server

Alternatively if you can also use CD34 -- country Publisher CD31 -- City Publisher

On Mon, Feb 5, 2018 at 10:54 AM, akolchin-MM notifications@github.com wrote:

All other values can be calculated by they depend on the certain value of TimeZone. But we need to discuss and agree on a certain solution for the TimeZone determining.

OWOX solution is using next approach "... according to a time zone of a Google Analytics view selected during streaming setup" Unfortunately, we don't have such explicit setting so we can't determine any certain meaningful TimeZone value for hits.

For now we use the same one hardcoded "Europe/Madrid" time zone in our code for ALL hits and calculate all time related values (date, time, hour, minute, etc.) using this timezon.

While we don't have any better idea we can continue to use this approach and we can implement this certain task by using hardcoded "Europe/Madrid" time zone too.

If can advise some better idea, please let us know.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ngulam-ai/Sherlock/issues/18#issuecomment-363034452, or mute the thread https://github.com/notifications/unsubscribe-auth/AfvWTSPbdbSl9xr6PTc6-tDV6WKYwX8Dks5tRs_igaJpZM4R4f7w .

[akolchin-MM]

You may calculate time zone with these datapoints...

Yes, we can. And if you think that it has sense, we can try to use. It will increase processing time for each one "hit" but I hope not critically.

For me it is difficult to assess every such decision. I only would like to mention that actually all fields required to calculate all these values (TimeZone, LocalTime, Day...), including time zone if we will use proposed by you approach, already contained in the BigQuery table. So if somebody needs them, they can be calculated at the next "data analyzing" stages in the SQL requests to the BigQuery or by creating some Views for it.

Of course, we also can calculate them in our "data collecting and storing" stage too if needed.

For me, it is really difficult to assess this trade-off. Therefore, I rely on your decision.

[ngulamai]

Hi I understand what you say, however as one of the two missing pipelines that we need to build is a click-through pipeline (similar to the measurement protocol's redirect feature https://developers.google.com/analytics/solutions/ios-install-tracking#redirect); we would need to calculate this at data collection time, so that when we rollout that pipeline, we can add that datapoint in the parameters

On Tue, Feb 6, 2018 at 11:36 AM, akolchin-MM notifications@github.com wrote:

You may calculate time zone with these datapoints...

Yes, we can. And if you think that it has sense, we can try to use. It will increase processing time for each one "hit" but I hope not critically.

For me it is difficult to assess every such decision. I only would like to mention that actually all fields required to calculate all these values (TimeZone, LocalTime, Day...), including time zone if we will use proposed by you approach, already contained in the BigQuery table. So if somebody needs them, they can be calculated at the next "data analyzing" stages in the SQL requests to the BigQuery or by creating some Views for it.

Of course, we also can calculate them in our "data collecting and storing" stage too if needed.

For me, it is really difficult to assess this trade-off. Therefore, I rely on your decision.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ngulam-ai/Sherlock/issues/18#issuecomment-363381558, or mute the thread https://github.com/notifications/unsubscribe-auth/AfvWTcyocc68FCGiyNcpzvYQ0BxoEYK3ks5tSCspgaJpZM4R4f7w .

[dchubiryaev]

@ngulamai Hello! I was did custom dimension, but I want clarify one moment. What format do you need these custom dimensions? Now we have: "CD96": "2018", "CD95": "02", "CD94": "Thursday", "CD93": "08", "CD92": "15:29:29.401", "CD91": "Europe/Madrid" CD91 have format example: (https://garygregory.wordpress.com/2013/06/18/what-are-the-java-timezone-ids/) If you need another format any CD, please say me.

[ngulamai]

I believe it is fine. We will let the team that is working on the machine learning workflow to decide if they need any transformation Thanks

On 9 Feb 2018, at 12:45, Danil notifications@github.com wrote:

@ngulamai https://github.com/ngulamai Hello! I was did custom dimension, but I want clarify one moment. What format do you need these custom dimensions? Now we have: "CD96": "2018", "CD95": "02", "CD94": "Thursday", "CD93": "08", "CD92": "15:29:29.401", "CD91": "Europe/Madrid" CD91 have format example: (https://garygregory.wordpress.com/2013/06/18/what-are-the-java-timezone-ids/ https://garygregory.wordpress.com/2013/06/18/what-are-the-java-timezone-ids/) If you need another format any CD, please say me.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ngulam-ai/Sherlock/issues/18#issuecomment-364412515, or mute the thread https://github.com/notifications/unsubscribe-auth/AfvWTbT0emNcCwH6WyTsChJxERVQRuCdks5tTC_rgaJpZM4R4f7w.

[akolchin-MM]

@dchubiryaev Need to improve solution as we have discussed it.