Add background emitter using Golang Tracker

[alexanderdean]

The idea is:

• Create a daemon written in Golang which should run all the time
• This daemon will embed the Golang Tracker and include some form of persistence (e.g. boltdb)
• The PHP Tracker will "fire and forget" all its events to the daemon

[alexanderdean]

Blocked by https://github.com/snowplow/snowplow-golang-tracker/issues/2

[bkvaiude]

Is there any progress on the PHP Tracker background batch of events processing?

[alexanderdean]

Hi @bkvaiude - afraid not. Is it something you'd like to take a look at? The basic principle is:

• Add a long-running "daemon" mode to the Golang Tracker
• Add a way (gRPC?) of sending an event to the daemon
• Update the PHP Tracker to support the daemon as an output

[bkvaiude]

Hi Alex,

Does implementation mentioned approach will give us 100% of event logging without information loss?

https://groups.google.com/forum/#!msg/snowplow-user/lb-BWvNXIww/YG_Q2GrtGQAJ Also have few questions, in above thread Josh mentioned

The PHP Tracker is still quite an immature project and does not have a lot of the processing features of the other Trackers such as robust persistent storage of events for network/collector failure and Asynchronous processing without the creation of long-running daemons. In that sense the Tracker is still in a somewhat less mature state than others we have available. I only suggest the other Emitters as they have less moving parts; no daemon processes; no writing to local files - there are just less things that can go wrong, and while they are blocking they are also capable of sending huge volumes of events very quickly.

As of now, I'm interested into the PHP Tracker and would like to integrate it in our production server. And I mostly concern about logging information loss, as our primary requirement 100% logs recording. Suppose, Custom events Size: 1.3M

Some of the events are get triggered at the same time, so want to know, is there any benchmark of recording or processing events per second or is there any chances of information loss on the 2000 hits per seconds (without buffer or using the batch limit 50, or even 1000 batches of 50 sent at the same time from different points to same destination)?

Is PHP Tracker ready for production environment?

[alexanderdean]

Hey @bkvaiude - for high performance use cases, yes the planned approach above integrating a Golang Tracking Daemon is the way to go.

[bkvaiude]

@alexanderdean I hope using PHP Tracker which is lacking the features as compared to other Trackers will not make any major difference in the performance. Thanks.

[mscwilson]

Closing because this is very old.