Triggering events in view code

[poswald]

Setting template context variables and having them output as Javascript is a good solution for events/properties that are triggered by GET requests. For view-based code it makes things a little harder because the POST-Redirect-GET pattern wipes out the context. In the Mixpanel documentation for django-analytical you recommend using something like the mixpanel-celery project to trigger events from view code. This solves the problem for mixpanel but it leaves the other methods unaddressed. There is a (rather spartan) kissmetrics API project that could be used for that one as well, however I think one of the nice things about django-analytical is that it brings all of these into one place. I think there would be value in trying to standardize the event/property triggering API.

Solutions could include:

• Leave it as it is and let users call into whatever library they want to to trigger events
• Use something like django-flash to persist the context across requests and let the user's browser send the event to the service on the next GET.
• Add a tasks.py file to the project with appropriate import tests so that iff the user has celery installed, they may call to have an event registered with the appropriate service and it will be handled async. These tasks could be written to depend on the 'official api' and therefore require a library or be custom written.

For now I am going to be implementing the first option in my code since its the easiest option. Are there enough similarities in the other API's to justify one of the other options?

[jcassee]

Hi Paul,

This is a very interesting topic. The Rails analytical Gem I was inspired by actually supports events, but the way it passes variables through the stack is not applicable to Django. I have left out events until now because I did not know how to generalize them over all services. (You noticed I had not documented KISSmetrics events.) With a second person interested in events, maybe the time has come to look into them again.

BTW, when you say "for now I am going to be implementing the first option", are you saying not doing events at all?

I like your third option a lot. It sounds like a useful feature, not too much work if it could be based on official api's and libraries. And I have seen little variation of the notion of 'event' in the services.

Would you be willing to spend time on this?

Regards, Joost

[poswald]

When I say I have implemented the first option, I mean I added the official KISSmetrics API into the project and then did something like this:

        # Record web metrics
        try:
            from settings import KISS_METRICS_API_KEY
            from KISSmetrics import KM
            if KISS_METRICS_API_KEY:
                km = KM(KISS_METRICS_API_KEY)
                km.identify(request.user.username)
                km.record('Signed Up', {
                    'Prop1': prop1,
                })
                logger.info("Sent 'Signed Up' event to KISSmetrics")
        except Exception, e:
            logger.error("Problem sending 'Signed Up' event to KISSmetrics")
            logger.exception(e)
            pass # always continue

I did it once but I don't want to be peppering my code with these kinds of blocks. Ideally, I want something like:

analytical.record( username, 'Signed Up', {
    'Prop1': prop1,
}) # send an event with optional properties
analytical.record( username, {
    'Prop1': prop1,
}) # send a property

... and I don't want that call to be able to throw any exceptions under any conditions. Recording an event isn't important enough to break flow control. Sort of like the Django send_robust signal call. In fact, I was also considering creating django events for all of these and then catching them all in a separate metrics application in my project.

Behind the scenes it could use a REST call, the vendor's API directly (blocking call to their lib which should be async), a django signal which calls their lib (which is also blocking until it calls their lib), a celery task (async) or something else.

In this case the event Signed Up is a KISSmetrics specific event. I would expect to have many of these events in my code and would also be defining my own. I suppose the heuristic is all events go to all services. KISSmetrics suggests defaults but also lets you map their default events to your own naming of the event. Google doesn't suggest event names at all. I don't know about the other services.

I am only evaluating KISSmetrics at this point so I cannot guarantee any time spent on it. That being said, there is a gap in functionality between both the official API and this project that would be nice to close. The official API doesn't have a template tag and this doesn't do events.

How do you handle sending these 3rd party events in your applications?

[jcassee]

Hi Paul,

Actually, I only use Mixpanel events currently, and the mixpanel-celery project currently suits my needs.

Now that I had some more time to think about it, I have changed my mind about the solving the POST-GET event problem. If you look at the mixpanel-celery code, it is very much a different beast from django-analytical. There would be no overlap in API or code between the Javascript stuff and service-specific API's. Your code sample (which I agree would be a nice API) has little to do with the way django-analytical works. (The original Rails app did have a similar API for the Javascript stuff, by the way.) So I think it would make more sense to generalize mixpanel-celery.

The problem I see with a flash-based solution is that if the user goes back in his history, the event might be triggered again.

By the way, many services suggest using a Javascript event on your form submit button. For example, see the documentation about forms for KISSmetrics. Mixpanel has a similar API. Both of these services support added key/value properties. I think this way of tracking form submission could be supported by django-analytical in a generic way.

Regards, Joost

[poswald]

I initially looked at that Javascript form submission hook but relying on the browser for that would cause false event submissions. I don't really understand the point of using the Javascript form-submission API in a non-Javascript server-side framework like django. Form validation is done server side and we don't know if an event should be sent to the metrics site until we are halfway through the view function. At that point, we're making changes to the server side state so we're also committed to sending out a redirect.

With a flash-context based solution the event would be fired once when the next page is loaded up. If you went back and then forward again, then the context has been blanked out so the code isn't loaded. If you go back and resubmit the form, then you've triggered the event again and it should be sent to the metrics site again. If you don't want that then you would need to test for that behavior in your view code anyway.

I agree that adding event handling code (like mixpanel-celery) would make it a different beast since it would go from being "Just generate the appropriate Javascript in a tag and let the browser handle the submission" to one where the server is directly using python or REST api's. That being said, I hoped this library would solve my problems with regards to metrics collection and right now it only half does the job. You're happy with this situation because mixpanel-celery exists but there's no corresponding kissmetrics version.

Anyway... that's too much talking from me. I've got it working using my code snippet above for now. As I add more events, I'll probably pull it into an API of sorts and/or create a kissmetrics-celery style app.

[jcassee]

Hi Paul,

There is just one question I have about a flash-based solution.

2011/9/21 Paul Oswald reply@reply.github.com:

With a flash-context based solution the event would be fired once when the next page is loaded up. If you went back and then forward again, then the context has been blanked out so the code isn't loaded. If you go back and resubmit the form, then you've triggered the event again and it should be sent to the metrics site again. If you don't want that then you would need to test for that behavior in your view code anyway.

Isn't this only true if the page you are redirected to is not cached?

Regards, Joost

Joost Cassee http://joost.cassee.net

[poswald]

I suppose so. The flash-context is very similar to the contrib.messages framework.

After a POST, when I redirect to a GET page I typically insert a message:

messages.info(request, 'Your new account was created.')

This messages are iterated (and hence deleted) on the next GET request. The rails-style flash context is a generalization of this to objects other than messages.

Naturally, if you redirect to a page that is being served out of an external cache, it won't go through the templating machinery. If you are just using the django cache backend for caching the template files, it will still run. If you're using per-view or per-site caching then it won't run reliably and yes you're going to have problems. I think web applications with logged in users are typically dynamic enough that caching on a per-template-fragment basis is probably the best you can do.

[jcassee]

Okay, I think a flash-based option is probably the best way of doing events in the POST-Redirect-GET cycle.

As for caching, the messages framework has the same problem. Any sane caching strategy will vary on cookies or deal with it in another way. Other flash contexts will just work the same way.

Although I have currently no time to work on this myself, I think it would be a very useful feature. I am leaving this issue open.

Thanks for the interesting discussion!