Enable self-serve portion of feature and monitor uptake

[azaslavsky]

This includes both a dashboard to track usage and some error reporting (via sentry, of course) so that we know when something is going wrong.