search

strongloop-archive / strong-gateway

DEPRECATED | Our new gateway product is available at https://github.com/strongloop/microgateway.
Other
112 stars 46 forks source link

Spike: Implement a middleware/policy for API metrics/analytics #3

Closed raymondfeng closed 9 years ago

bajtos commented 9 years ago

@raymondfeng @chandadharap Please add more details to the story.

chandadharap commented 9 years ago

Needs a tech design doc: specs or something. An approach.

bajtos commented 9 years ago

Backend/protocol integration is out of scope of this story. A simple key-value store will be used in this story.

bajtos commented 9 years ago

Discussion with @ritch:

We need to specify the metrics that are tracked by a 3rd party system like segment.io, e.g. requests per user per day, requests per url per day.

Expected end user experience: open a dashboard (e.g. segment.io) and see the number of unique authenticated visitors per hour for the last 48 hours.

Implementation details:

  1. How to define a bucket "number of unique userIds per hour". Note that bucket [userId, hour] would contains requests per bucket, that's something else.
  2. Forward this data to a 3rd party service - see #13.

The implementation depends on the 3rd party backends - e.g. segment.io collects events, we don't need pre-processing in the gateway.

bajtos commented 9 years ago

Plan for the spike: create a non-customizable middleware forwarding relevant data to segment.io, learn more about the capabilities of such solution. Discuss the findings with Issac and/or Al to verify the direction, work out the desired user experience and fill proper user stories (work items).

chandadharap commented 9 years ago

Al to detail the end-to-end user story of feature.

chandadharap commented 9 years ago

Cannot tie to segment.io Statsd, PM API

altsang commented 9 years ago

Our current use case being worked on in the gateway are the same ones as released in the OSS loopback gateway:

use case scenario

  1. as a REST client - send request for an API endpoint to the gateway, e.g. GET /api/coffeeshop
  2. run a policy to see that the endpoint is protected by Oauth2, this implies that endpoints can be protected by different auth schemes and that policies are used to match endpoints to auth schemes
  3. check auth credentials via Oauth2 as a provider to the user
  4. run a policy that captures the invocation of the API request (whether or not authorized)
  5. run and check a policy that matches the user to a rate limitation of X calls per Y interval, e.g. 5000 calls per hour

In step 4, we would want to take ideally take advantage of our executing strong-agent functionality. Strong-agent would gather raw data. Through a collection process, we would do some form of aggregation into a conventional data store.

The data store should be accessible so that business intelligence (such as a Microstrategy, Cognos, etc)or dashboard/reporting tooling (something as simple as Excel) can be run on top of it to allow a business user to run queries knowing only a published schema. This implies a lowest common denominator of SQL. Taking this approach - we relieve the pressure of having to build a full fledged analytics API to anticipate the type of reporting that needs to be done until more concrete use cases are known. However - we could have an analytics API provided to us for "free" by leveraging LoopBack

In step 4 - I would expect data to be like the following:

raw data collected through agent interface:

{user: altsang, endpoint: "GET /api/coffeeshop", timestamp: "2014-12-24T13:15:15"}

reporting schema:

user endpoint counts
altsang /api/coffeshop 6

spike approach for spiking purposes, as discussed with @bajtos , we'll first explore the reporting schema to demonstrate what the reporting would look like and a REST api built using LB.

Secondarily, we will figure out what would be the process for raw data collection and then subsequent aggregation (see #13).

bajtos commented 9 years ago

Submitted a pull request - see https://github.com/strongloop/strong-gateway/pull/20.

Until the patch is landed, you can try out the GUI by checking out the branch feature/metrics.

bajtos commented 9 years ago

Blog post: https://strongloop.atlassian.net/wiki/display/SL/2015/01/09/API+usage+metrics+-+Architecture+overview

Follow-up stories:

Reassigning to @altsang for verification.

/cc @chandadharap

altsang commented 9 years ago

reviewed as a team, the spike is good for now to leave options on the table