search

DinoChiesa / ApigeeEdge-API-Traffic-Summarizer

A nodejs command-line tool that produces a summary sheet of traffic for an Apigee Edge organization
Apache License 2.0
10 stars 7 forks source link
apigee nodejs

readme

Apigee Traffic Summarizer

This tool queries the Apigee Analytics API to retrieve message_count (traffic volume) statistics for an organization, over a given year.

There are two options for output:

Disclaimer

This example is not an official Google product, nor is it part of an official Google product.

LICENSE

This material is copyright 2018-2022 Google LLC. and is licensed under the Apache 2.0 license. See the LICENSE file.

Usage

Before you can use this tool, you need to get the pre-requisites. These are:

And before using the tool, you must install the pre-requisite packages this tool depends on:

$ npm install

After doing that, you can run the tool. Here's an example, invoking it without any options, which shows the usage message.

$ node ./trafficByApiSummarizer.js -help
Usage:
  node trafficByApiSummarizer.js [OPTION]

Options:
  -M, --mgmtserver=ARG the base path, including optional port, of the Apigee mgmt server. Defaults to https://api.enterprise.apigee.com .
  -u, --username=ARG   org user with permissions to read Apigee configuration.
  -p, --password=ARG   password for the org user.
  -n, --netrc          retrieve the username + password from the .netrc file. In lieu of -u/-p
  -o, --org=ARG        the Apigee organization.
  -Z, --ssoZone=ARG    specify the SSO zone to use when authenticating.
      --ssoUrl=ARG     specify the SSO url to use when authenticating.
  -C, --passcode=ARG   specify the passcode to use when authenticating.
  -J, --keyfile=ARG    the keyfile for a service account, for use with apigee.googleapis.com.
      --token=ARG      use this explicitly-provided oauth token.
      --apigeex        use apigee.googleapis.com for the mgmtserver.
  -T, --notoken        do not try to obtain an oauth token.
  -N, --forcenew       force obtain a new oauth token.
  -v, --verbose
  -h, --help
  -P, --prior          optional. use the prior (N-1) year or month. Default: the current year/month.
  -d, --daily          optional. collect daily data for the period. Default: collect monthly data.
  -S, --sheet          optional. create a Google Sheet with the data. Default: emit .csv file.
      --start=ARG      optional. a starting date in YYYYMMDD or YYYYMM format. Analyzes data until "now". Supercedes -P.
      --through_eom    optional. use with "start", analyzes until end of the month of "start". Default: until "now".
      --nocache        optional. do not use cached data; retrieve from stats API

This tool works by invoking the Apigee /stats APIs, and to do that, it requires authentication as an Apigee administrator or operator, with the proper authorization to do that.

Creating a Spreadsheet with the Data

Previously the tool supported a -S option, which told it to create a Google sheet. The description of this now-unsupported feature is here:

If you ask for a Google sheet as output (the -S option), then the tool will also connect to Google sheets APIs. The tool will require a separate authentication for that. In this case, when Google cloud requests authentication to enable access to the Google sheets API, it will also prompt you for your consent, to allow the tool to create a sheet on your behalf. You need to grant consent to get the sheet.

Because of changes in the OAuth requirements for Google APIs, the -S option is not currently working. To allow it to work, I would need to publish this tool, and get it approved for wide distribution. Sadly, that's not something I can do at this time.

BUT, this shouldn't be a problem. You can just generate the .CSV file, and import it into your own sheet manually, and you get the same experience. The -S option just automated that experience.

One drawback is, you'd need to create your own charts within the sheet to visualize it. I'll work on a solution using AppsScript that you can embed in your own sheet, to get the chart easily. (As of November 2023, this feature is in the backlog.)

Usage Example 1

These four examples all generate a CSV file that summarizes the traffic volume data for the current year, for an organization. They differ only in how they authenticate to Apigee.

In the first case, the user provides a token that has been obtained separately, perhaps via the Apigee get_token tool.

In case 2, the script will prompt the user for an administrative password to Apigee in order to query the Admin API.

In case 3, the script will use the credentials in the .netrc file for api.enterprise.apigee.com. No need to prompt for password.

In case 4, In the third case, the tool will exchange the username and the passcode for a token; this works for organizations that require multi-factor authentication (MFA).

In all cases, the user will separately be prompted to authenticate to Google, in order to grant consent to the "API Traffic Summarizer" app to generate a spreadsheet. Though the scope allows the tool to create and view sheets, the tool merely creates a new sheet. It does not read any existing sheets stored in Google drive. You need to grant consent once, the first time you run the tool.

The tool will perform several queries to the /stats API for Apigee, then with the resulting data, will create a spreadsheet with 2 sheets and 2 charts; one sheet will list the "per API Proxy" traffic volumes, and one will summarize the traffic by environment. Then 2 charts corresponding to the data in those sheets.

Sheet1

Chart1

This can take a long time to run, if there's lots of data. You may want to use the -v option to see verbose output, to monitor the progress.

Usage Example 2

Generate a Google sheets document that summarizes the monthly traffic volume data since March 2023, until now, for an organization.

node ./trafficByApiSummarizer.js -n -v -o my-org-name  -S --start 202203

Usage Example 3

Generate a .csv file that summarizes the traffic volume data for the current year, for an Apigee organization.

 node ./trafficByApiSummarizer.js -n -o my-org-name

When you invoke the program without the -S option, a .csv file is emitted, and no Google sheets document is created. The .csv file includes the raw "per API proxy" data. It does not include a rollup count of API requests "per environment". Again, this can take a long time to run.

Usage Example 4

Generate a .csv file that summarizes the traffic volume data, by month, for the prior year (-P) for an Apigee organization.

 node ./trafficByApiSummarizer.js -n -o my-org-name -P

Usage Example 5

Generate a google sheet that summarizes the traffic volume data, by day, for the prior month, for an organization.

 node ./trafficByApiSummarizer.js -n -o my-org-name -d -P -S

Usage Example 5

Generate a CSV that summarizes the traffic volume data, by day, for a specific month, for an organization.

 node ./trafficByApiSummarizer.js -n -o my-org-name --daily --start 20220301 --through_eom

How it works

The tool just layers on top of the documented Apigee Edge /stats API, which is documented here and here.

An example of a call that this script sends out to Apigee:

GET https://api.enterprise.apigee.com/v1/organizations/ORG/environments/ENV/stats/apis?select=sum(message_count)&timeUnit=month&timeRange=11/01/2023 07:00~11/30/2023 07:59

Bugs