search

bcgov / DBC-APIM

DataBC OPEN API Services
Apache License 2.0
1 stars 0 forks source link

API Gateway Stats - CSV output #38

Closed BK01 closed 5 years ago

BK01 commented 7 years ago

key requirement - Scripted / scheduled CSV export Elastic 6.1

Investigate and describe a proposed solution to query the following metrics from Kong/Cassandra for each API using the DataBC API Gateway and record to a csv file in GOGS.

Run scripts overnight.

Total hits per API (per month)

If time permits, additional metrics can be added.

Challenge:

Multiple logs (each node), run script against each and produce one output (csv) for each node. Aggregation - via another script for the time being (perhaps Elasticsearch in future for aggregation and UI once new box with more storage available) - mid-Feb 2018?

BK01 commented 7 years ago

May be possible using same csv file as issue #39

BK01 commented 7 years ago

No longer possible as Kong/Cassandra log no longer includes virtual host in addition to an issue with parser.

BK01 commented 6 years ago

@ll911 to send @banders a sample log for parsing script test.

BK01 commented 6 years ago

@BK01 Investigate export from Elk to csv via python script launched in Jenkins

BK01 commented 5 years ago

Analytics no longer offered within API Gateway.