kubosuke / url_shortener

0 stars 0 forks source link

A real-time dashboard displaying usage metrics for each shortened URL. #1

Closed kubosuke closed 9 months ago

kubosuke commented 9 months ago

Goal

Plan1

Count how many times they visit with Redis, then show with LiveView and Chart.js => poor and I/O causes bad performance, don't want to go this way if possible

Plan2

Use BetterStack to visualize logs

image

it'd be nice for each list of URLs to have its usage report dashboard, that contains source geoIP, UA, etc

Plan3

Push metrics into ELK and visualize in Kibana

Plan4

Put CDN in front of fly.io and let it visualize metrics (is there such a CDN service?)

kubosuke commented 9 months ago

image

fastly: able to visualize metrics but not per /goto/ paths

kubosuke commented 9 months ago

If we could deploy workloads to Google Cloud we can do:

  1. expose logs to Cloud Logging
  2. integrate logs with BigQuery
  3. query logs (source IP), convert into country, and visualize https://cloud.google.com/blog/products/data-analytics/geolocation-with-bigquery-de-identify-76-million-ip-addresses-in-20-seconds

but it's too much and in this assignment we need to use fly.io so cannot go with this way

kubosuke commented 9 months ago

reinventing wheels but realistic approach:

image
kubosuke commented 9 months ago

sample sample

how to map ip => continent:

  1. Use https://github.com/navinpeiris/geoip to convert ip to country code
  2. convert country code to continent name by https://gist.github.com/stevewithington/20a69c0b6d2ff846ea5d35e5fc47f26c
  3. store the number of access with key: ${hash}:${continent}
kubosuke commented 9 months ago

Closed by https://github.com/kubosuke/url_shortener/pull/3