feat: Adds support for -server mode and concurrently downloading dashboards

vanugrah commented 3 months ago

This is a useful script. However it is a pain to manually re-run the script to keep track of a migration. This PR:

Adds an HTTP Server which returns the list of angular dashboard as a JSON response on the /output route.
Adds a flag to configure the server port (default 8080).
Setups up a background go routine to update on a specified interval (default 10m).
Adds a flag to specify custom refresh interval.
Moves flag parsing into a new package.
Extends LeveledLogger with an ErrorLogger.
Adds readiness probe since detections can take 5+ minutes to run.
Adds signal handling.
Adds support for concurrently download dashboards to speed up detection runs (default 5)
General tidying.

This can now be hooked up directly to Grafana via the infinity data source plugin for live migration tracking.

Usage:

detect-angular-dashboards -server -interval 5m -max-concurrency 10 https://<grafana_instance>/api

Note: There was a strange bug with using JSONOutputter.Output to filter the angular dashboards which resulted in a increasing number of duplicates with each request to the /output endpoint. My suspicion is its something to do with in place filtering of the Dashboards slice. I eventually decided to write my own version which seems to be working as expected filterAngularDashboards.

CLAassistant commented 3 months ago

All committers have signed the CLA.

xnyo commented 2 months ago

I will also try to debug the issue with JSONOutputter.Output to filter the angular dashboards that's causing some duplicates after each request :slightly_smiling_face:

vanugrah commented 2 months ago

Hey @xnyo - thanks a lot for the review. I have incorporated the majority of your feedback and merged in your PR with the changes to still enable the CLI mode.

Only net new changes are a runServer method which sets up the HTTP server with signal handling. As far as I can tell http.ListenAndServe actually does not setup signal handlers appropriately.

Can you please take another look?

vanugrah commented 2 months ago

Hey @xnyo - Agree with all of your suggestions. I've updated the PR accordingly.

vanugrah commented 2 months ago

Done - sorry I have not been looking at this PR very closely. Thanks for your thorough review.

grafana / detect-angular-dashboards

feat: Adds support for -server mode and concurrently downloading dashboards #21