mrvirus9898
commented
5 years ago Est Time: 2 Hours Time Spent: 4 Hours
I ended up spending more time then estimated because I started enjoying the dashboard, and tinkering with various things. The picture below is the current Dashboard
Production Level Invocations Per Day Displays the number of times a Scraper has been called in a day. Should generally be 1. Non-1 numbers may indicate an error with the triggering mechanism. Note this does not include tests done on the lambda dashboard.
Total Entries Displays the total number of entries the Scraper has placed into the Database.
Throttles of All Scrapers A Line chart that displays any throttles, and how often AWS is throttling a given scraper, and when that throttle occurred. A change from 0 may indicate a poorly performing scraper.
Errors Thrown per Day Displays the total number of errors thrown by AWS per Scraper per day. One or two errors in normal as this would indicate a slow website, or a package that needed updating. More then then a few errors could indicate that a website has greatly changed, or that a package has broken.
Time to Write Total write time on the Database. Does not correlate to execution time, seen below. Abnormally high write time could indicate that the Database is being flooded with junk data.
Duration of Execution, Correlates to Cost of Execution Duration of Execution, Correlates to Cost of Execution displays how long a Lambda was active in AWS when it is tested or invoked. More time may indicate a higher end cost for the user, or may indicate that a website is responsive, but lagging.
The dashboard has been set up, but there are some particulars with what data is being displayed. Investigate and implement some quality of life improvements for the dashboard.
Please indicate the time spent on this, any issues that you are having, any good references you found for this subject, and credit anyone helped you out.