User representative synthetic transactions

[hollyallen]

Monitoring/alerting systems like pingdom work best when they have a set of synthetic transactions to run periodically that mimic real user behavior. This ticket is to define and create those synthetic transactions.

[ultrasaurus]

We should consider something that does this on the dev server too

[abisker]

@yozlet since we're going to NR now, is this ticket moot, and if so can you close?

[yozlet]

I think the ticket's still valid, just with a different source. NR has a Synthetics product for this. I'm going to rename the ticket.

[yozlet]

I've registered a couple of simple Synthetics monitors which load a URL and test for content in that URL:

• CS Front Page - loads https://collegescorecard.ed.gov/ and looks for the text No repayment needed
• CS Berkeley-Woodland Test - loads https://collegescorecard.ed.gov/search/?name=Berkeley&sort=advantage:desc and looks for the text Woodland Park. (This will test both the web app and its connection to api.data.gov, though depending on caching layers it won't necessarily test that the CS API is up. Fortunately, we have other monitors for that.)

The monitors are attached to the CS Web App alert policy, which will both post to #collegescorecard on Slack and alert @abisker, @sharms, @ultrasaurus, @shawnbot & me through the NR mobile app (assuming we all have it installed and our devices are connected to a network).

One of the nice features about Synthetics is that it also collects and graphs latency data, so you can see how long the pageload is from the various test points (two in the US, one in Ireland). Here's the CS Front Page test result graph.

NR is telling me that we - that is, 18F - aren't currently paying for these Synthetics features, so we have them on a 14-day trial. Let's see how we do with these over the next few days.

[yozlet]

BTW, the two monitors I've added so far aren't fully synthetic - they're just loading a page and looking for text. We can add further monitors that explicitly interact with the page; however, due to the relatively simple nature of the CS app, it's unlikely that it'd turn up any problems not caught by other systems.