website functional testing

[Mikanebu]

As a developer I want to have whole system under test so that whenever I make new changes I am sure that nothing else is broken.

Acceptance criteria

• [x] Show case pages work ok: finance-vix or co2-ppm [more later]
  • [x] Verify page loads - 200 OK and contains content you would expect
  • [x] Verify that pages load in a reasonable time (for now just record that time). Time to first byte, time to html loaded, time to all data loaded.
  • [x] Verify that download links work [works = 200]
  • [x] Verify that downloaded data makes sense [content is as expected, check zip files]
  • [x] Verify that views appear correctly [last of all as requires headless - maybe do headless to start with ...]
  • [x] Save outcome stats to a csv (so we have a history)

Tasks

• [x] read about chromless and puppeter and understand how they work
• [x] decide what aspects you need to test on the showcase page.
  • check showcase page returns 200
  • dataset title is there
  • page title is there
  • README is there
  • has some content
  • table is there
  • has some content
  • check resources are there
  • check all download links work fine(csv,json,zip)
  • check zip content
  • datapackage.json is there
  • check views
  • hover over graph results with some points
• [x] write simple script and assert expected results (response is 200, titles, readme)
• [x] refactor to handle more complicated case(table, resources, downloads, dp.json)
• [x] tests graphs, hovers, etc..
• [x] page loading time
  • [x] find out how to track page loading time
• [x] Output result in CSV
  • decide on column names
  • page_status [200 | statusCode + error.message]
  • total_load_time [xxx in seconds rounded until two decimal points]
  • first_byte_load_time [xxx in seconds rounded until two decimal points]
  • html_load_time [xxx in seconds rounded until two decimal points]
  • data_load_time [xxx in seconds rounded until two decimal points]
  • page title [ok | not ok]
  • dataset title [ok | not ok]
  • readme [ok | not ok]
  • resources [ok | resources missing]
  • download_links [ok | links missing]
  • csv_links [200 | statusCodes+error.message]
  • json_links [200 | statusCodes+error.message]
  • zip_links [200 | statusCodes+error.message]
  • zip_content
  • datapackage.json [ok | missing]
  • tables [ok | not ok]
  • graphs [ok | not ok]

Analysis

• How do we test stuff that involves JS?
  • Use headless chrome? e.g. see
  • https://github.com/GoogleChrome/puppeteer,
  • https://medium.com/@kensoh/chromeless-chrominator-chromy-navalia-lambdium-ghostjs-autogcd-ef34bcd26907
  • https://github.com/graphcool/chromeless

[Mikanebu]

Issue moved to datahq/pm #42 via ZenHub