record_wpr does not report a failure when a page fails to load.

[mythrialle]

I get a 404 page not found error when recording the following web pages: https://assembly.com/ https://www.coursera.org/ http://yapp.us http://mattermark.com

I was using python version 2.7.6. The record_wpr itself does not throw any error, it says it passed all the tests. Upgrading the python version to 2.7.13 fixes it. It is possibly related to this bug: https://github.com/chromium/web-page-replay/issues/73. It will be helpful if it throws an error or reports a failure instead of reporting it as a pass.

This is the command I use to record wpr: ./tools/perf/record_wpr --browser=release test_new_page_set

This is the test page set:

from telemetry.page import page as page_module
from telemetry.page import shared_page_state
from telemetry import story
URL_LIST = [
  'https://assembly.com/', # 404
  'https://www.coursera.org/', # 404
  'http://yapp.us', # 404
  'http://mattermark.com', # 404
]

class SimplePage(page_module.Page):
  def __init__(self, url, page_set, credentials='', name=''):
    super(SimplePage, self).__init__(
        url, page_set=page_set, name=name,
        credentials_path='data/credentials.json')
    self.credentials = credentials

  def RunPageInteractions(self, action_runner):
    # We wait for 20 seconds to make sure we capture enough information
    # to calculate the interactive time correctly.
    action_runner.Wait(20)

class TestNewPageSet(story.StorySet):
  """Sample pages """

  def __init__(self):
    super(TestNewPageSet, self).__init__(
      archive_data_file='data/test_new.json',
      cloud_storage_bucket=story.INTERNAL_BUCKET)
    for url in URL_LIST:
      self.AddStory(SimplePage(url, self))

[perezju]

That related bug mentions that an error should be shown if the Python version is too low.

@nedn any idea why the error does not propagate to Telemetry?