I2I: Cache network requests dispatched from tests

[rileyajones]

Summary

Network requests dispatched from tests both slow down test suites and cause instability. By routing these requests through a proxy server which caches the responses we can speed up test runs and reduce flakes.

Motivation

AMP developers spend a lot of time waiting for tests to run and are forced to repeat this process when flakes occur. amphtml Pipelines amphtml Jobs insights

Other options:

• Prevent new tests which use network requests from being added and then gradually removing the requests that exist.
• Update the hosts file to broadly redirect requests and stub them.

/cc @ampproject/wg-approvers

Design Doc

26754

[rileyajones]

This seems like a fairly straightforward solution to a problem that has affected a lot of us so I wouldn't be surprised if I am not the first person to consider such an idea. I'm very interested to know if anyone else has spent any time thinking about or has come up with a reason that this could not work.

[jridgewell]

Do you have an example of network requests being made?

This needs to be controlled at the Browser level, it can't be done in JS. Eg, we could have Karma spin up Chrome with --proxy-server: https://www.chromium.org/developers/design-documents/network-settings

[rileyajones]

The --proxy-server flag is a great suggestion. Most of the examples occur in the e2e tests and often take the form of the fixture loading remote assets, for example the amp-img tag in test/fixtures/e2e/amp-bind/bind-amp4email.html loads https://foo.com/foo.jpg and while that is not a real image, the runner will still wait for it to be loaded.

While it isn't a great metric for a variety of reasons, searching https within the tests directory returns 8128 results so it seems safe to say this is fairly widespread.

[samouri]

Shouldn't we be mocking the network out? This would be even faster than caching.

[jridgewell]

Yah, my first thought is to reject any request that isn't to a local dev server. But we could also cache the foreign requests into a local response blob, and read that from disk when serving (anything uncached would be rejected).

[rileyajones]

Mocking the network would be better, but there are so many existing requests that it seems nearly impossible to stub them all.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.