Increase speed of compilation handshake for large Gistbooks

[jamesplease]

Codepen:

Type	Time
boomerang	112ms
page request	111ms
total	223ms

Gistbook:

Type	Time
boomerang	903ms
page request	418ms
total	1.321s

6x slower! Why? Is it Cloudflare? Or just my Express app?

Here are the results from a local compilation:

Type	Time
boomerang	290ms
page request	91ms
total	381ms

[jamesplease]

I turned off the Cloudflare page rules, which did not decrease the time. This effectively rules out Cloudflare from the equation.

The page request time actually makes sense; it's justified by the size of the requests. The returned npm packages are pretty darn big. mathjs alone returns a 2mb bundle. And this file needs to be downloaded from the server.

Type	Time
Waiting	57ms
Downloading	353ms

On the other hand, the latency is 1s for the compile endpoint. Why is this?

I ran a request without any modules whatsoever, and the results were pretty awesome:

Type	Time
boomerang	32ms
page request	24ms

So as long as I don't include any require statements then I am way faster than Codepen. But why does the request delay for 1s when there are packages?

[jamesplease]

Requirebin uses a unique pattern that prevents a literal http request. It loads the src from a data-uri.

Chris Coyier once tweeted to me the reason why the boomerang pattern exists in the first place. This is what he said:

Turns out its kinda complicated. One reason is if you just inject new source, resources requested don’t send any header info.

Does Requirebin's pattern solve for this problem? What other problems might there be?

[jamesplease]

My guess is that this 1s is the time it takes for jquery to serialize 2mb of javascript into a string for the http request?

To test this, I should make a series of progressively larger requests to see if the wait time increases with request body size.

[jasonLaster]

Interesting problem.

[jamesplease]

Here are the results:

content-length	Latency (ms)
267	210
1574	57
34599	112
169971	178
473275	290
1260996	687
2318578	1020
3870835	1860

While the data set isn't as large as it could be, this gives me confidence that there is likely a linear relationship between request size and latency. Now to figure out how to combat this!

The first idea is deduping when possible, but these tests were loading a handful of libraries without any deduping. I'm really hoping that the data-uri solution resolves all of this.

What I need to do is:

1. Reproduce the problem Chris Coyier described. Inject HTML into an iframe and observe the headers
2. Create the same situation using the boomerang pattern. Do I get the correct headers?
3. Create the same situation using the data-uri pattern. What are the headers like now?

If the headers are there, then perhaps I should shoot Chris another msg to see if he has considered using data-uri.