search

jspm / jspm.io

jspm.io website
https://jspm.io
25 stars 41 forks source link

Local mirror? #60

Open dakami opened 6 years ago

dakami commented 6 years ago

It's pretty easy to end up w/ a dependency you haven't already built, and you'll time out before cloudflare gives up on you:

wget -S -O - https://dev.jspm.io/plotly.js --2018-08-12 10:05:15-- https://dev.jspm.io/plotly.js Resolving dev.jspm.io (dev.jspm.io)... 104.18.62.242, 104.18.63.242, 2400:cb00:2048:1::6812:3ef2, ... Connecting to dev.jspm.io (dev.jspm.io)|104.18.62.242|:443... connected. HTTP request sent, awaiting response... HTTP/1.1 502 Bad Gateway Date: Sun, 12 Aug 2018 17:05:36 GMT Content-Type: text/html; charset=UTF-8 Content-Length: 4126 Connection: keep-alive Set-Cookie: __cfduid=d4b6efa3f1cd70be13275b5f7f2659ba21534093515; expires=Mon, 12-Aug-19 17:05:15 GMT; path=/; domain=.jspm.io; HttpOnly CF-Cache-Status: MISS Vary: Accept-Encoding Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct" Set-Cookie: cf_ob_info=502:449486996933933c:SJC; path=/; expires=Sun, 12-Aug-18 17:06:06 GMT Set-Cookie: cf_use_ob=443; path=/; expires=Sun, 12-Aug-18 17:06:06 GMT Expires: Thu, 01 Jan 1970 00:00:01 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Server: cloudflare CF-RAY: 449486996933933c-SJC 2018-08-12 10:05:36 ERROR 502: Bad Gateway.

I don't mind throwing up a local server, but unlike browserfy-cdn, there's no tips how to do that. I've actually gotten some pretty nice stuff going with your code, see:

https://codepen.io/dakami/pen/ejxabR?editors=1000

But it really only works for a handful of modules right now.

guybedford commented 6 years ago

One of those files, this one in fact - https://unpkg.com/plotly.js@1.39.4/src/components/legend/style.js is causing an overflow on the transforms and is actually crashing its transform process using up 1GB of memory, and hence your timeout (notice it does the same for unpkg!).

Will look into a better failure scenario here.

The software is currently not open source, and will only be open source when jspm 2.0 is release (this is the whole reason for the jspm CLI to provide workflow equivalence), but I'm open to providing paid licenses in the mean time to help sponsor the development further to get jspm 2.0 out the door.

guybedford commented 6 years ago

Ok, it turns out this is due to the explosive memory usage of Babel running 10s of transforms at the same time, and not any actual transform loop recursion issue.

I've upped the per-node memory and ensured the worker count is within safe memory limits for Babel.

If for example you try loading another version of Plotly it seems to work within the limits for me now (given that that version is previously fully uncached).

Please let me know if you notice anything odd, handling these recovery cases well is important.

guybedford commented 6 years ago

To explain the timing issues here further - when the out-of-memory crash happened, the task timeout takes 5 minutes to clear the task queue and restart the failed operation.

I'll be working on better handling for purging tasks whose transform process has crashed for any reason, which would avoid this as well.