quantified-uncertainty / metaforecast

Fetch forecasts from prediction markets/forecasting platforms to make them searchable. Integrate these forecasts into other services.
https://metaforecast.org/
MIT License
57 stars 5 forks source link

Switch to Vercel #52

Closed berekuk closed 2 years ago

berekuk commented 2 years ago

So, I was still annoyed by #44 and looked further, and it turns out that, uh, Netlify is just slow.

I spinned up a test instance on Vercel to compare, and the difference is huge.

Frontpage from my server in Europe:

# ab -n 30 'https://metaforecast.vercel.app/'
[...]

Percentage of the requests served within a certain time (ms)
  50%    334
  66%    367
  75%    454
  80%    520
  90%    617
  95%    797
  98%    823
  99%    823
 100%    823 (longest request)
# ab -n 30 'https://metaforecast.org/'
[...]

Percentage of the requests served within a certain time (ms)
  50%   1369
  66%   1396
  75%   1400
  80%   1429
  90%   1516
  95%   1521
  98%   1522
  99%   1522
 100%   1522 (longest request)

Frontpage from a temporary DO droplet in NYC1:

#  ab -n 30 'https://metaforecast.vercel.app/'
[...]

Percentage of the requests served within a certain time (ms)
  50%    167
  66%    183
  75%    209
  80%    238
  90%    265
  95%    332
  98%   4294
  99%   4294
 100%   4294 (longest request)

(4294ms response is an outlier, I left this run as an example, but most of my runs with -n 30 give ~300 ms longest request time)

# ab -n 30 'https://metaforecast.org/'
[...]

Percentage of the requests served within a certain time (ms)
  50%    683
  66%    700
  75%    720
  80%    733
  90%    815
  95%    897
  98%    993
  99%    993
 100%    993 (longest request)

Static pages, e.g. /about (from NYC1):

# ab -n 30 'https://metaforecast.vercel.app/about'
[...]

Percentage of the requests served within a certain time (ms)
  50%     42
  66%     43
  75%     45
  80%     51
  90%     62
  95%     75
  98%    115
  99%    115
 100%    115 (longest request)
# ab -n 30 'https://metaforecast.org/about'
[...]

Percentage of the requests served within a certain time (ms)
  50%    240
  66%    241
  75%    241
  80%    241
  90%    241
  95%    243
  98%    322
  99%    322
 100%    322 (longest request)

I don't know why Netlify is so slow, but I believe it's worth switching from.

Vercel is a bit costlier ($20 per team member vs $15/$19 on Netlify), but it has (mostly) better limits:

There are some other limits, it's hard to compare all of them, see https://vercel.com/pricing vs https://www.netlify.com/pricing, but I don't see anything that would be a problem until metaforecast grows 10x or more.

NunoSempere commented 2 years ago

Talked with Ozzie, makes sense. I sent you an invite to Vercel

berekuk commented 2 years ago

Cool, I'll set it up in a day or two then.