meltano / hub

The single source of truth for all Meltano plugins, including all available Singer Taps and Targets: https://hub.meltano.com
https://hub.meltano.com
50 stars 68 forks source link

πŸ› BUG: very slow load time (10mins) from RSA πŸ‡ΏπŸ‡¦ #1069

Closed kgpayne closed 1 year ago

kgpayne commented 1 year ago

Note: This issue template is for issues related to the MeltanoHub or plugin definitions. If you're having a problem with a tap or target, please open an issue on that repository so the maintainers can address it.

Describe the bug Loading hub.meltano.com with a clear browser cache takes 10.1 minutes from South Africa.

Expected behavior Ideally CDN coverage would support faster load times than that. Other regions experience millisecond load times.

Screenshots

Screenshot 2022-12-06 at 20 23 04 Screenshot 2022-12-06 at 20 26 17

Desktop (please complete the following information if applicable):

kgpayne commented 1 year ago

Actually my first screenshot was premature πŸ˜… It took 8.8 minutes for DOMContentLoaded but some files took as long as 13.9 minutes. Top offenders are:

https://hub.meltano.com/assets/js/app.e09f6ba4.js at 1.3mb https://hub.meltano.com/assets/js/page--src--templates--plugins-vue.1ced4414.js at 1.2mb https://hub.meltano.com/assets/js/page--src--pages--extractors-vue~page--src--pages--files-vue~page--src--pages--loaders-vue~page--src~e555b7da.6477f79a.js at 1.2mb

Screenshot 2022-12-06 at 20 36 41
tayloramurphy commented 1 year ago

@kgpayne holy crap! That's awful. Is there something we can adjust in Netlify to help with this @aaronsteers ?

I'm unassigning Pat b/c I don't think there's anything he or I can do to help 😬

aaronsteers commented 1 year ago

@kgpayne - I can't infer much from the above screenshots in terms of root cause. I think it's important to distinguish time to 'fully load' the page/site, versus time to get responsiveness.

Gridsome is designed to progressively download first low-res images and then gradually higher and higher resolution images. That said, in the meanwhile while higher resolution images are downloading, there should still be an earlier-available ability to perform basic operations on the site.

We have not put any checks or limits on the logo sizes, but it could be worth reviewing the largest logos and downgrading quality for any that are outside an acceptable size limit. That said, I don't see any very large logos. Lots of them are about half an MB, which is possibly larger than they would need to be if we wanted to tune them for performance.

https://gridsome.org/docs/images/

Details ```txt aj@ajs-macbook-pro hub % git ls-files | xargs ls -l | sort -nrk5 | head -n 30 -rw-r--r-- 1 aj staff 2646861 Oct 12 22:48 static/assets/videos/tap-gitlab/personal-access-token.mov -rw-r--r-- 1 aj staff 637251 Oct 19 21:26 static/assets/logos/extractors/dagster.png -rw-r--r-- 1 aj staff 556818 Oct 12 22:48 package-lock.json -rw-r--r-- 1 aj staff 501054 Oct 12 22:48 yarn.lock -rw-r--r-- 1 aj staff 474231 Oct 12 22:48 static/assets/logos/extractors/sleeper.png -rw-r--r-- 1 aj staff 456818 Dec 6 21:50 _data/variant_metrics.yml -rw-r--r-- 1 aj staff 415260 Oct 12 22:48 static/assets/images/tap-shopify/private-app-api-password.png -rw-r--r-- 1 aj staff 308376 Oct 12 22:48 static/assets/logos/extractors/pivotal-tracker.png -rw-r--r-- 1 aj staff 287606 Oct 12 22:48 src/assets/images/meltano-background.jpeg -rw-r--r-- 1 aj staff 276399 Oct 12 22:48 static/assets/images/tap-facebook/access-token-debugger.png -rw-r--r-- 1 aj staff 255875 Oct 12 22:48 static/assets/logos/extractors/rockgympro.png -rw-r--r-- 1 aj staff 238231 Oct 12 22:48 static/assets/images/tap-google-analytics/02-ga-service-account-configuration-new-account.png -rw-r--r-- 1 aj staff 201126 Oct 19 21:26 static/assets/logos/extractors/nhl.png -rw-r--r-- 1 aj staff 192488 Oct 12 22:48 static/assets/logos/extractors/frontapp.png -rw-r--r-- 1 aj staff 190240 Oct 12 22:48 static/assets/logos/utilities/superset.png -rw-r--r-- 1 aj staff 190240 Oct 12 22:48 static/assets/logos/files/superset.png -rw-r--r-- 1 aj staff 185577 Oct 12 22:48 static/assets/logos/extractors/workday-raas.png -rw-r--r-- 1 aj staff 184858 Dec 6 21:50 static/assets/logos/extractors/shortcut.png -rw-r--r-- 1 aj staff 184556 Oct 12 22:48 static/assets/images/tap-google-analytics/account-selector-open.png -rw-r--r-- 1 aj staff 182076 Oct 12 22:48 static/assets/logos/extractors/twilio.png -rw-r--r-- 1 aj staff 171701 Oct 12 22:48 static/assets/logos/extractors/platformpurple.png -rw-r--r-- 1 aj staff 170106 Oct 12 22:48 static/assets/logos/extractors/instagram.png -rw-r--r-- 1 aj staff 169083 Oct 12 22:48 static/assets/images/tap-google-analytics/02-ga-service-account-configuration.png -rw-r--r-- 1 aj staff 163096 Oct 19 21:26 static/assets/logos/extractors/trustrace.png -rw-r--r-- 1 aj staff 160907 Oct 12 22:48 static/assets/logos/loaders/apprise.png -rw-r--r-- 1 aj staff 159925 Oct 12 22:48 static/assets/images/tap-google-analytics/01-ga-account-explorer.png -rw-r--r-- 1 aj staff 158248 Oct 12 22:48 static/assets/logos/extractors/ringcentral.png -rw-r--r-- 1 aj staff 157807 Oct 12 22:48 static/assets/logos/extractors/ireckonu.png -rw-r--r-- 1 aj staff 156812 Oct 12 22:48 static/assets/logos/extractors/webcrawl.png -rw-r--r-- 1 aj staff 154612 Oct 12 22:48 static/assets/images/tap-google-analytics/04-ga-reporting-api.png aj@ajs-macbook-pro hub % ```

Regarding network optimizations with Netlify, I can't speak to our options here but others might have ideas/options.

cc @rwfeather, @cjohnhanson

kgpayne commented 1 year ago

I think it's important to distinguish time to 'fully load' the page/site, versus time to get responsiveness.

Responsiveness was representatively awful πŸ˜… Search wasn't available until the full page load. It seems much better today (with browser cache disabled), so maybe I have triggered a CDN POP cache somewhere local to me. I'd be interested to know if Netlify gives us region page-view data, and region latencies stats too πŸ€”

As it seems to have resolved itself (full load in 23.40s), I'd be happy to close this issue as a transient bug.

cjohnhanson commented 1 year ago

It seems much better today (with browser cache disabled), so maybe I have triggered a CDN POP cache somewhere local to me

My guess would be something like this. I poked around in Netlify a bit, we can get location page view data if we upgraded our plan but I didn't immediately see any configuration options that could help out here.

+1 to closing as a transient bug.