Node-fetch vs puppeteer

HockeyDevs / mahl-web-scraper

An updated UI created by scraping data from MAHL so people can actually register.

0 stars 0 forks source link

Node-fetch vs puppeteer #4

Open MrMonsk opened 5 years ago

MrMonsk commented 5 years ago

both of these will return the HTML of a given URL. I'm not entirely sure how much of MAHL is generated w/ js, but it doesn't look like very much at all.
Don't merge yet, I think we should set up a basic express server for this as well

nvanselow commented 5 years ago

Have you tried to run puppeteer in a lambda? I have a feeling that won't work or will be tricky requiring us to switch to something like ECS to host the backend. I think even the headless version of Chromium isn't running in plain node. But I haven't looked at it in a while so I'm unsure.

nvanselow commented 5 years ago

Looks like there might be some solutions for running Chrome headless on a lambda. https://github.com/sambaiz/puppeteer-lambda-starter-kit

However, I think I still lean toward fetch until we discover we need the added complexity of puppeteer.

MrMonsk commented 5 years ago

Cool beans, will update this in appropriate directory as well