search

sambaiz / puppeteer-lambda-starter-kit

Starter Kit for running Headless-Chrome by Puppeteer on AWS Lambda.
MIT License
580 stars 82 forks source link

puppeteer v1.0.0 #11

Closed okiash closed 6 years ago

okiash commented 6 years ago

Thanks for this great project! Can you guys update it to support puppeteer v1.0.0?

Thank you

luminous8 commented 6 years ago

@okiash It works fine with puppeteer v.1.0.0. Did you try to update it manually?

Kikobeats commented 6 years ago

@luminous8 do you update the chrome binary? what version are you using?

I'm testing with dev-headless-chromium-65.0.3325.31-amazonlinux-2017-03.zip from https://github.com/adieuadieu/serverless-chrome/releases but it doesn't work as expected

.cc @Praveena0989

luminous8 commented 6 years ago

@Kikobeats I updated puppeteer in the package.json + used a stable release from serverless-chrome (don't remember which one) and it works on lambda

danielsantiago commented 6 years ago

I try updating puppeteer in the package.json to version 1.0.0 and use "HeadlessChrome-65.0.3325.31" without success. I got the following error: UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Failed to launch chrome! spawn /tmp/headless_shell ENOENT

luminous8 commented 6 years ago

I used the version with chrome hosted on s3, maybe it change something.

danielsantiago commented 6 years ago

The problem what that the compressed files from serverless-chrome has the file named "headless-chromium", and puppeteer-lambda-starter-kit expect it to be "headless_shell".

My problem now is that all the pages that I load return an empty content:

{
  "html": "<html><head></head><body></body></html>",
  "responses": {},
  "cookies": ""
}

This is the log in debug mode on:

START RequestId: fec71432-0ab8-11e8-aead-3f8b84075d9f Version: $LATEST
2018-02-05T21:10:33.474Z    fec71432-0ab8-11e8-aead-3f8b84075d9f    setup s3 chrome
2018-02-05T21:10:37.645Z    fec71432-0ab8-11e8-aead-3f8b84075d9f    setup done
[0205/211037.734944:WARNING:resource_bundle.cc(368)] locale_file_path.empty() for locale 
prctl(PR_SET_NO_NEW_PRIVS) failed
[0205/211037.742357:WARNING:discardable_shared_memory_manager.cc(186)] Less than 64MB of free space in temporary directory for shared memory files: 0
[0205/211037.759626:ERROR:gpu_process_transport_factory.cc(1019)] Lost UI shared context.

DevTools listening on ws://127.0.0.1:32873/devtools/browser/e3ceb6a6-ea8d-46f6-aa39-2306315c0313
[0205/211037.781979:WARNING:histograms.cc(40)] Started multiple compositor clients (Browser, Renderer) in one process. Some metrics will be disabled.
Fontconfig warning: "/etc/fonts/conf.d/30-metric-aliases.conf", line 84: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/30-metric-aliases.conf", line 84: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/30-metric-aliases.conf", line 93: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/30-metric-aliases.conf", line 93: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/30-metric-aliases.conf", line 102: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/30-metric-aliases.conf", line 102: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 33: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 59: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 68: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 68: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 79: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 79: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 79: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/40-nonlatin.conf", line 88: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 23: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 23: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 23: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 23: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 23: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 23: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 23: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 23: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 39: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 39: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 39: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 39: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 39: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 39: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 39: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 39: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 39: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 56: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 56: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 56: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 56: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 56: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 56: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 56: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 56: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 56: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 56: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 66: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 66: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 66: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 75: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/conf.d/45-latin.conf", line 75: Having multiple <family> in <alias> isn't supported and may not work as expected
Fontconfig warning: "/etc/fonts/fonts.conf", line 142: blank doesn't take any effect anymore. please remove it from your fonts.conf
2018-02-05T21:10:39.579Z    fec71432-0ab8-11e8-aead-3f8b84075d9f    launch done: HeadlessChrome/65.0.3325.31
2018-02-05T21:10:39.643Z    fec71432-0ab8-11e8-aead-3f8b84075d9f    http://google.com
[0205/211039.904387:WARNING:resource_bundle.cc(586)] locale resources are not loaded
[0205/211039.904430:WARNING:resource_bundle.cc(586)] locale resources are not loaded
[0205/211039.904444:WARNING:resource_bundle.cc(586)] locale resources are not loaded
[0205/211039.904458:WARNING:resource_bundle.cc(586)] locale resources are not loaded
[0205/211039.904469:WARNING:resource_bundle.cc(586)] locale resources are not loaded
[0205/211039.904479:WARNING:resource_bundle.cc(586)] locale resources are not loaded
[0205/211039.904490:WARNING:resource_bundle.cc(586)] locale resources are not loaded
[0205/211039.904501:WARNING:resource_bundle.cc(586)] locale resources are not loaded
[0205/211039.904511:WARNING:resource_bundle.cc(586)] locale resources are not loaded
[0205/211039.904520:WARNING:resource_bundle.cc(586)] locale resources are not loaded
2018-02-05T21:10:40.887Z    fec71432-0ab8-11e8-aead-3f8b84075d9f    GOTO done
END RequestId: fec71432-0ab8-11e8-aead-3f8b84075d9f
REPORT RequestId: fec71432-0ab8-11e8-aead-3f8b84075d9f  Duration: 7456.88 ms    Billed Duration: 7500 ms Memory Size: 1024 MB   Max Memory Used: 246 MB

I notice an error: "ERROR:gpu_process_transport_factory.cc(1019)] Lost UI shared context."

danielsantiago commented 6 years ago

@luminous8 can you please tell us exactly which version of chromium-65 is working for you? Thanks

@Kikobeats you were able to run the last version of puppeteer on Lambda?

Kikobeats commented 6 years ago

nope, I'm in the same point than you :(

danielsantiago commented 6 years ago

I made a working fork with Puppeteer v1.0.0 and Chrome 66.0.3343.0: https://github.com/danielsantiago/puppeteer-lambda-starter-kit

We can't use the builds from "serverless-chrome". I build my own using "https://github.com/danielsantiago/puppeteer-lambda-starter-kit/blob/master/chrome/buildChrome.sh" on a AWS EC2 machine.

.cc @Kikobeats

Kikobeats commented 6 years ago

@danielsantiago oh, can you share the binary? 🙏

danielsantiago commented 6 years ago

@Kikobeats the binary is on the repository: https://github.com/danielsantiago/puppeteer-lambda-starter-kit/blob/master/chrome/HeadlessChrome-66.0.3343.0.tar.gz

Kikobeats commented 6 years ago

@danielsantiago not sure why but the file size is 50MB higher than the previous version 🤔

Kikobeats commented 6 years ago

I think this script it a bit sophisticated

https://github.com/adieuadieu/serverless-chrome/blob/master/packages/lambda/builds/chromium/build/build.sh

unfortunately I don't have am EC2 to test it and compare both sizes, I will appreciate if somebody do that, the file size of your fork make impossible deploy it without compress and this is a little annoying.

sambaiz commented 6 years ago

I updated to support puppeteer v1.1.1 (latest) and chromium binary.