Puppeteer@14.1.0 - Githubissues

Sparticuz commented 2 years ago

This PR updates the build process for chromium and gets puppeteer support up to 14.1.0.

Closes: #240 #248 #258 #254 #242 #274 #275 #276

Notes:

Ignore the branch name, I originally created the PR for Puppeteer 13.5.0
I am not going to include the binaries in this PR. If you would like to use my binaries or my lambda layer, please see my latest comments
If you would like to use a scoped npm package that will be kept up to date until this PR is merged, check out @sparticuz/chrome-aws-lambda

Due to some changes in WebGL/SWAngle, this PR does not included the needed fixes for lambdafs. If you need WebGL, please use my scoped package.

deldrid1 commented 2 years ago

@Sparticuz - do you mind to commit your bin/chromium.br and bin/swiftshader.tar.br? I'd love to test this in my build environment with Typescript and webpack but don't have the bandwidth to spin up the compilation side at the moment...

Sparticuz commented 2 years ago

After cleaning everything up, I'm receiving the same WebGL error's that @alixaxel was getting. (I'm unsure why my initial compile seemed to work, but I haven't been able to reproduce that). It looks like in Chrome 96, swiftshader was dropped and replaced with swangle, so I'm doing some research into what needs to be done to switch to that. I even got vulkan working on my local box and got better performance, but I'm unsure if that would work on lambda or not.

Sparticuz commented 2 years ago

I believe I've found the files needed to be bundled for Chrome 96+.

libEGL.so
libGLESv2.so
libvk_swiftshader.so
libvulkan.so.1
vk_swiftshader_icd.json
(obviously headless_shell as chromium as well)

Notice that the swiftshader/libEGL.so and swiftshader/libGLESv2.so are not included.

Also the cli flags that needed to be changed are:

--use-gl=angle
--use-angle=swiftshader

With those files and those flags, chromium runs without error and an inspect shows a working webgl.

I'm running a compile with the latest ansible instructions in order to bundle the needed files. I won't commit the brotli files. I'll let @alixaxel run a build for security reasons, but I will post them so they can be tested. The compile takes about an hour. @alixaxel I also won't commit the changes I made to the ec2 instance, but I'll tell you I upgraded to a c6i.12xlarge.

After I test this out I'll push the latest changes as Ready for review

Sparticuz commented 2 years ago

~~https://raw.githubusercontent.com/Sparticuz/chrome-aws-lambda/61139ebc4a62f8a2576ad79be858e13204c010b4/bin/chromium.br~~

~~https://raw.githubusercontent.com/Sparticuz/chrome-aws-lambda/61139ebc4a62f8a2576ad79be858e13204c010b4/bin/swiftshader.tar.br~~

EDIT: If you use these, you'll need to edit chrome-aws-lambda's index file to include the flags in the previous comment.

Edit2: Please see my branch here for the latest binaries https://github.com/Sparticuz/chrome-aws-lambda/tree/13.5_bin/bin

Sparticuz commented 2 years ago

~~Here is a Lambda layer: https://raw.githubusercontent.com/Sparticuz/chrome-aws-lambda/b4584a6c65e1c33d7a1afc1edbb825ee4f2509a6/chrome_aws_lambda.zip~~

I've tested it with the SAM application in the _/amazon folder and it worked. I'll be testing it internally as well.

EDIT: Please see my branch here for the latest layer: https://github.com/Sparticuz/chrome-aws-lambda/blob/13.5_bin/chrome_aws_lambda.zip

deldrid1 commented 2 years ago

@Sparticuz - just to let you know, I've merged in with your _bin branch on top of my changes for webpack and so far so good! I'm doing some larger sized testing today, but nothing to complain about with the updates! (I don't require webgl so I can't speak to that, but basic functionality looks good!)

ibqn commented 2 years ago

@Sparticuz great contribution! I am looking forward to see this PR getting approved.

owlyowl commented 2 years ago

@Sparticuz thanks so much for getting this up and running it will be a huge help to the community!

Are there any particular lambda config changes that are required to support the new layer/version?

bestplay9384 commented 2 years ago

Hey @Sparticuz,

I'm testing Your compilation on AWS Lambda and with DEBUG enabled i got this on chromium boot.

I have chromium flags you suggested above. Any idea what can it be? :)

EDIT: flag that is not included in chrome-aws-lambda flags - --disable-software-rasterizer made errors disappear :) I thought it is an issue with webGL/driver changes you've made.

Sparticuz commented 2 years ago

It looks like it can't find libglesv2.so. are you using the zip image, or did you build and upload?

bestplay9384 commented 2 years ago

It looks like it can't find libglesv2.so. are you using the zip image, or did you build and upload?

I was using brotli compressed archives from Your fork :) Those You have linked above :) I have it all packed to lambda layer and then attached to lambda.

EDIT: I've just found out that my custom decompression code for lambda layer build extracts swiftshader libnraries to bin/swiftshader directory - not in bin itself. It is probably an issue - paths mismatch :)

EDIT 2: Yup, fixed paths, removed --disable-software-rasterizer flag and it works as it should :)

Sparticuz commented 2 years ago

I've updated my PR to be Puppeteer 13.6.0, and my _bin branch to have 13.6.0 as well as chromium 101. Layer here. I have not tested it

lewisl9029 commented 2 years ago

I gave the 13.6.0 layer a shot, and first few invocations seem to work fine (albeit quite a bit slower than the version on main for some reason), but afterwards I get errors that look like this when trying to take a screenshot:

2022-04-21T02:06:48.862Z pw:browser [pid=103][err] [0421/020648.861048:ERROR:shared_image_representation.cc(219)] Attempt to read from an uninitialized SharedImage. Initialized region: (0, 0, 0, 0) Size: (1280, 256)
2022-04-21T02:06:48.864Z pw:browser [pid=103][err] [0421/020648.864208:ERROR:shared_image_representation.cc(219)] Attempt to read from an uninitialized SharedImage. Initialized region: (0, 0, 0, 0) Size: (832, 192)
2022-04-21T02:06:48.864Z pw:browser [pid=103][err] [0421/020648.864616:ERROR:shared_image_representation.cc(219)] Attempt to read from an uninitialized SharedImage. Initialized region: (0, 0, 0, 0) Size: (1280, 64)
2022-04-21T02:06:48.930Z pw:browser [pid=103][err] [0421/020648.929945:ERROR:validation_errors.cc(117)] Invalid message: VALIDATION_ERROR_DESERIALIZATION_FAILED
2022-04-21T02:06:48.930Z pw:browser [pid=103][err] [0421/020648.929974:WARNING:network_service.cc(202)] Mojo error in NetworkService: Validation failed for viz.mojom.CopyOutputResultSender.0  [VALIDATION_ERROR_DESERIALIZATION_FAILED]
2022-04-21T02:06:48.930Z pw:browser [pid=103][err] [0421/020648.929983:ERROR:interface_endpoint_client.cc(664)] Message 500872390 rejected by interface viz.mojom.CopyOutputResultSender
2022-04-21T02:06:48.954Z pw:browser [pid=103][err] [0421/020648.954338:ERROR:shared_image_representation.cc(219)] Attempt to read from an uninitialized SharedImage. Initialized region: (0, 0, 0, 0) Size: (576, 192)
2022-04-21T02:06:48.955Z pw:browser [pid=103][err] [0421/020648.955260:ERROR:shared_image_manager.cc(227)] SharedImageManager::ProduceSkia: Trying to Produce a Skia representation from a non-existent mailbox.
2022-04-21T02:06:48.956Z pw:browser [pid=103][err] [0421/020648.956010:ERROR:shared_image_manager.cc(227)] SharedImageManager::ProduceSkia: Trying to Produce a Skia representation from a non-existent mailbox.
2022-04-21T02:06:48.957Z pw:browser [pid=103][err] [0421/020648.957278:ERROR:shared_image_manager.cc(227)] SharedImageManager::ProduceSkia: Trying to Produce a Skia representation from a non-existent mailbox.
2022-04-21T02:06:48.963Z pw:browser [pid=103][err] [0421/020648.963574:ERROR:shared_image_manager.cc(227)] SharedImageManager::ProduceSkia: Trying to Produce a Skia representation from a non-existent mailbox.
2022-04-21T02:06:48.964Z pw:browser [pid=103][err] [0421/020648.964137:ERROR:shared_image_representation.cc(219)] Attempt to read from an uninitialized SharedImage. Initialized region: (0, 0, 0, 0) Size: (1280, 64)
2022-04-21T02:06:49.026Z pw:browser [pid=103][err] [0421/020649.026090:ERROR:validation_errors.cc(117)] Invalid message: VALIDATION_ERROR_DESERIALIZATION_FAILED
2022-04-21T02:06:49.026Z pw:browser [pid=103][err] [0421/020649.026117:WARNING:network_service.cc(202)] Mojo error in NetworkService: Validation failed for viz.mojom.CopyOutputResultSender.0  [VALIDATION_ERROR_DESERIALIZATION_FAILED]
2022-04-21T02:06:49.026Z pw:browser [pid=103][err] [0421/020649.026127:ERROR:interface_endpoint_client.cc(664)] Message 500872390 rejected by interface viz.mojom.CopyOutputResultSender
2022-04-21T02:06:49.047Z    eb6fc018-3b22-4243-b9f3-743a206c7cdc    ERROR   Invoke Error    {"errorType":"Error","errorMessage":"page.screenshot: Protocol error (Page.captureScreenshot): Unable to capture screenshot\n=========================== logs ===========================\ntaking page screenshot\n============================================================","name":"Error","stack":["page.screenshot: Protocol error (Page.captureScreenshot): Unable to capture screenshot","=========================== logs ===========================","taking page screenshot","============================================================","    at /var/task/main.cjs:174:10","    at measure (/var/task/main.cjs:35:24)","    at Runtime.module.exports.handler (/var/task/main.cjs:173:28)"]}
END RequestId: eb6fc018-3b22-4243-b9f3-743a206c7cdc

FWIW, I'm using playwright instead of puppeteer, but curious if others had success with taking screenshots using this version (puppeteer or playwright)?

bestplay9384 commented 2 years ago

Hey @Sparticuz, I've got a chance to test quite much Your changes for 13.5.0/13.5.1. I'm using puppeteer 13.5.1 with chromium 100 (from Your fork). I'm doing around 100k invokes od lambda daily, doing work for me using puppeteer, none of these invokes failed because of chromium/puppeteer issues, so i'm pretty sure it works fine :) I'm just scraping websites so I'm not sure if any of webGL stuff had a chance to occur :/ Just givin' ya all info it works for my needs.

Sparticuz commented 2 years ago

I gave the 13.6.0 layer a shot, and first few invocations seem to work fine (albeit quite a bit slower than the version on main for some reason), but afterwards I get errors that look like this when trying to take a screenshot:

2022-04-21T02:06:48.862Z pw:browser [pid=103][err] [0421/020648.861048:ERROR:shared_image_representation.cc(219)] Attempt to read from an uninitialized SharedImage. Initialized region: (0, 0, 0, 0) Size: (1280, 256)
2022-04-21T02:06:48.864Z pw:browser [pid=103][err] [0421/020648.864208:ERROR:shared_image_representation.cc(219)] Attempt to read from an uninitialized SharedImage. Initialized region: (0, 0, 0, 0) Size: (832, 192)
2022-04-21T02:06:48.864Z pw:browser [pid=103][err] [0421/020648.864616:ERROR:shared_image_representation.cc(219)] Attempt to read from an uninitialized SharedImage. Initialized region: (0, 0, 0, 0) Size: (1280, 64)
2022-04-21T02:06:48.930Z pw:browser [pid=103][err] [0421/020648.929945:ERROR:validation_errors.cc(117)] Invalid message: VALIDATION_ERROR_DESERIALIZATION_FAILED
2022-04-21T02:06:48.930Z pw:browser [pid=103][err] [0421/020648.929974:WARNING:network_service.cc(202)] Mojo error in NetworkService: Validation failed for viz.mojom.CopyOutputResultSender.0  [VALIDATION_ERROR_DESERIALIZATION_FAILED]
2022-04-21T02:06:48.930Z pw:browser [pid=103][err] [0421/020648.929983:ERROR:interface_endpoint_client.cc(664)] Message 500872390 rejected by interface viz.mojom.CopyOutputResultSender
2022-04-21T02:06:48.954Z pw:browser [pid=103][err] [0421/020648.954338:ERROR:shared_image_representation.cc(219)] Attempt to read from an uninitialized SharedImage. Initialized region: (0, 0, 0, 0) Size: (576, 192)
2022-04-21T02:06:48.955Z pw:browser [pid=103][err] [0421/020648.955260:ERROR:shared_image_manager.cc(227)] SharedImageManager::ProduceSkia: Trying to Produce a Skia representation from a non-existent mailbox.
2022-04-21T02:06:48.956Z pw:browser [pid=103][err] [0421/020648.956010:ERROR:shared_image_manager.cc(227)] SharedImageManager::ProduceSkia: Trying to Produce a Skia representation from a non-existent mailbox.
2022-04-21T02:06:48.957Z pw:browser [pid=103][err] [0421/020648.957278:ERROR:shared_image_manager.cc(227)] SharedImageManager::ProduceSkia: Trying to Produce a Skia representation from a non-existent mailbox.
2022-04-21T02:06:48.963Z pw:browser [pid=103][err] [0421/020648.963574:ERROR:shared_image_manager.cc(227)] SharedImageManager::ProduceSkia: Trying to Produce a Skia representation from a non-existent mailbox.
2022-04-21T02:06:48.964Z pw:browser [pid=103][err] [0421/020648.964137:ERROR:shared_image_representation.cc(219)] Attempt to read from an uninitialized SharedImage. Initialized region: (0, 0, 0, 0) Size: (1280, 64)
2022-04-21T02:06:49.026Z pw:browser [pid=103][err] [0421/020649.026090:ERROR:validation_errors.cc(117)] Invalid message: VALIDATION_ERROR_DESERIALIZATION_FAILED
2022-04-21T02:06:49.026Z pw:browser [pid=103][err] [0421/020649.026117:WARNING:network_service.cc(202)] Mojo error in NetworkService: Validation failed for viz.mojom.CopyOutputResultSender.0  [VALIDATION_ERROR_DESERIALIZATION_FAILED]
2022-04-21T02:06:49.026Z pw:browser [pid=103][err] [0421/020649.026127:ERROR:interface_endpoint_client.cc(664)] Message 500872390 rejected by interface viz.mojom.CopyOutputResultSender
2022-04-21T02:06:49.047Z  eb6fc018-3b22-4243-b9f3-743a206c7cdc    ERROR   Invoke Error    {"errorType":"Error","errorMessage":"page.screenshot: Protocol error (Page.captureScreenshot): Unable to capture screenshot\n=========================== logs ===========================\ntaking page screenshot\n============================================================","name":"Error","stack":["page.screenshot: Protocol error (Page.captureScreenshot): Unable to capture screenshot","=========================== logs ===========================","taking page screenshot","============================================================","    at /var/task/main.cjs:174:10","    at measure (/var/task/main.cjs:35:24)","    at Runtime.module.exports.handler (/var/task/main.cjs:173:28)"]}
END RequestId: eb6fc018-3b22-4243-b9f3-743a206c7cdc

FWIW, I'm using playwright instead of puppeteer, but curious if others had success with taking screenshots using this version (puppeteer or playwright)?

Bfcache was disabled in puppeteer 13.6, Try that and see if you are getting the same results https://github.com/puppeteer/puppeteer/issues/8182

lewisl9029 commented 2 years ago

A few more updates:

I gave disabling bfcache a try, but unfortunately it didn't change anything. Also tried an earlier version of the layer at https://github.com/alixaxel/chrome-aws-lambda/pull/264#issuecomment-1064166840, and it had the same issue.

Afterwards, I tried migrating the script to puppeteer, which worked (albeit with a screenshot time of ~1s compared to ~200ms with playwright on main)! So this seems to be a playwright specific thing. Going to dig into playwright's default flags to see if there's anything I'm missing that might resolve this.

EDIT: Another update:

Figured it out! Apparently this was caused by my custom pre-inflating layer I was using with the version on main to improve cold starts. Seems like that approach no longer worked with this version for some reason.

Everything started working again once I reverted back to using the standard await chromeAwsLambda.executablePath approach with the layer you provided. Apologies for the false alarm, and thank you so much for working on this!

ferbs commented 2 years ago

I'm using a build based on this fork and it seems to work perfectly. Am running it in a container image based on public.ecr.aws/lambda/nodejs:14. Thank you @sparticuz!

Hard to know when this fork will get merged in, so in case anyone needs it, here are instructions for installing from the fork:

git clone https://github.com/Sparticuz/chrome-aws-lambda.git
# ^^^ note: almost 2GB. Maybe do a depth=1 clone of just the puppeteer@13.5.0 branch. (I don't remember that git command though) 

git checkout puppeteer@13.5.0
# ^^^ keep an eye for newer, in case it moves since this was posted. 

npm install
npm pack

# next, replace $your_branch below with actual path:
mv chrome-aws-lambda-13.6.0.tgz $your_branch && cd $your_branch 
npm i --save ./chrome-aws-lambda-13.6.0.tgz

Sparticuz commented 2 years ago

git checkout puppeteer@13.5.0
# ^^^ keep an eye for newer, in case it moves since this was posted.

I'll keep the branch the same for the purposes of this PR, but I'll update the PR title and comment when there is a new version of Puppeteer that requires a new build of chromium.

Ideally we can get this merged, then get some automated testing on it. I still don't know enough about the typescript that @alixaxel has written to be confident enough to say one way or the other if what I've done is production ready because of the lack of automated testing. There is only the one SAM test, but it's only testing that chromium takes a screenshot. It doesn't test any other functionality (webgl, etc...)

That said, a number of users ARE using it in production.

ferbs commented 2 years ago

@Sparticuz Maybe running the core puppeteer test suite against this build might be a good solution? From a skim of the code, it looks like puppeteer is using an integration test approach, hitting an actual browser for everything. It uses process.env.BINARY to set executablePath (in test/mocha-utils.ts) so plopping their entire repo into a Lambda and running BINARY=$chrome-aws-lambda npm run test might be a decent solution.

byF commented 2 years ago

@Sparticuz I'm getting following Chromium error for both 13.5 and 13.6 layers built by you. It doesn't crash the browser and its functionality seems to be fine (I've got a different problem with fonts, but I'm investigating if it's related or not EDIT: very likely not)

[0427/144243.098580:ERROR:egl_util.cc(74)] Failed to load GLES library: /tmp/libGLESv2.so: /tmp/libGLESv2.so: cannot open shared object file: No such file or directory
[0427/144243.104908:ERROR:gpu_channel_manager.cc(831)] ContextResult::kFatalFailure: Failed to create shared context for virtualization.
[0427/144243.104968:ERROR:shared_image_stub.cc(537)] SharedImageStub: unable to create context
[0427/144243.104976:ERROR:gpu_channel.cc(571)] GpuChannel: Failed to create SharedImageStub
[0427/144243.104999:ERROR:gpu_channel_manager.cc(831)] ContextResult::kFatalFailure: Failed to create shared context for virtualization.
[0427/144243.105004:ERROR:shared_image_stub.cc(537)] SharedImageStub: unable to create context
[0427/144243.105008:ERROR:gpu_channel.cc(571)] GpuChannel: Failed to create SharedImageStub

Sparticuz commented 2 years ago

[0427/144243.098580:ERROR:egl_util.cc(74)] Failed to load GLES library: /tmp/libGLESv2.so: /tmp/libGLESv2.so: cannot open shared object file: No such file or directory

Others have noted that this might happen when using custom decompression. It's basically failing on loading up the WebGL stuff, so if you aren't using webgl, you can ignore that.

Sparticuz commented 2 years ago

I've updated this PR to 13.7.0, and updated my '13.5_bin' branch to include the binaries and the layer.

brunolemos commented 2 years ago

is this on npm somewhere?

Sparticuz commented 2 years ago

I've been contemplating it, but it would be namespaced and deprecated after it's merged. Maybe I'll do that soon. @alixaxel, my preference would be to get this merged.

Sparticuz commented 2 years ago

I've updated this PR to 14.0.0, and updated my '13.5_bin' branch to include the binaries and the layer.

I think I have decided to go ahead and publish a namespaced npm package. If v14 proves to be good, I'll post it later this week.

Note that puppeteer 14 drops node 12 support and includes ESM package support. I didn't update any of the chrome-aws-linux typescript, only updated the chromium binary, so please post if there are issues, I won't have time to update my deployment from 13.7 until later this week or next, so I haven't tested 14 yet.

yakirza17 commented 2 years ago

Hi @Sparticuz, can you put the 14 binary files inside a new branch/tag called v14?

Thanks for your updates

otaviosoares commented 2 years ago

I've updated this PR to 14.0.0, and updated my '13.5_bin' branch to include the binaries and the layer.

I tried to use the layer version 14 but got the error Cannot find module '/opt/nodejs/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Browser'. could I be missing something? Thanks

terebentina commented 2 years ago

@Sparticuz using the latest layer you published I get this error:

error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory

My serverless layer is defined like this:

layers:
  HeadlessChrome:
    name: HeadlessChrome
    compatibleRuntimes:
      - nodejs16.x
    description: Required for headless chrome
    package:
      artifact: layer.zip

(layer.zip is your layer)

hmazter commented 2 years ago

@terebentina This is not compatible with nodjs16.x as far as I know. There is a PR for that: #274

terebentina commented 2 years ago

Right, I see. Tried with node 14 as well and now I also get the error reported above: Cannot find module '/opt/nodejs/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Browser'

Sparticuz commented 2 years ago

I'm also getting that error, I haven't had time to look into it yet. If anyone finds a solution let me know.

I'll pull in the node16 patch. I'm currently migrating to 16 as well.

otaviosoares commented 2 years ago

Adding the .js extension to the require seems to fix it.

Super = require('puppeteer-core/lib/cjs/puppeteer/common/Browser.js').Browser;

It's probably because puppeteer 14 added an exports block to its package.json.

Sparticuz commented 2 years ago

Adding the .js extension to the require seems to fix it.

Fantastic, I'll get this added asap.

terebentina commented 2 years ago

Sparticuz, the layer has chrome-aws-lambda installed but it should have @Sparticuz/chrome-aws-lambda

Sparticuz commented 2 years ago

Sorry, I was trying to change the branch name and I guess Github closed the PR.

Sparticuz commented 2 years ago

Sparticuz, the layer has chrome-aws-lambda installed but it should have @Sparticuz/chrome-aws-lambda

I'm going to keep this PR using chrome-aws-lambda, but I am prepping to do a scoped package.

Sparticuz commented 2 years ago

Here is a scoped npm package for chrome-aws-lambda. @sparticuz/chrome-aws-lambda

I will continue to keep this PR update to date, but will probably not keep the layer up to date as it can be built via my scoped package.

tomoima525 commented 2 years ago

Thank you @Sparticuz it's working with Node v16 environment on AWS Lambda!

Antoine-C commented 2 years ago

Thanks for the hard work @Sparticuz, I'm wondering if anyone can access https://get.webgl.org/ with a working "spinning cube" ? On my end with your binaries I only have the following error message "Hmm. While your browser seems to support WebGL, it is disabled or unavailable. If possible, please ensure that you are running the latest drivers for your video card."

With https://browserleaks.com/webgl I can see, for the WebGL Support Detection, the following :

This browser supports WebGL : False (supported, but disabled in browser settings, or blocked by extensions).
This browser supports WebGL 2 : False (supported, but disabled in browser settings, or blocked by extensions).

I've been looking at https://peter.sh/experiments/chromium-command-line-switches/ but can't see an option that could cause this. I've set --disable-webgl=false , --disable-webgl-image-chromium=false and --disable-webgl2=false for the chromium.args but without luck.

I know WebGL is rendering when using the --use-gl=swiftshader but as soon as I swtich to --use-gl=andle and --use-angle=swiftshader it's not working anymore.

Edit : Also what's weird is this mailchimp webgl website works while webgl.org doesn't.

Do you have any idea ?

Sparticuz commented 2 years ago

@Antoine-C I would double check your code and verify that you are running my fork, or this PR. It's working on my end.

Antoine-C commented 2 years ago

@Antoine-C I would double check your code and verify that you are running my fork, or this PR. It's working on my end.

Thanks a lot for checking on this one, I'll double check everything I must have something wrong somewhere.

otisg commented 2 years ago

Just scanned the responses here and it looks like @alixaxel is not really maintaining this project. No comments from him in this PR (since March 8, 2022) and no merged PRs since last year (since September 2021).

Would it make sense to abandon this project and for people to switch to using https://www.npmjs.com/package/@sparticuz/chrome-aws-lambda instead? @Sparticuz - do you have interest in maintaining the project on an ongoing basis?

Sparticuz commented 2 years ago

@otisg I plan on maintaining my fork, (as well as this PR). I also see no need to change from the established api, so it should also be an easy drop in replacement.

I would love to see some PRs with more automated testing. I think this is one of the main reasons devs lose the ability to maintain a package. The only tests that are currently run test that chromium opens and goes to a website. None of the hooks or puppeteer overrides are tested. Check out _/amazon/handlers/index.js. Personally, I like ava and the node18 testing api seems extremely similar when we are able to update

yakirza17 commented 2 years ago

Hi, @Sparticuz, thanks for your great work! I'm trying to use the binary files you are published but cannot make WebGL work.

I can see this on stderr: [0522/162950.046166:ERROR:gpu_channel_manager.cc(831)] ContextResult::kFatalFailure: Failed to create shared context for virtualization. [0522/162950.046194:ERROR:shared_image_stub.cc(537)] SharedImageStub: unable to create context [0522/162950.046203:ERROR:gpu_channel.cc(572)] GpuChannel: Failed to create SharedImageStub

And in get.webgl.org I see that my browser doesn't support WebGL.

There is some configuration that we need to do to make this works inside regular docker lambda? (We are using Node 14)

More details:

We added these flags to the browser: '--use-gl=angle', // https://chromium.googlesource.com/chromium/src/+/main/docs/gpu/swiftshader.md '--use-angle=swiftshader', // https://chromium.googlesource.com/chromium/src/+/main/docs/gpu/swiftshader.md
We build dokcer file from this template FROM public.ecr.aws/lambda/nodejs:14-x86_64
In older versions of this package version, all works, we just want to upgrade our version.

Antoine-C commented 2 years ago

@Sparticuz Can confirm the issue with WebGL as @yakirza17 mentioned, double checked everything, used your fork with binaries and webgl is not working using the official lambda docker image FROM amazon/aws-lambda-nodejs:14 I also tried --use-angle=swiftshader-webgl without luck.

Sparticuz commented 2 years ago

@yakirza17 @Antoine-C I'm also able to confirm that webgl isn't working properly inside of lambda. Previously, I had only tested the chromium binaries directly (in WSL2) and never tested them (with webgl) inside lambda. (If you don't need webgl, my fork and PR work fine)

I've started working on trying to get some automated webgl testing, and I've come to the conclusion that the binaries are extracting incorrectly inside of /tmp. (Where chrome-aws-lambda expects them to be). All the swiftshader files are extracted to /tmp/swiftshader/ where I need them to be in /tmp/. This looks like it might need to be addressed in @alixaxel/lambdafs, but I'm looking into it.

yakirza17 commented 2 years ago

@Sparticuz Extract swiftshader.tar.br into the main directory instead of /swiftshader directory solves the issue.

Thank you very much

Sparticuz commented 2 years ago

I've published 14.1.1 and I believe I've addressed the webgl problem. @Antoine-C @yakirza17 Let me know if it works and I'll put my updates in this PR.

I had to 'fork' lambdafs in order to get the files. Check out source/lambdafs.ts https://github.com/Sparticuz/chrome-aws-lambda/blob/master/source/lambdafs.ts. (instead of forking, I've just moved the relevant code into this package and dropped the dependency). The main change was if we are extracting swiftshader.tar.bz then we use /tmp as the output path instead of /tmp/__filename__. This puts the files in the same basedir as the chromium executable.

aeristhy commented 2 years ago

does this fix this issue #276 ?

Sparticuz commented 2 years ago

does this fix this issue #276 ?

Yes

alixaxel / chrome-aws-lambda

Puppeteer@14.1.0 #264