search

getredash / redash

Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.
http://redash.io/
BSD 2-Clause "Simplified" License
26.5k stars 4.39k forks source link

puppeteer issue with Apple Silicon chips #6885

Open square-li opened 7 months ago

square-li commented 7 months ago

Issue Summary

With Apple Silicon chips (ARM), execute yarn to install dependencies will fail like 

error /Users/.../redash/node_modules/puppeteer: Command failed.
Exit code: 1
Command: node install.js
Arguments:
Directory: /Users/.../redash/node_modules/puppeteer
Output:
The chromium binary is not available for arm64:
If you are on Ubuntu, you can install with:
 apt-get install chromium-browser

Steps to Reproduce

  1. run yarn on MacBook with Apple Silicon

Any other info e.g. Why do you consider this to be a bug? What did you expect to happen instead?

Technical details:

square-li commented 7 months ago

My temporary solution:

Following this guide: https://github.com/puppeteer/puppeteer/issues/6634#issuecomment-755430921

OR TLDR:

add these to package.json file

"resolutions": {
    "puppeteer": "https://github.com/seanaye/puppeteer/releases/download/v5.5.0-apple-silicon/puppeteer-core-5.5.0-post.tgz"
},
justinclift commented 7 months ago

@gaecoli From memory, you use macOS yeah?

If that's the case, then this ^^^ sounds like it'll be useful and we could add it to the package.json. :smile:

justinclift commented 7 months ago

@square-li Thanks heaps, that does sound useful. :grin:

square-li commented 7 months ago

From what I understand, that package is used for frontend automated test. If anyone who just wants to test and run the app, they shouldn't be blocked on this.

gaecoli commented 7 months ago

@gaecoli From memory, you use macOS yeah?

If that's the case, then this ^^^ sounds like it'll be useful and we add it to the package.json. 😄

image image

The same result.

gaecoli commented 7 months ago

@justinclift This seems to be a problem and we should look at how to fix it. 😄

gaecoli commented 7 months ago

https://github.com/puppeteer/puppeteer/issues/6622

I have found some solutions, but it's best to allow users to install dependencies directly through yarn without doing any extra work.

justinclift commented 7 months ago

No worries. It sounds like we should create an extra page on the wiki for people using macOS, so we can give people the macOS specific steps.

If there's not much difference to the main "Local dev setup" page then we don't have to include the whole thing, we can just tell people what differences to install first or something.

What do you reckon? :smile:

justinclift commented 7 months ago

If anyone who just wants to test and run the app ...

That kind of sounds like "people who just want to use the app rather than develop it". If that's what you're meaning (?), then for that situation we should probably extend the official setup.sh repository scripting to support macOS.

That might not be too hard. Not really sure yet though as I've not tried it. :smile:

justinclift commented 7 months ago

... it's best to allow users to install dependencies directly through yarn without doing any extra work.

Yeah, I think we're in agreement here. If we can update things to automatically work for people so they don't need to follow extra instructions, that would be best.

If "automatic" doesn't work though, then at least having extra instructions for macOS users is probably the right approach.

gaecoli commented 7 months ago

... it's best to allow users to install dependencies directly through yarn without doing any extra work.

Yeah, I think we're in agreement here. If we can update things to automatically work for people so they don't need to follow extra instructions, that would be best.

If "automatic" doesn't work though, then at least having extra instructions for macOS users is probably the right approach.

Ok

square-li commented 7 months ago

@gaecoli From memory, you use macOS yeah?

If that's the case, then this ^^^ sounds like it'll be useful and we add it to the package.json. 😄

image image

The same result.

Sorry I was missing to mention you would need to install chromium for arm as well.

"brew install chromium" then modify the package.json solves my issue.

justinclift commented 7 months ago

@square-li Any interest in creating a new page on the wiki with instructions for macOS users trying to get a dev environment set up? :smile:

square-li commented 7 months ago

@square-li Any interest in creating a new page on the wiki with instructions for macOS users trying to get a dev environment set up? 😄

For sure. I would love to.

justinclift commented 7 months ago

Awesome, please do. You should already have the right permissions for making changes on the wiki, and if you hit issues or have questions then please ask here or anywhere else suitable. :smile:

If Discord is your kind of thing, then we have a chat server for developers to discuss stuff. You're welcome to join it:

    https://discord.gg/tN5MdmfGBp

wtfiwtz commented 7 months ago

On Node >=18:

PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 yarn add puppeteer
node node_modules/puppeteer/install.mjs

Based on this: https://github.com/puppeteer/puppeteer/issues/6622#issuecomment-1908747462

I was also able to change @percy/agent to @percy/cli to resolve this but it broke some tests, so that needs more investigation time.

justinclift commented 7 months ago

Cool, that looks potentially useful too. Thanks @wtfiwtz. :smile:

gaecoli commented 7 months ago

Cool! This good to me.

AndrewChubatiuk commented 7 months ago

added PR #6912, which upgrades percy and cypress, which fixes this issue. please test it if it works for you

justinclift commented 3 months ago

As a data point, I'm working through a guide for setting up a local dev environment on macOS ARM64, using an M1 Mac Mini:

https://github.com/getredash/redash/wiki/Local-development-setup-%E2%80%90-macOS-Sonoma-%2814.6%29

It's still a work in progress, but the main Python and NodeJS dependencies seem to work now (~or will after #7140 is merged~ done).

justinclift commented 3 months ago

It sounds like @AndrewChubatiuk's above PR had some useful pieces in it, so it's probably a good idea for someone (probably me?) to take a look through it and see what it takes to upgrade Cypress, and maybe Percy too.

justinclift commented 2 months ago

@gaecoli @square-li @lucydodo I've just completed a first draft of the local development setup guide for macOS:

https://github.com/getredash/redash/wiki/Local-development-setup-%E2%80%90-macOS-Sonoma-(14.6)

Does anyone have time & interest to try it out to verify it works for them?

I'd do it myself, but macOS virtualisation is a problem on M1 cpus due to not supporting nested virtualisation. ie I'd have to completely wipe this macOS installation I just set up. (!)

gaecoli commented 2 months ago

When i add "resolutions": { "puppeteer": "https://github.com/seanaye/puppeteer/releases/download/v5.5.0-apple-silicon/puppeteer-core-5.5.0-post.tgz" },

it's work for me.

I think we should not let users to brew some lib, we should add some code to package.json to solve this problem!

gaecoli commented 2 months ago
image
justinclift commented 2 months ago

@gaecoli Cool. That approach seems like it has potential too.

Would you be ok to try it with the full make test and Cypress tests too (on your local pc), as I think it's mostly the Cypress tests that actually use it. :smile:

gaecoli commented 2 months ago

@gaecoli Cool. That approach seems like it has potential too.

Would you be ok to try it with the full make test and Cypress tests too (on your local pc), as I think it's mostly the Cypress tests that actually use it. 😄

let me try it

gaecoli commented 2 months ago

@gaecoli Cool. That approach seems like it has potential too.

Would you be ok to try it with the full make test and Cypress tests too (on your local pc), as I think it's mostly the Cypress tests that actually use it. 😄

It's ok, but we test maybe has bugs.

justinclift commented 2 months ago

When i add "resolutions": { "puppeteer": "https://github.com/seanaye/puppeteer/releases/download/v5.5.0-apple-silicon/puppeteer-core-5.5.0-post.tgz" },

it's work for me.

I think we should not let users to brew some lib, we should add some code to package.json to solve this problem!

I've been thinking this over. I think we're probably better off to have people use the version of Chromium that's in Homebrew, as that way people will always be testing with a modern release of Chromium.

For development on macOS we already have people install other dependencies via Homebrew:

$ brew install chromium freetds mysql-client pwgen

So it's probably fine. If the Homebrew version of Chromium does turn out to break a lot or cause issues, then I guess we can take another look then. :smile:

lucydodo commented 2 months ago

IME, 'chromium' is not code-signed, notarized, and I had to provide the --no-quarantin option to get it run properly: brew install --cask chromium --no-quarantin

justinclift commented 2 months ago

Oh. Sounds like we'd better update the wiki page for macOS then. :smile:

With that brew install command, do you know if the --cask option is still needed? I thought that --cask (as a command line option) had been deprecated.

justinclift commented 2 months ago

@lucydodo As a data point, this PR is why I thought that option has been deprecated. Haven't actually checked if that's the case though.

lucydodo commented 2 months ago

Oh. Sounds like we'd better update the wiki page for macOS then. 😄

No worries, I updated the wiki at the same time I left the comment.

The --cask option has not been deprecated. but you can omit the --cask option if the formula and cask names do not duplicate, but I prefer to be more explicit because sometimes they do overlap, as in the following case:

$ brew install docker (docker engine only)
$ brew install --cask docker (Docker Desktop)
justinclift commented 2 months ago

Awesome, thanks for getting that done. :smile: