philippta / flyscrape

Flyscrape is a command-line web scraping tool designed for those without advanced programming skills.
https://flyscrape.com
Mozilla Public License 2.0
1.04k stars 31 forks source link

Browser mode fails WSL missing dependencies (+ solution) #63

Open dynabler opened 6 months ago

dynabler commented 6 months ago

To get browser mode working on WSL is as follows:

$ wsl --install
$ sudo apt update && sudo apt upgrade -y
$ sudo apt install libnss3-dev libgdk-pixbuf2.0-dev libgtk-3-dev libxss-dev libasound2
$ curl -fsSL https://flyscrape.com/install | bash
$ flyscrape run script.js

Below are various attempts to make it work. Attempt no. 2 worked, because dependencies for Google Chrome were installed. So I figure it must be missing dependencies. It could be 230 of them to get Chromium that Flyscrape downloads working. Every attempt was done on a clean installation of WSL Ubuntu.

Attempt 1

$ wsl --install
$ curl -fsSL https://flyscrape.com/install | bash
$ flyscrape run script.js

Failed to launch the browser, the doc might help https://go-rod.github.io/#/compatibility?id=os: /home/dynabler/.cache/rod/browser/chromium-1131657/chrome: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory

Attempt 2

guide: https://www.tiredsg.dev/blog/install-google-chrome-wsl/

$ wls --unregister Ubuntu
remove Ubuntu (Settings > Apps > Ubuntu > Uninstall)
$ wsl --install
$ sudo apt update && sudo apt upgrade -y
$ wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
$ sudo apt -y install ./google-chrome-stable_current_amd64.deb
$ which google-chrome
$ google-chrome --version
$ curl -fsSL https://flyscrape.com/install | bash
flyscrape run script.js

Attempt 3

guide: https://stackoverflow.com/questions/58134793/error-while-loading-shared-libraries-libnss3-so-while-running-gtlab-ci-job-to

$ wls --unregister Ubuntu
$ remove Ubuntu (Settings > Apps > Ubuntu > Uninstall)
$ wsl --install
$ sudo apt update && sudo apt upgrade -y
$ sudo apt-get install chromium-browser
$ which chromium-browser
$ curl -fsSL https://flyscrape.com/install | bash
$ flyscrape run script.js

failed to launch browser: [launcher] Failed to launch the browser, the doc might help https://go-rod.github.io/#/compatibility?id=os: /home/dynabler/.cache/rod/browser/chromium-1131657/chrome: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory

$ sudo apt install libnss3 libatk1.0-0 libcups2 libxkbcommon0 libxcomposite1 libxdamage1 libxfixes3 libasound2
$ flyscrape run script.js

Attempt 4

$ wls --unregister Ubuntu
$ remove Ubuntu (Settings > Apps > Ubuntu > Uninstall)
$ wsl --install
$ sudo apt update && sudo apt upgrade -y
$ sudo apt install libnss3 libatk1.0-0 libcups2 libxkbcommon0 libxcomposite1 libxdamage1 libxfixes3 libasound2
$ curl -fsSL https://flyscrape.com/install | bash
$ flyscrape run script.js

failed to launch browser: [launcher] Failed to launch the browser, the doc might help https://go-rod.github.io/#/compatibility?id=os: /home/dynabler/.cache/rod/browser/chromium-1131657/chrome: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory

$ apt install libnss3-dev libgdk-pixbuf2.0-dev libgtk-3-dev libxss-dev
$ flyscrape run script.js

Attempt 5

This one is how it should be.

$ wls --unregister Ubuntu
remove Ubuntu (Settings > Apps > Ubuntu > Uninstall)
$ wsl --install
$ sudo apt update && sudo apt upgrade -y
$ sudo apt install libnss3-dev libgdk-pixbuf2.0-dev libgtk-3-dev libxss-dev libasound2
$ curl -fsSL https://flyscrape.com/install | bash
$ flyscrape run script.js