linkedtales / scrapedin

LinkedIn Scraper (currently working 2020)
Apache License 2.0
594 stars 171 forks source link

"Failed to launch chrome" Error in Firebase Cloud Functions #65

Open marcoizzo opened 4 years ago

marcoizzo commented 4 years ago

Hi all,

thank you very much and congratulation for the library! I'm trying to integrate it in a Firebase Cloud Function but I get this error when trigger the execution

Error: Failed to launch chrome! [12:17:0204/203657.942572:ERROR:file_path_watcher_linux.cc(71)] Failed to read /proc/sys/fs/inotify/max_user_watches

(chrome:12): Gtk-WARNING **: 20:36:58.019: cannot open display:

TROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md

at onClose (/srv/node_modules/scrapedin/node_modules/puppeteer/lib/Launcher.js:348:14)
at ChildProcess.helper.addEventListener (/srv/node_modules/scrapedin/node_modules/puppeteer/lib/Launcher.js:338:60)
at emitTwo (events.js:131:20)
at ChildProcess.emit (events.js:214:7)
at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)

This is my code:

const functions = require('firebase-functions');
const scrapedin = require('scrapedin')

const runtimeOpts = {
    timeoutSeconds: 500,
    memory: '2GB'
}

exports.scrapeLinkedinProfile = functions.runWith(runtimeOpts).https.onRequest(async (req, res) => {
    const options = {
        hasToLog: true,
        isHeadless: false,
        puppeteerArgs: { args: ['--no-sandbox', '--disable-setuid-sandbox'] },
        email: 'XXX',
        password: 'YYY'
    }
    const profileScraper = await scrapedin(options)
    const profile = await profileScraper('https://www.linkedin.com/in/XXX')  
    res.status(200).send(JSON.stringify(profile));
})

I've tried to use puppeteer library itself (v2.1.0) and seems work properly.

Do you have any suggestion?

Thank you very much!

leonardiwagner commented 4 years ago

Hi @marcoizzo , it's really a puppeteer issue, and it's probably related to the puppeteer version as we're using 1.13.

Could you test the 2.1.0 version on scrapedin and check if it works in your scenario?

You have to go on \node_modules\scrapedin and change puppeteer version on package.json and perform a npm i to install that new version.

I'm also not aware if on v2 has breaking changes

leonardiwagner commented 4 years ago

closed due to lack of feedback

trns1997 commented 4 years ago

Hey guys, It is indeed a puppeteer issue. I tried to upgrade the puppeteer version as suggested by @leonardiwagner. Sadly it did not work. Though I managed to get it running successfully on gclouds app engine by using docker. Hope this answer helps someone in the future. Useful link for docker file: https://github.com/buildkite/docker-puppeteer/blob/master/Dockerfile You basically need to get chrome installed to get everything working properly.

leonardiwagner commented 4 years ago

Hi @trns1997 , thank you for the feedback, I'll put that on the docs!