berstend / puppeteer-extra

💯 Teach puppeteer new tricks through plugins.
https://extra.community
MIT License
6.54k stars 744 forks source link

Puppeteer not found when running script as cron job #350

Closed kekearif closed 4 years ago

kekearif commented 4 years ago

I am having a strange issue when attempting to run puppeteer with a cron job. Code as follows:

const puppeteer = require('puppeteer-extra');
const stealthPlugin = require('puppeteer-extra-plugin-stealth')();

stealthPlugin.enabledEvasions.delete('user-agent-override');
puppeteer.use(stealthPlugin);

const userAgent = UserAgentOverride({
    userAgent: 'some agent,
    locale: 'en-US,en'
});
puppeteer.use(userAgent);

const options = {
    headless: true,
    ignoreHTTPSErrors: true,
    args: [
        '--disable-sync',
        '--ignore-certificate-errors'
    ],
    ignoreDefaultArgs: ['--enable-automation']
};

const browser = await puppeteer.launch(options);

When I run the code in the terminal there are no issue however the code fails if run from a cron job. I printed the output of the cron job:

Puppeteer is missing. :-)

Note: puppeteer is a peer dependency of puppeteer-extra,
which means you can install your own preferred version.

- To get the latest stable version run: 'yarn add puppeteer' or 'npm i puppeteer'

Alternatively:
- To get puppeteer without the bundled Chromium browser install 'puppeteer-core'
- To use puppeteer-firefox install 'puppeteer-firefox' and use the 'addExtra' export

I've got no idea why it say it is missing if it works fine in the terminal. Looking at my package.json I have:

  "dependencies": {
    "bcryptjs": "^2.4.3",
    "cookie-parser": "^1.4.5",
    "cors": "^2.8.5",
    "express": "^4.17.1"
    "jsonwebtoken": "^8.5.1",
    "mongoose": "^5.10.2",
    "nodemon": "^2.0.4",
    "puppeteer": "^5.3.1",
    "puppeteer-extra": "^3.1.15",
    "puppeteer-extra-plugin-stealth": "^2.6.1"
  }

I tried not using puppeteer-extra and just regular puppeteer but I get this error:

    throw new errors.TypeError('ERR_INVALID_ARG_TYPE', 'original', 'function');
    ^

TypeError [ERR_INVALID_ARG_TYPE]: The "original" argument must be of type function
    at promisify (internal/util.js:209:11)
    at Object.<anonymous> (/home/kekearif/my-project/node_modules/extract-zip/index.js:11:18)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Module.require (module.js:596:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/home/kekearif/my-project/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:48:39)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Module.require (module.js:596:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/home/kekearif/my-project/node_modules/puppeteer/lib/cjs/puppeteer/node/Launcher.js:43:29)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)

Any ideas what I am doing wrong here? Any pointers would be greatly appreciated. thanks!

berstend commented 4 years ago

It'd be helpful if you'd copy your crontab entry here.

A quirk that often affects cron jobs: The PATH is much different from your regular PATH and contains only the bare minimum.

I suggest you create a super minimal vanilla puppeteer example and see if that causes issues with your cron setup.

kekearif commented 4 years ago

It'd be helpful if you'd copy your crontab entry here.

A quirk that often affects cron jobs: The PATH is much different from your regular PATH and contains only the bare minimum.

I suggest you create a super minimal vanilla puppeteer example and see if that causes issues with your cron setup.

crontab entry is just:

* * * * * node /home/kekearif/bot/bot.js

Does this look right?

berstend commented 4 years ago

This might be too minimal, try to cd into that directory before and check your PATH.

Pretty sure you'll have this issue with vanilla pptr as well, hence I'm closing this.