search

segment-boneyard / nightmare

A high-level browser automation library.
https://open.segment.com
19.55k stars 1.08k forks source link

Deploy Nightmare js to Heroku #1600

Open RonDaha opened 4 years ago

RonDaha commented 4 years ago

After a long struggle, I came here eventually seeking for any clue for a solution

I am trying to deploy nightmarejs app to heroku (which works perfectly on my mac). I followed all the guides that I could found on the internet, including: oscarmorrison johann.pardanaud kireerik

I also tried this

I kept getting errors that some packages missing so I adjusted the Aptfile followed by what I found online.

eventually the error I receive now is - electron:stderr /app/node_modules/electron/dist/electron: symbol lookup error: /app/node_modules/electron/dist/electron: undefined symbol: udata_setCommonData_59 nightmare electron child process exited with code 127: command not found - you may not have electron installed correctly

my buildpacks are:

  1. https://github.com/heroku/heroku-buildpack-apt
  2. https://github.com/captain401/heroku-buildpack-xvfb.git
  3. https://github.com/ubuntugod/heroku-electron-buildpack.git
  4. heroku/nodejs

using heroku-18 as the stack.

My Aptfile:

x11-xkb-utils xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic libxfont1 libnotify4 libxss1 libgconf-2-4 libgtkextra-dev libgconf2-dev libnss3 libasound2 libxtst-dev libxinerama1 libcairo-gobject2 x11-apps clang libdbus-1-dev libgtk2.0-dev libnotify-dev libgnome-keyring-dev libasound2-dev libcap-dev libcups2-dev libnss3-dev gcc-multilib g++-multilib

there are probably some packages that don't really necessary in the Aptfile but after so many tries I'm totally confuse

My Procfile:

web: DEBUG=* xvfb-run --auto-servernum --server-args="-screen 0 1280x1028x24 -ac +extension GLX +render" node --harmony ./build/app.js

nightmare version - 3.0.2 nodejs version - 12.13.0 npm version - 6.12.0

let me know if there is anything else that need to be share here.

if someone can provide any information I'll be grateful!

enrimon15 commented 4 years ago

I have the same problem, but I'm not getting an error similar to yours, I'm getting a connection timeout. Do you have the electron dependency in your package.json? I also tried with those solutions but nothing... Also because in the guides it is said to use cedar-14 but it is deprecated, can we also use heroku-18? I use express with nightmare and in heroku dashboard i turn off web dyno (npm start) and turn on the worker (xvfb-run.... the content of procfile), when i try to get data from a route i receive an Error: cannot get route, web is not available.

My procfile is: DEBUG=* xvfb-run --auto-servernum --server-args="-screen 0 1280x1028x24 -ac +extension GLX +render" node --harmony app.js

(without ./build/app.js) maybe this can be the problem? I need to add ./build/app.js?

RonDaha commented 4 years ago

@enrimon15 Hey sorry for the late response. Regarding the heroku-18 instead of the cedar-14, I'm also do not know if that even possible for this stack. still could not find any solution to my problem, will update if I do.

Now about your timeout, I am not sure but it seems from your error that the "web" is not available, maybe you can try switch from "worker" to "web". again, I am not sure it is just what I can think of from the error you received.

The ./build/app.js is just my path to run the build results file. It might be different for you so you don't have to copy it exactly like mine. Let me know if you still get the error and if so, please add the full error message.

enrimon15 commented 4 years ago

I found a "solution". I had to switch to cedar-14 because it is the only working stack and it is deprecated, it will be kept only until November 2, 2020. So it's a temporary solution. I needed to add these buildpacks: 1 - apt 2- xvfb 3- node js then I added the apt file you find here to my project, without .profile and procfile. Then I followed this guide by installing xvfb via npm and starting it with the dedicated instruction and I simply run the app with npm start (node app.js).

So with apt I install xvfb on the server and with the xvfb library included in the project with npm I start the virtual server display that makes electron and consequently nightmare work. If you don't have xvfb installed in your pc, it’ll just throw an error and still start the app.

vininjr commented 4 years ago

After a long struggle, I came here eventually seeking for any clue for a solution

I am trying to deploy nightmarejs app to heroku (which works perfectly on my mac). I followed all the guides that I could found on the internet, including: oscarmorrison johann.pardanaud kireerik

I also tried this

I kept getting errors that some packages missing so I adjusted the Aptfile followed by what I found online.

eventually the error I receive now is - electron:stderr /app/node_modules/electron/dist/electron: symbol lookup error: /app/node_modules/electron/dist/electron: undefined symbol: udata_setCommonData_59 nightmare electron child process exited with code 127: command not found - you may not have electron installed correctly

my buildpacks are:

  1. https://github.com/heroku/heroku-buildpack-apt
  2. https://github.com/captain401/heroku-buildpack-xvfb.git
  3. https://github.com/ubuntugod/heroku-electron-buildpack.git
  4. heroku/nodejs

using heroku-18 as the stack.

My Aptfile:

x11-xkb-utils xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic libxfont1 libnotify4 libxss1 libgconf-2-4 libgtkextra-dev libgconf2-dev libnss3 libasound2 libxtst-dev libxinerama1 libcairo-gobject2 x11-apps clang libdbus-1-dev libgtk2.0-dev libnotify-dev libgnome-keyring-dev libasound2-dev libcap-dev libcups2-dev libnss3-dev gcc-multilib g++-multilib

there are probably some packages that don't really necessary in the Aptfile but after so many tries I'm totally confuse

My Procfile:

web: DEBUG=* xvfb-run --auto-servernum --server-args="-screen 0 1280x1028x24 -ac +extension GLX +render" node --harmony ./build/app.js

nightmare version - 3.0.2 nodejs version - 12.13.0 npm version - 6.12.0

let me know if there is anything else that need to be share here.

if someone can provide any information I'll be grateful!

@RonDaha I solved this problem using a docker to build for heroku, using this link

Dockerfile

FROM ubuntu:18.04

WORKDIR /app

RUN echo "LC_ALL=en_US.UTF-8" >> /etc/environment RUN echo "LANG=en_US.UTF-8" >> /etc/environment RUN echo "NODE_ENV=development" >> /etc/environment RUN more "/etc/environment"

RUN apt-get update

RUN apt-get install -y wget curl gcc g++ make

RUN curl -sL https://deb.nodesource.com/setup_10.x | bash RUN apt-get install --yes nodejs

RUN apt-get install -y \ xvfb \ x11-xkb-utils \ xfonts-100dpi \ xfonts-75dpi \ xfonts-scalable \ xfonts-cyrillic \ x11-apps \ clang \ libdbus-1-dev \ libgtk2.0-dev \ libnotify-dev \ libgnome-keyring-dev \ libgconf2-dev \ libasound2-dev \ libcap-dev \ libcups2-dev \ libxtst-dev \ libxss1 \ libnss3-dev \ gcc-multilib \ g++-multilib

RUN cd /tmp &&\ wget -q https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb &&\ apt-get install -y libappindicator3-1 fonts-liberation libgbm1 xdg-utils &&\ dpkg -i google-chrome-stable_current_amd64.deb &&\ apt-get install -f -y;

ENV CHROME_BIN /usr/bin/google-chrome ENV DISPLAY :99

ADD . /app

RUN npm install CMD Xvfb :99 -screen 0 1024x768x24 -ac -nolisten tcp -nolisten unix > /dev/null 2>&1 &

heroku.yml

build: docker: web: Dockerfile #must be in root together with heroku.yml = run: web: DEBUG=nightmare:actions xvfb-run --auto-servernum --server-args="-screen 0 1024x768x24 :99.0" node index.js

RonDaha commented 4 years ago

@vininjr Docker had other problems when I try to deployed it to Heroku but I will try with your example and will update what's the results. Thank you!

RonDaha commented 4 years ago

For everyone how will have those issues in the future, after a very very long struggle I moved to puppeter.js. took me maybe 15 min to change my code since the API is very similar. manages to deploy it to Heroku with zero problems