Error deploying on Heroku - FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

[mayordwells]

Please help me, I don't understand the meaning of this error. See the error log in the gist below. I will appreciate a brief explanation of what exactly is going on and what I can do to get a solution. Thanks. The error is over 1 week old and I've tried every possible means to find a solution to no avail.

https://gist.github.com/mayordwells/6a5579f0528dcafd01b0e692c74f4c51

A direct reply from Heroku support did not help either.

[joelsouza]

Running assets:precompile on your local env fails with the same error?

RAILS_ENV=production rake assets:precompile --trace

[mayordwells]

@joelsouza Yes, I got the same error when I ran that command on my local environment. I have created another gist to show the output. - https://gist.github.com/mayordwells/1646fdd1b5921568d2375a102cd11fcc

Looking forward to a proposed solution to this problem, please. Thank you.

[joelsouza]

@mayordwells You need to provide more info about your env. Anyway, it seems more related to Node than Webpacker itself.

Try to isolate the problem. Create a new fresh clean app and run RAILS_ENV=production rake assets:precompile --trace again to make sure that isn't a problem with your project assets.

[mayordwells]

@joelsouza thanks, I'm using rails 5.2.0.rc1 and ruby 2.5.0 I create a new test app, assets:precompile ran successfully, here's the output - https://gist.github.com/mayordwells/b48b8dcbfa4840bd778d25fbace44043

What other info about my env do you want me to provide? And if its a node problem how can we fix it?

[joelsouza]

@mayordwells So, if it runs successfully with a clean app it's a problem with one of your assets causing the javascript/node/v8 FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory not Webpacker. :-)

Better close the issue.

[mayordwells]

@woniesong92 I have no idea what produced or manufactured this error in my project. Project assets is everything in your assets folder, including css, js, and images of course. But I was also experiencing this even in my local environment before deploying to Heroku. I'm currently in the process of migrating to Digital Ocean to have more control over my production environment.

[mayordwells]

We found a fix for the error by using this library, basically, the error has to do with the various imports of the JS libraries in different files in our project, those imports even though, the same for example - import React from 'react'; in 10 different files means that it's loading 10 times which results in such a huge size hence the error JavaScript heap out of memory. I suppose this could have been avoided by a technique known as tree-shaking in webpack but maybe it's not implemented in webpacker yet? Anyway, we used Autodll-webpack-plugin to compile our project and shake off repeated imports in our file. Which in turn reduced our compile time by at least 50% both in development and production. Took us more than 3 weeks to find a fix for this error. But I hope it helps someone else in the future.

[mayordwells]

Hi anyone got a better solution for webpack configuration in a large rails + react + webpacker project?

I'm still getting this error when running my project without the Autodll-webpack-plugin. Getting exactly the same error when I was running ruby 2.5.0 + rails 5.2.0.rc2, here's the log of the heap error - https://gist.github.com/mayordwells/c2562e67e294bb1fab09a7f28ae45241 Today I'm running rails 5.2.0 + webpacker 4.0.x in a separate branch because I thought upgrading to the new webpacker will fix the problem. But it doesn't.

My Experience with rails 5.2.0 + webpacker 3.5

My previous solution with Autodll is not the best, and that is because whenever we make any changes to package.json file we have to clear cache both in production and development mode to have access to latest updates if not the page just remains blank and it feels like the site is not working. Here's the config - https://gist.github.com/mayordwells/2ebf22e4a6280bbf0c4906d138924fe8

Also most time, in development env, we have to run find . -name "*vendor*" and then copy to public folder using this command - cp ./node_modules/.cache/autodll-webpack-plugin/development_instance_0_3939a249926c468da4f1492991531438/vendor.dll.js public/

The file is then used our app by importing it to the application layout like so - = javascript_include_tag '/vendor.dll.js', 'data-turbolinks-track': 'reload'

The reason why cache needs to be cleared whenever we make changes to the package.json is that a new hash is generated and appended to vendor.dll.js and application.js file each time a change is made, by change I mean, either upgrading a package in the file or adding/installing a new package. The major problem with this is that not all our client/previous visitors to our site know that they need to clear the cache to be able to see the site. They just simply say the site is not working and close it. Also, this kind of permanently puts our react in development mode forever, because all the changes I've made to put our react in production has failed.

My experience with rails 5.2.0 + webpacker 4.0.x

Using this same config - https://gist.github.com/mayordwells/2ebf22e4a6280bbf0c4906d138924fe8 (Btw, To reproduce the error JavaScript heap out of memory error we simply disable the plugins array from Line 13 - 33. That is what produced this error log that I shared above).

The app is successfully compiled using the same config with the plugins array / using autodll but I still get some error in the browser console even when the cache is cleared, here's a screenshot

So basically we can't upgrade to webpacker 4 and we can't fix the configuration problems of the past that we temporarily fixed using autodll-webpack-plugin. Now that our user base is beginning to grow, its becoming impossible to tell each and every one of them to clear their cache everytime they get a blank screen.

Please help us find a lasting and final solution to these problems.

I appreciate your comments.

Also @joelsouza the simple answer to your last comment is that the app is simply not big enough that's why it compiles successfully in a clean app. Try importing and using react in at least 20 - 50 different files, you are definitely going to reproduce this error.

[fernandes]

@mayordwells facing the same problem here, still investigating... no progress at all :/

[gauravtiwari]

@mayordwells @fernandes Please could you double check if there is any circular dependency in your app i.e. if a pack file is being used elsewhere (CSS pack or an image)?

[fernandes]

hi @gauravtiwari

actually mine is a SPA, I just use the tag on a view and runs the SPA, would fall in this case you mentioned?

ps: I'm just having this problem when building at heroku, locally works fine (consuming 1.1GB when building the RAILS_ENV=production rake assets:precompile, but works heheh)

thank you for your reply

[mayordwells]

We fixed this problem by restructuring our frontend app. I think this was the root of the problem. Now our app is running on the bleeding edge version of webpacker v4.0.x The packs folder should only contain the entry file application.js

The structure of the frontend folder generated by webpacker now looks like this. javascript/packs inside the packs folder we have the entry file application.js javascript/components inside the components folder we have the whole app / all other components.

And then you can import files from the components folder inside the entry file like so - import '../components/Home/Index';

This perfectly fixed our almost 1year old error with the JavaScript heap out of memory.

This should definitely be documented.

[gauravtiwari]

@mayordwells We have documented this briefly here: https://github.com/rails/webpacker#usage (but maybe need to be a bit more clear). Could you please make a PR?

[gauravtiwari]

actually mine is a SPA, I just use the tag on a view and runs the SPA, would fall in this case you mentioned?

@fernandes If it's one pack file then probably not, but worth checking rest of the app (in case there is a circular dependency there)

consuming 1.1GB when building the RAILS_ENV=production rake assets:precompile, but works heheh

Sounds like there is some problem in structuring ^^

[mayordwells]

@gauravtiwari yes I can send a PR, no problem.

Just seeing the brief mention in the documentation now, for the first time - # only webpack entry files here

Will try to update the documentation. It's funny how skipping a very little detail could have given us so much pain and headache.

[fernandes]

@gauravtiwari mine structure is like @mayordwells commented

app/javascript/packs/app.js

all other files inside: app/javascript/src

not sure what's happening

[fernandes]

just giving a follow up, in case can help someone in the future... webpack-dev-server, was indicating some [big] packs, and as I have an app pack (that contains the SPA), was really big... so I checked:

1. all vendor (node_modules) were bundled together
2. some big dependencies like plotly.js

so used common-chunks-plugin, and configured with webpack as described in the docs plugin sections because the requirement is to prepend in the plugins list.

after that, now I have app, vendor, plotly packs.. and heroku build is happy! 👍

I still think the build is using so much memory, build at least it deployed and I can check with some time the dep tree, if you have any guide on how to debug and could you send me the link @gauravtiwari I do really appreciate.

[mayordwells]

@fernandes I think you need to watch this. You are going to find a lot of tips in the video - https://www.youtube.com/watch?v=ivQ7HrnBJe8&feature=youtu.be

Thank me later. Good luck.

[fernandes]

@mayordwells I'm thanking you now! 🙏 heheheh thanks for the advice! I'm gonna watch this in a few minutes

[eduardobvale]

@mayordwells My team was struggling with this problem, but restructuring the frontend app did the job. We spent a whole day debugging webpack so we could notice that we had a lot of entry points. Thanks for sharing!

[s1mpl3]

This issue has been driving crazy and I have spent way too many hours trying to address it.
The video sent by @mayordwells has been priceless (thank you). In particular from that video, what did the trick for us was to move every exportable component and library outside the packs folder (ex. javascript/components vs javascript/packs/components) and leave only the .jsx files that call ReactDOM.render. Now the crashes are gone and the compilation times are 4-5 times faster.

[apiep]

Hi, facing the same problem here (out of heap memory) and after googling I found that it was a problem with webpack-dev-server (in my case), and can be temporary fixed by passing an argument to node. But my bin/webpack-dev-server was managed by webpacker, so how do I pass and argument to node?

Here is the issue on webpack-dev-server https://github.com/webpack/webpack-dev-server/issues/1433

Edit: found a solution here (to pass node option) https://github.com/rails/webpacker/issues/1189

[prasanthrubyist]

I also faced the same problem, After spending of some hours I find my mistake, The problem is due to importing bulk JSON, like this import codes from "./pincode.json";

Hope this will help someone, Thank you

[gauravtiwari]

Please try the latest RC, otherwise feel free to open a new issue with more details.

[fernandes]

hi @gauravtiwari , I've been using 4.0.0 since August and couldn't be happier... with webpack 4 + split chunks, all my problems were solved. thank you very much