search

Emurgo / cardano-serialization-lib

This is a library, written in Rust, for serialization & deserialization of data structures used in Cardano's Haskell implementation of Alonzo along with useful utility functions.
Other
231 stars 125 forks source link

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory #586

Open amnambiar opened 1 year ago

amnambiar commented 1 year ago

Hi,

On trying to use the latest version of asmjs package, to request a transaction from the wallet from a react-app, trying to bundle the app with npm run start throws JavaScript Heap error. I am using the below version of the package: "@emurgo/cardano-serialization-lib-asmjs": "^11.2.1"

The below is the error I can see in my terminal:

Starting the development server...

<--- Last few GCs --->

[82298:0x55970c0]    98980 ms: Scavenge 4132.0 (4232.5) -> 4128.0 (4232.5) MB, 6.6 / 0.0 ms  (average mu = 0.783, current mu = 0.849) allocation failure 
[82298:0x55970c0]   100348 ms: Scavenge 4133.0 (4232.5) -> 4129.9 (4232.5) MB, 7.9 / 0.0 ms  (average mu = 0.783, current mu = 0.849) allocation failure 
[82298:0x55970c0]   100738 ms: Scavenge 4135.7 (4232.5) -> 4131.9 (4248.5) MB, 11.7 / 0.0 ms  (average mu = 0.783, current mu = 0.849) allocation failure 

<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb309a0 node::Abort() [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
 2: 0xa4219e node::FatalError(char const*, char const*) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
 3: 0xd2389e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
 4: 0xd23c17 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
 5: 0xedd045  [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
 6: 0xeee48d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
 7: 0xef11be v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
 8: 0xeb2bda v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
 9: 0x1233038 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
10: 0x16364b9  [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]

I have found that the basic fix to JavaScript heap errors is to increase the --max-old-space-size allocation to >8GB within NODE_OPTIONS globally or include it in the scripts. Also, the allocation size seemed to vary among host machines, and we couldn't come to a conclusion of what size to use. It didn't make sense to blindly allocate a huge space.

But this doesn't seem like the only and main reason behind this error.

I believe that it is some issue within this version of the package or the package itself, because while trying to use a lower version like 10.2.0 as suggested in https://stackoverflow.com/a/73627677, these heap errors did not occur. But instead, there were not found errors for methods .from_bech32, .coins_per_utxo_byte etc.

Can someone please have a look at it, or correct me if I am wrong.

lisicky commented 1 year ago

Hi @amnambiar ! Could you provide code example to reproduce the error ?

amnambiar commented 1 year ago

Hey @lisicky ,

I've tried to quickly put together a repo here: https://github.com/amnambiar/my-app

Just try to fork/download the folder and npm install, npm run start

You should be able to see the below error , for which the trace is not exactly same to the one mentioned above - still the same 


<--- Last few GCs --->

[94159:0x56cd200]   121844 ms: Mark-sweep 4038.8 (4143.9) -> 4036.1 (4143.8) MB, 4422.6 / 0.0 ms  (average mu = 0.194, current mu = 0.008) allocation failure scavenge might not succeed
[94159:0x56cd200]   125536 ms: Mark-sweep 4040.7 (4143.9) -> 4036.4 (4144.2) MB, 3663.4 / 0.1 ms  (average mu = 0.113, current mu = 0.008) allocation failure scavenge might not succeed

<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb309a0 node::Abort() [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
 2: 0xa4219e node::FatalError(char const*, char const*) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
 3: 0xd2389e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
 4: 0xd23c17 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
 5: 0xedd045  [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
 6: 0xeee48d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
 7: 0xef11be v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
 8: 0xeb2912 v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
 9: 0xeaadec v8::internal::FactoryBase<v8::internal::Factory>::AllocateRawArray(int, v8::internal::AllocationType) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
10: 0xeaae95 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArrayWithFiller(v8::internal::Handle<v8::internal::Map>, int, v8::internal::Handle<v8::internal::Oddball>, v8::internal::AllocationType) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
11: 0x1112c4d v8::internal::Handle<v8::internal::EphemeronHashTable> v8::internal::HashTable<v8::internal::EphemeronHashTable, v8::internal::ObjectHashTableShape>::New<v8::internal::Isolate>(v8::internal::Isolate*, int, v8::internal::AllocationType, v8::internal::MinimumCapacity) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
12: 0x1112ef3 v8::internal::Handle<v8::internal::EphemeronHashTable> v8::internal::HashTable<v8::internal::EphemeronHashTable, v8::internal::ObjectHashTableShape>::EnsureCapacity<v8::internal::Isolate>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::EphemeronHashTable>, int, v8::internal::AllocationType) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
13: 0x1113625 v8::internal::ObjectHashTableBase<v8::internal::EphemeronHashTable, v8::internal::ObjectHashTableShape>::Put(v8::internal::Isolate*, v8::internal::Handle<v8::internal::EphemeronHashTable>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
14: 0x111380b v8::internal::JSWeakCollection::Set(v8::internal::Handle<v8::internal::JSWeakCollection>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
15: 0x1229617 v8::internal::Runtime_WeakCollectionSet(int, unsigned long*, v8::internal::Isolate*) [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]
16: 0x16364b9  [/home/anusree/.nvm/versions/node/v17.9.0/bin/node]

I have the transaction specific code mentioned within App.tsx itself. Can you help me if it is in any way related to the way I am building the transaction?

amnambiar commented 1 year ago

Hi,

An additional update regarding the issue. The node version I am on right now is 17.9.0

lisicky commented 1 year ago

@amnambiar I need some time to investigate it probably it is normal behavior. Also you can use GENERATE_SOURCEMAP=false env or use cardano-serialization-lib-browser package it should be lighter than asmjs version. Here https://github.com/Emurgo/cardano-serialization-lib/issues/575#issuecomment-1396202135 you can find some information for troubleshooting if you decide to use cardano-serialization-lib-browser

amnambiar commented 1 year ago

@lisicky Thank you,

Trying to use cardano-serialization-lib-browser instead of the asmjs, It threw errors, an excerpt below:

ERROR in ./node_modules/@emurgo/cardano-serialization-lib-browser/cardano_serialization_lib_bg.js 24962:45-67
Can't import the named export '__wbindgen_malloc' (imported as 'wasm') from default-exporting module (only default export is available)

ERROR in ./node_modules/@emurgo/cardano-serialization-lib-browser/cardano_serialization_lib_bg.js 24964:44-66
Can't import the named export '__wbindgen_malloc' (imported as 'wasm') from default-exporting module (only default export is available)

ERROR in ./node_modules/@emurgo/cardano-serialization-lib-browser/cardano_serialization_lib_bg.js 24966:6-22
Can't import the named export 'vrfcert_new' (imported as 'wasm') from default-exporting module (only default export is available)

ERROR in ./node_modules/@emurgo/cardano-serialization-lib-browser/cardano_serialization_lib_bg.js 24975:6-42
Can't import the named export '__wbindgen_add_to_stack_pointer' (imported as 'wasm') from default-exporting module (only default export is available)

ERROR in ./node_modules/@emurgo/cardano-serialization-lib-browser/cardano_serialization_lib_bg.js 24994:4-30
Can't import the named export '__wbg_vrfkeyhash_free' (imported as 'wasm') from default-exporting module (only default export is available)

ERROR in ./node_modules/@emurgo/cardano-serialization-lib-browser/cardano_serialization_lib_bg.js 25002:21-57
Can't import the named export '__wbindgen_add_to_stack_pointer' (imported as 'wasm') from default-exporting module (only default export is available)

ERROR in ./node_modules/@emurgo/cardano-serialization-lib-browser/cardano_serialization_lib_bg.js 25003:44-66
Can't import the named export '__wbindgen_malloc' (imported as 'wasm') from default-exporting module (only default export is available)
lisicky commented 1 year ago

@amnambiar I have a simple react app with the CSL. You can compare it with your project config https://github.com/lisicky/csl_react_test Also you can search a error text among CSL issues

amnambiar commented 1 year ago

@lisicky , Thanks a lot in putting this up!

I tried to run this app in my local and here is the results - image Can you help me debug it.

Also, tried the similar approach in my application, and the results are the same as in my previous comment

lisicky commented 1 year ago

@amnambiar I have updated config for you. Check the repo

amnambiar commented 1 year ago

@lisicky Great, a Quick one! Yes, your app works.

Trying it in the sample app https://github.com/amnambiar/my-app, I am getting the below error: image

But, doesn't work in my main application. Still got the same set of errors - image image

lisicky commented 1 year ago

@amnambiar check please configs, even package.json and config-overrides.js

lisicky commented 1 year ago

@amnambiar and check scripts section in package.json

amnambiar commented 1 year ago

@amnambiar and check scripts section in package.json

This worked !! Thanks a lot

bakon11 commented 1 year ago

Hey,

I'm not really seeing anything new under scripts in package.json except react-app-rewired added to dev dependencies.

can you tell me what you did different or added to the scripts?

I am trying to follow this example to get my WSM working in react with Webpack5 as well as recommended in my issue #585 .

Thanks, Mike

bakon11 commented 1 year ago

Ahh I see now the github example from @lisicky.

Do I need to run the eject script first?

Thanks.

bakon11 commented 1 year ago

NVM Got it working

TeamEnlighten commented 1 year ago

Does anyone get this warning?

WARNING in ./node_modules/@emurgo/cardano-serialization-lib-browser/cardano_serialization_lib_bg.js 26907:14-53 Critical dependency: the request of a dependency is an expression