Core Dump when serializeModel()

[jajoe]

When i serialize the model, I have always a core dump. Here is the error message :

<--- Last few GCs --->

[21316:0x2f845c0] 73982 ms: Mark-sweep 710.7 (743.3) -> 709.0 (743.3) MB, 50.4 / 0.0 ms allocation failure GC in old space requested [21316:0x2f845c0] 74032 ms: Mark-sweep 709.0 (743.3) -> 708.3 (743.3) MB, 50.2 / 0.0 ms allocation failure GC in old space requested [21316:0x2f845c0] 74087 ms: Mark-sweep 708.3 (743.3) -> 708.3 (734.8) MB, 55.5 / 0.0 ms last resort [21316:0x2f845c0] 74138 ms: Mark-sweep 708.3 (734.8) -> 708.3 (734.8) MB, 50.9 / 0.0 ms last resort

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0xcc0670a9891 2: expandFileStorage [/home/jajoe/Desktop/draft-nodejs/hog-image-js/demo/node_modules/libsvm-js/dist/asm/libsvm.js:~1] [pc=0x173becd4b03c](this=0x183056869951 <an Object with map 0x252fcc6762f1>,node=0x1fcfe67bbc91 ,newCapacity=75210092) 3: write [/home/jajoe/Desktop/draft-nodejs/hog-image-js/demo/node_modules/libsvm-js/dist/asm/libsvm.js:~1] [pc=0x173becd4a878](this=0x1ebdb60...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory 1: node::Abort() [node] 2: 0x13647ec [node] 3: v8::Utils::ReportOOMFailure(char const, bool) [node] 4: v8::internal::V8::FatalProcessOutOfMemory(char const, bool) [node] 5: v8::internal::Factory::NewUninitializedFixedArray(int) [node] 6: 0xe90de3 [node] 7: v8::internal::Runtime_GrowArrayElements(int, v8::internal::Object*, v8::internal::Isolate) [node] 8: 0x173bec70437d Aborted (core dumped)

[stropitek]

@jajoe can you provide a self contained example that reproduces this error?

[jajoe]

https://github.com/jajoe/tutorial-nodejs/blob/master/image-classification/index.js line 93 : uncomment to enable the serializeModel()

[stropitek]

The model contains all support vectors. Because it is text format it's very inefficient (every feature is encoded with many characters). Therefore the size of the model explodes.

To fix this we can:

• Use a pre-computed kernel. The model size will be much smaller and depend only on the number of samples in the training set, not on the dimensionality of each sample. BTW, using a pre-computed kernel will allow us to use the mljs family of kernels, which is cool! But doing this will likely impact performance. See #2
• Serialize using a more compact model format (binary format). This should ideally be done in the original libsvm repo, but I think a pull request there is unlikely to be merged.

[stropitek]

Use pre-computed kernel. It's even faster when performing cross validation