NaturalNode / natural

general natural language facilities for node
MIT License
10.57k stars 861 forks source link

wordnet invalid query freezes node #455

Open crystalfp opened 6 years ago

crystalfp commented 6 years ago

I know wordnet is still experimental, but maybe the following behavior could help its development Installed natural and wordnet-db (latest as per today), then run the following in node. The synnet is just a random number.

const natural = require("natural")
const wordnet = new natural.WordNet();
wordnet.get(4424419, 'n', function(result) {console.log(result)})
undefined
>

Then the window where I run Node freezes and after ten seconds the following is output:

<--- Last few GCs --->
ll[6500:0000021A46B005A0]    82988 ms: Mark-sweep 1398.8 (1406.8) -> 1398.8 (1406.8) MB, 1629.6 / 0.0 ms  (+ 0.0 ms in 4 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 1677 ms) (average mu = 0.108, current mu = 0.030) all[6500:0000021A46B005A0]    85440 ms: Mark-sweep 1398.8 (1406.8) -> 1398.8 (1406.8) MB, 2302.5 / 0.0 ms  (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 2452 ms) (average mu = 0.081, current mu = 0.061) all

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 00000157483841C1]
Security context: 000000368DE9E6C9 <JSObject>
    1: keys [000000368DE84BA9](this=000000368DE84519 <JSFunction Object (sfi = 00000089EB810301)>,00000145A02BA1E1 <JSArray[69755664]>)
    2: formatValue(aka formatValue) [000000946E118591] [util.js:498] [bytecode=000001CC1C978D49 offset=603](this=00000089EB8022E1 <undefined>,ctx=000000CE37E82469 <Object map = 000001F3E9BCD101>,value=00000145A02BA1E1 <J...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 00007FF6AB896AE5
 2: 00007FF6AB872756
 3: 00007FF6AB873EB0
 4: 00007FF6ABD69D3E
 5: 00007FF6ABD69C73
 6: 00007FF6ABC48BD4
 7: 00007FF6ABC3F6B7
 8: 00007FF6ABC3DC3C
 9: 00007FF6ABC3D0E5
10: 00007FF6ABC46916
11: 00007FF6ABB4D228
12: 00007FF6ABB4C436
13: 00007FF6ABB58663
14: 00007FF6ABADC9D9
15: 00007FF6ABB76116
16: 00007FF6ABB6998A
17: 00007FF6ABB93100
18: 00007FF6ABB9450D
19: 00007FF6ABB94215
20: 00007FF6ABBA8953
21: 00000157483841C1
PS D:\Projects\NLP>
PS D:\Projects\NLP>
PS D:\Projects\NLP>
PS D:\Projects\NLP>
PS D:\Projects\NLP> ls wordnet.get(4424419, 'n', function(result) {console.log(result)}) wordnet.get(4424419, 'n', funct
ion(result) {console.log(result)})
At line:1 char:29
+ ls wordnet.get(4424419, 'n', function(result) {console.log(result)})  ...
+                             ~
Missing expression after ','.
At line:1 char:30
+ ls wordnet.get(4424419, 'n', function(result) {console.log(result)})  ...
+                              ~~~~~~~~
Unexpected token 'function' in expression or statement.
At line:1 char:29
+ ls wordnet.get(4424419, 'n', function(result) {console.log(result)})  ...
+                             ~
Missing closing ')' in expression.
At line:1 char:68
+ ... net.get(4424419, 'n', function(result) {console.log(result)}) wordnet ...
+                                                                 ~
Unexpected token ')' in expression or statement.
At line:1 char:95
+ ... tion(result) {console.log(result)}) wordnet.get(4424419, 'n', functio ...
+                                                                  ~
Missing expression after ','.
At line:1 char:96
+ ... lt) {console.log(result)}) wordnet.get(4424419, 'n', function(result) ...
+                                                          ~~~~~~~~
Unexpected token 'function' in expression or statement.
At line:1 char:95
+ ... tion(result) {console.log(result)}) wordnet.get(4424419, 'n', functio ...
+                                                                  ~
Missing closing ')' in expression.
At line:1 char:134
+ ... )}) wordnet.get(4424419, 'n', function(result) {console.log(result)})
+                                                                         ~
Unexpected token ')' in expression or statement.
    + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : MissingExpressionAfterToken
Hugo-ter-Doest commented 6 years ago

Yes, I get similar results. The implementation is not robust for invalid synset offsets. I tried the same offset with https://github.com/morungos/wordnet which is an improved implementation of natural's Wordnet connector, and that one returns null as a result. Clearly, natural's implementation needs further development.