search

arjunmenon / smalltext

Classify short texts with neural network.
MIT License
16 stars 1 forks source link

How long it takes to train? #4

Closed kokizzu closed 3 years ago

kokizzu commented 3 years ago

I used around 500k input, 23 minutes haven't completed yet '__')

kokizzu commented 3 years ago

also would --jit flag helps?

kokizzu commented 3 years ago

2 hours and haven't completed yet, using 40GB of RAM (swapping), the CPU now gone down to 1% seems like no more process but it stuck at training..

kokizzu commented 3 years ago

and segfault XD

/var/lib/gems/2.7.0/gems/porter2stemmer-1.0.1/lib/porter2stemmer/implementation.rb:29: [BUG] Segmentation fault at 0x0000000000000000
ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-linux-gnu]

-- Control frame information -----------------------------------------------
c:0012 p:---- s:0070 e:000069 CFUNC  :gsub!
c:0011 p:0013 s:0064 e:000063 METHOD /var/lib/gems/2.7.0/gems/porter2stemmer-1.0.1/lib/porter2stemmer/implementation.rb:29
c:0010 p:0026 s:0059 e:000058 METHOD /var/lib/gems/2.7.0/gems/porter2stemmer-1.0.1/lib/porter2stemmer/implementation.rb:273
c:0009 p:0005 s:0051 e:000050 BLOCK  /var/lib/gems/2.7.0/gems/smalltext-0.1.0/lib/smalltext.rb:134 [FINISH]
c:0008 p:---- s:0047 e:000046 CFUNC  :map!
c:0007 p:0017 s:0043 e:000042 BLOCK  /var/lib/gems/2.7.0/gems/smalltext-0.1.0/lib/smalltext.rb:134 [FINISH]
c:0006 p:---- s:0036 e:000035 CFUNC  :each
c:0005 p:0026 s:0032 e:000031 METHOD /var/lib/gems/2.7.0/gems/smalltext-0.1.0/lib/smalltext.rb:128
c:0004 p:0039 s:0027 e:000026 METHOD /var/lib/gems/2.7.0/gems/smalltext-0.1.0/lib/smalltext.rb:120
c:0003 p:0023 s:0023 e:000022 METHOD /var/lib/gems/2.7.0/gems/smalltext-0.1.0/lib/smalltext.rb:39
c:0002 p:0148 s:0010 E:0016a0 EVAL   classify.rb:38 [FINISH]
c:0001 p:0000 s:0003 E:0014b0 (none) [FINISH]

-- Ruby level backtrace information ----------------------------------------
classify.rb:38:in `<main>'
/var/lib/gems/2.7.0/gems/smalltext-0.1.0/lib/smalltext.rb:39:in `train'
/var/lib/gems/2.7.0/gems/smalltext-0.1.0/lib/smalltext.rb:120:in `preprocess'
/var/lib/gems/2.7.0/gems/smalltext-0.1.0/lib/smalltext.rb:128:in `prepare_bow'
/var/lib/gems/2.7.0/gems/smalltext-0.1.0/lib/smalltext.rb:128:in `each'
/var/lib/gems/2.7.0/gems/smalltext-0.1.0/lib/smalltext.rb:134:in `block in prepare_bow'
/var/lib/gems/2.7.0/gems/smalltext-0.1.0/lib/smalltext.rb:134:in `map!'
/var/lib/gems/2.7.0/gems/smalltext-0.1.0/lib/smalltext.rb:134:in `block (2 levels) in prepare_bow'
/var/lib/gems/2.7.0/gems/porter2stemmer-1.0.1/lib/porter2stemmer/implementation.rb:273:in `porter2_stem'
/var/lib/gems/2.7.0/gems/porter2stemmer-1.0.1/lib/porter2stemmer/implementation.rb:29:in `porter2_preprocess'
/var/lib/gems/2.7.0/gems/porter2stemmer-1.0.1/lib/porter2stemmer/implementation.rb:29:in `gsub!'

-- Machine register context ------------------------------------------------
 RIP: 0x00007f3609800ed5 RBP: 0x698e3b812bf1c530 RSP: 0x00007ffe82593180
 RAX: 0x0000000000000000 RBX: 0x0000564d39c8e020 RCX: 0xcccccccccccccccd
 RDX: 0x698e3b812bf1c530 RDI: 0x000056454b97e170 RSI: 0x698e3b812bf1c530
  R8: 0x000056454b9ce690  R9: 0x0000000000000060 R10: 0x000056454b97c010
 R11: 0x00007f360973abe0 R12: 0x000056454b97e170 R13: 0x698e3b812bf1c000
 R14: 0x0000564d39c68b00 R15: 0x000056454b97e238 EFL: 0x0000000000010206

-- C level backtrace information -------------------------------------------
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f360996cf49) [0x7f360996cf49]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f360996d17c) [0x7f360996d17c]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f36097e641b) [0x7f36097e641b]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f36098e9f3d) [0x7f36098e9f3d]
/lib/x86_64-linux-gnu/libc.so.6(0x7f3609595210) [0x7f3609595210]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f3609800ed5) [0x7f3609800ed5]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f3609801085) [0x7f3609801085]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f360980153a) [0x7f360980153a]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f360980549c) [0x7f360980549c]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f36098060ba) [0x7f36098060ba]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f3609806739) [0x7f3609806739]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f3609806813) [0x7f3609806813]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f3609806df6) [0x7f3609806df6]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f36098c239a) [0x7f36098c239a]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f36098c605f) [0x7f36098c605f]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f360990d962) [0x7f360990d962]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f360994ff86) [0x7f360994ff86]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f36099430d6) [0x7f36099430d6]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f360995401a) [0x7f360995401a]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(rb_vm_exec+0x1d0) [0x7f360995a710]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(rb_yield+0x271) [0x7f3609967891]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f36097760b0) [0x7f36097760b0]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f360994ff86) [0x7f360994ff86]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f36099430d6) [0x7f36099430d6]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f3609954091) [0x7f3609954091]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(rb_vm_exec+0x1d0) [0x7f360995a710]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(rb_yield+0x271) [0x7f3609967891]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(rb_ary_each+0x3c) [0x7f360976e55c]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f360994ff86) [0x7f360994ff86]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f360995cc4b) [0x7f360995cc4b]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f36099430d6) [0x7f36099430d6]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f3609954091) [0x7f3609954091]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(rb_vm_exec+0x1d0) [0x7f360995a710]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(0x7f36097ea661) [0x7f36097ea661]
/usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7(ruby_run_node+0x4e) [0x7f36097ef4de]
ruby(0x56454b8a717f) [0x56454b8a717f]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7f36095760b3] ../csu/libc-start.c:308
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main) (null):0
ruby(_start+0x2e) [0x56454b8a71ce]

-- Other runtime information -----------------------------------------------

* Loaded script: classify.rb

* Loaded features:

    0 enumerator.so
    1 thread.rb
    2 rational.so
    3 complex.so
    4 ruby2_keywords.rb
    5 /usr/lib/x86_64-linux-gnu/ruby/2.7.0/enc/encdb.so
    6 /usr/lib/x86_64-linux-gnu/ruby/2.7.0/enc/trans/transdb.so
    7 /usr/lib/x86_64-linux-gnu/ruby/2.7.0/rbconfig.rb
    8 /usr/lib/ruby/2.7.0/rubygems/compatibility.rb
    9 /usr/lib/ruby/2.7.0/rubygems/defaults.rb
   10 /usr/lib/ruby/2.7.0/rubygems/deprecate.rb
   11 /usr/lib/ruby/2.7.0/rubygems/errors.rb
   12 /usr/lib/ruby/2.7.0/rubygems/version.rb
   13 /usr/lib/ruby/2.7.0/rubygems/requirement.rb
   14 /usr/lib/ruby/2.7.0/rubygems/platform.rb
   15 /usr/lib/ruby/2.7.0/rubygems/basic_specification.rb
   16 /usr/lib/ruby/2.7.0/rubygems/stub_specification.rb
   17 /usr/lib/ruby/2.7.0/rubygems/util.rb
   18 /usr/lib/ruby/2.7.0/rubygems/text.rb
   19 /usr/lib/ruby/2.7.0/rubygems/user_interaction.rb
   20 /usr/lib/ruby/2.7.0/rubygems/specification_policy.rb
   21 /usr/lib/ruby/2.7.0/rubygems/util/list.rb
   22 /usr/lib/ruby/2.7.0/rubygems/specification.rb
   23 /usr/lib/ruby/2.7.0/rubygems/exceptions.rb
   24 /usr/lib/ruby/vendor_ruby/rubygems/defaults/operating_system.rb
   25 /usr/lib/ruby/2.7.0/rubygems/bundler_version_finder.rb
   26 /usr/lib/ruby/2.7.0/rubygems/dependency.rb
   27 /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_gem.rb
   28 /usr/lib/x86_64-linux-gnu/ruby/2.7.0/monitor.so
   29 /usr/lib/ruby/2.7.0/monitor.rb
   30 /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb
   31 /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_warn.rb
   32 /usr/lib/ruby/2.7.0/rubygems.rb
   33 /usr/lib/ruby/2.7.0/rubygems/path_support.rb
   34 /usr/lib/ruby/2.7.0/did_you_mean/version.rb
   35 /usr/lib/ruby/2.7.0/did_you_mean/core_ext/name_error.rb
   36 /usr/lib/ruby/2.7.0/did_you_mean/levenshtein.rb
   37 /usr/lib/ruby/2.7.0/did_you_mean/jaro_winkler.rb
   38 /usr/lib/ruby/2.7.0/did_you_mean/spell_checker.rb
   39 /usr/lib/ruby/2.7.0/did_you_mean/spell_checkers/name_error_checkers/class_name_checker.rb
   40 /usr/lib/ruby/2.7.0/did_you_mean/spell_checkers/name_error_checkers/variable_name_checker.rb
   41 /usr/lib/ruby/2.7.0/did_you_mean/spell_checkers/name_error_checkers.rb
   42 /usr/lib/ruby/2.7.0/did_you_mean/spell_checkers/method_name_checker.rb
   43 /usr/lib/ruby/2.7.0/did_you_mean/spell_checkers/key_error_checker.rb
   44 /usr/lib/ruby/2.7.0/did_you_mean/spell_checkers/null_checker.rb
   45 /usr/lib/ruby/2.7.0/did_you_mean/formatters/plain_formatter.rb
   46 /usr/lib/ruby/2.7.0/did_you_mean/tree_spell_checker.rb
   47 /usr/lib/ruby/2.7.0/did_you_mean.rb
   48 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/core_ext/awesome_method_array.rb
   49 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/core_ext/string.rb
   50 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/core_ext/method.rb
   51 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/core_ext/object.rb
   52 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/core_ext/class.rb
   53 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/core_ext/kernel.rb
   54 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/custom_defaults.rb
   55 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/indentator.rb
   56 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/inspector.rb
   57 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/colorize.rb
   58 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/formatters/base_formatter.rb
   59 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/formatters/object_formatter.rb
   60 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/formatters/struct_formatter.rb
   61 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/formatters/hash_formatter.rb
   62 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/formatters/array_formatter.rb
   63 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/formatters/simple_formatter.rb
   64 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/formatters/method_formatter.rb
   65 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/formatters/class_formatter.rb
   66 /usr/lib/ruby/2.7.0/shellwords.rb
   67 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/formatters/dir_formatter.rb
   68 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/formatters/file_formatter.rb
   69 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/formatters.rb
   70 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/formatter.rb
   71 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print/version.rb
   72 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/awesome_print.rb
   73 /var/lib/gems/2.7.0/gems/awesome_print-1.9.2/lib/ap.rb
   74 /usr/lib/ruby/2.7.0/forwardable/impl.rb
   75 /usr/lib/ruby/2.7.0/forwardable/version.rb
   76 /usr/lib/ruby/2.7.0/forwardable.rb
   77 /usr/lib/ruby/2.7.0/English.rb
   78 /usr/lib/x86_64-linux-gnu/ruby/2.7.0/date_core.so
   79 /usr/lib/ruby/2.7.0/date.rb
   80 /usr/lib/x86_64-linux-gnu/ruby/2.7.0/stringio.so
   81 /usr/lib/ruby/2.7.0/csv/fields_converter.rb
   82 /usr/lib/ruby/2.7.0/csv/match_p.rb
   83 /usr/lib/x86_64-linux-gnu/ruby/2.7.0/strscan.so
   84 /usr/lib/ruby/2.7.0/csv/delete_suffix.rb
   85 /usr/lib/ruby/2.7.0/csv/row.rb
   86 /usr/lib/ruby/2.7.0/csv/table.rb
   87 /usr/lib/ruby/2.7.0/csv/parser.rb
   88 /usr/lib/ruby/2.7.0/csv/writer.rb
   89 /usr/lib/ruby/2.7.0/csv/version.rb
   90 /usr/lib/ruby/2.7.0/csv/core_ext/array.rb
   91 /usr/lib/ruby/2.7.0/csv/core_ext/string.rb
   92 /usr/lib/ruby/2.7.0/csv.rb
   93 /usr/lib/ruby/2.7.0/time.rb
   94 /usr/lib/x86_64-linux-gnu/ruby/2.7.0/pathname.so
   95 /usr/lib/ruby/2.7.0/pathname.rb
   96 /usr/lib/ruby/2.7.0/bundler/version.rb
   97 /usr/lib/ruby/2.7.0/bundler/constants.rb
   98 /usr/lib/ruby/2.7.0/bundler/rubygems_integration.rb
   99 /usr/lib/ruby/2.7.0/bundler/current_ruby.rb
  100 /usr/lib/ruby/2.7.0/bundler/shared_helpers.rb
  101 /usr/lib/ruby/2.7.0/rubygems/ext/builder.rb
  102 /usr/lib/ruby/2.7.0/bundler/setup.rb
  103 /var/lib/gems/2.7.0/gems/porter2stemmer-1.0.1/lib/porter2stemmer/constants.rb
  104 /var/lib/gems/2.7.0/gems/porter2stemmer-1.0.1/lib/porter2stemmer/implementation.rb
  105 /var/lib/gems/2.7.0/gems/porter2stemmer-1.0.1/lib/porter2stemmer.rb
  106 /var/lib/gems/2.7.0/gems/tokenizer-0.3.0/lib/tokenizer/tokenizer.rb
  107 /var/lib/gems/2.7.0/gems/tokenizer-0.3.0/lib/tokenizer/version.rb
  108 /var/lib/gems/2.7.0/gems/tokenizer-0.3.0/lib/tokenizer.rb
  109 /var/lib/gems/2.7.0/gems/numo-narray-0.9.1.9/lib/numo/narray.so
  110 /var/lib/gems/2.7.0/gems/numo-narray-0.9.1.9/lib/numo/narray/extra.rb
  111 /var/lib/gems/2.7.0/gems/numo-narray-0.9.1.9/lib/numo/narray.rb
  112 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/version.rb
  113 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/cli/trollop.rb
  114 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/exceptions.rb
  115 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/distribution.rb
  116 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/distributions.rb
  117 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/distributions/triangular.rb
  118 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/distributions/credit_card.rb
  119 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/distributions/binomial.rb
  120 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/distributions/cauchy.rb
  121 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/distributions/degenerate.rb
  122 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/distributions/uniform.rb
  123 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/distributions/exponential.rb
  124 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/distributions/normal.rb
  125 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/distributions/gamma.rb
  126 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/distributions/nbinomial.rb
  127 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/distributions/poisson.rb
  128 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/distributions/bernoulli.rb
  129 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/distributions/geometric.rb
  130 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier/cli/application.rb
  131 /var/lib/gems/2.7.0/gems/croupier-1.6.0/lib/croupier.rb
  132 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/comparable.rb
  133 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/compressible.rb
  134 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/compressor.rb
  135 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/configuration/properties.rb
  136 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/configuration/provider_collection.rb
  137 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/configuration.rb
  138 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/container.rb
  139 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/enumerable.rb
  140 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/inspectable.rb
  141 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/invalid_operation.rb
  142 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/readers/plain_text.rb
  143 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/readers.rb
  144 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/serializers/file.rb
  145 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/serializers/marshal.rb
  146 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/serializers/yaml.rb
  147 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/serializers/zip.rb
  148 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/serializers.rb
  149 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/stringifyable.rb
  150 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/nodes/node.rb
  151 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/nodes/missing.rb
  152 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/nodes/compressed.rb
  153 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/nodes/raw.rb
  154 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/nodes.rb
  155 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie/version.rb
  156 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling/trie.rb
  157 /var/lib/gems/2.7.0/gems/rambling-trie-2.2.0/lib/rambling-trie.rb
  158 /var/lib/gems/2.7.0/gems/smalltext-0.1.0/lib/smalltext/version.rb
  159 /var/lib/gems/2.7.0/gems/smalltext-0.1.0/lib/smalltext.rb

* Process memory map:

56454b8a6000-56454b8a7000 r--p 00000000 08:75 7081184                    /usr/bin/ruby2.7
56454b8a7000-56454b8a8000 r-xp 00001000 08:75 7081184                    /usr/bin/ruby2.7
56454b8a8000-56454b8a9000 r--p 00002000 08:75 7081184                    /usr/bin/ruby2.7
56454b8a9000-56454b8aa000 r--p 00002000 08:75 7081184                    /usr/bin/ruby2.7
56454b8aa000-56454b8ab000 rw-p 00003000 08:75 7081184                    /usr/bin/ruby2.7
56454b97c000-56549011b000 rw-p 00000000 00:00 0                          [heap]
7f36044da000-7f36049c9000 rw-p 00000000 00:00 0 
7f3604b23000-7f3604d13000 r--s 00000000 08:75 2883963                    /lib/x86_64-linux-gnu/libc-2.31.so
7f3604d13000-7f3604f14000 rw-p 00000000 00:00 0 
7f3604f6d000-7f36052a2000 r--s 00000000 08:75 7081213                    /usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7.2
7f36052a2000-7f36055ec000 rw-p 00000000 00:00 0 
7f3605687000-7f360568d000 r--p 00000000 08:75 14435923                   /var/lib/gems/2.7.0/gems/numo-narray-0.9.1.9/lib/numo/narray.so
7f360568d000-7f3605786000 r-xp 00006000 08:75 14435923                   /var/lib/gems/2.7.0/gems/numo-narray-0.9.1.9/lib/numo/narray.so
7f3605786000-7f36057ac000 r--p 000ff000 08:75 14435923                   /var/lib/gems/2.7.0/gems/numo-narray-0.9.1.9/lib/numo/narray.so
7f36057ac000-7f36057ad000 ---p 00125000 08:75 14435923                   /var/lib/gems/2.7.0/gems/numo-narray-0.9.1.9/lib/numo/narray.so
7f36057ad000-7f36057ae000 r--p 00125000 08:75 14435923                   /var/lib/gems/2.7.0/gems/numo-narray-0.9.1.9/lib/numo/narray.so
7f36057ae000-7f36057af000 rw-p 00126000 08:75 14435923                   /var/lib/gems/2.7.0/gems/numo-narray-0.9.1.9/lib/numo/narray.so
7f36057af000-7f36057b1000 rw-p 00000000 00:00 0 
7f36057b1000-7f36057b5000 r--p 00000000 08:75 7494538                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/date_core.so
7f36057b5000-7f36057e0000 r-xp 00004000 08:75 7494538                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/date_core.so
7f36057e0000-7f36057e8000 r--p 0002f000 08:75 7494538                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/date_core.so
7f36057e8000-7f36057e9000 ---p 00037000 08:75 7494538                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/date_core.so
7f36057e9000-7f36057ea000 r--p 00037000 08:75 7494538                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/date_core.so
7f36057ea000-7f36057eb000 rw-p 00038000 08:75 7494538                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/date_core.so
7f36057eb000-7f36057ec000 rw-p 00000000 00:00 0 
7f36057ec000-7f36057ed000 ---p 00000000 00:00 0 
7f36057ed000-7f360588e000 rw-p 00000000 00:00 0 
7f360588e000-7f360588f000 ---p 00000000 00:00 0 
7f360588f000-7f3605930000 rw-p 00000000 00:00 0 
7f3605930000-7f3605931000 ---p 00000000 00:00 0 
7f3605931000-7f36059d2000 rw-p 00000000 00:00 0 
7f36059d2000-7f36059d3000 ---p 00000000 00:00 0 
7f36059d3000-7f3605a74000 rw-p 00000000 00:00 0 
7f3605a74000-7f3605a75000 ---p 00000000 00:00 0 
7f3605a75000-7f3605b16000 rw-p 00000000 00:00 0 
7f3605b16000-7f3605b17000 ---p 00000000 00:00 0 
7f3605b17000-7f3605bb8000 rw-p 00000000 00:00 0 
7f3605bb8000-7f3605bb9000 ---p 00000000 00:00 0 
7f3605bb9000-7f3605c5a000 rw-p 00000000 00:00 0 
7f3605c5a000-7f3605c5b000 ---p 00000000 00:00 0 
7f3605c5b000-7f3605cfc000 rw-p 00000000 00:00 0 
7f3605cfc000-7f3605cfd000 ---p 00000000 00:00 0 
7f3605cfd000-7f3605d9e000 rw-p 00000000 00:00 0 
7f3605d9e000-7f3605d9f000 ---p 00000000 00:00 0 
7f3605d9f000-7f3605e40000 rw-p 00000000 00:00 0 
7f3605e40000-7f3605e41000 ---p 00000000 00:00 0 
7f3605e41000-7f3605ee2000 rw-p 00000000 00:00 0 
7f3605ee2000-7f3605ee3000 ---p 00000000 00:00 0 
7f3605ee3000-7f3605f84000 rw-p 00000000 00:00 0 
7f3605f84000-7f3605f85000 ---p 00000000 00:00 0 
7f3605f85000-7f3606026000 rw-p 00000000 00:00 0 
7f3606026000-7f3606027000 ---p 00000000 00:00 0 
7f3606027000-7f36060c8000 rw-p 00000000 00:00 0 
7f36060c8000-7f36060c9000 ---p 00000000 00:00 0 
7f36060c9000-7f360616a000 rw-p 00000000 00:00 0 
7f360616a000-7f360616b000 ---p 00000000 00:00 0 
7f360616b000-7f360620c000 rw-p 00000000 00:00 0 
7f360620c000-7f360620d000 ---p 00000000 00:00 0 
7f360620d000-7f36062ae000 rw-p 00000000 00:00 0 
7f36062ae000-7f36062af000 ---p 00000000 00:00 0 
7f36062af000-7f3606350000 rw-p 00000000 00:00 0 
7f3606350000-7f3606351000 ---p 00000000 00:00 0 
7f3606351000-7f36063f2000 rw-p 00000000 00:00 0 
7f36063f2000-7f36063f3000 ---p 00000000 00:00 0 
7f36063f3000-7f3606494000 rw-p 00000000 00:00 0 
7f3606494000-7f3606495000 ---p 00000000 00:00 0 
7f3606495000-7f3606536000 rw-p 00000000 00:00 0 
7f3606536000-7f3606537000 ---p 00000000 00:00 0 
7f3606537000-7f36065d8000 rw-p 00000000 00:00 0 
7f36065d8000-7f36065d9000 ---p 00000000 00:00 0 
7f36065d9000-7f360667a000 rw-p 00000000 00:00 0 
7f360667a000-7f360667b000 ---p 00000000 00:00 0 
7f360667b000-7f360671c000 rw-p 00000000 00:00 0 
7f360671c000-7f360671d000 ---p 00000000 00:00 0 
7f360671d000-7f36067be000 rw-p 00000000 00:00 0 
7f36067be000-7f36067bf000 ---p 00000000 00:00 0 
7f36067bf000-7f3606860000 rw-p 00000000 00:00 0 
7f3606860000-7f3606861000 ---p 00000000 00:00 0 
7f3606861000-7f3606902000 rw-p 00000000 00:00 0 
7f3606902000-7f3606903000 ---p 00000000 00:00 0 
7f3606903000-7f36069a4000 rw-p 00000000 00:00 0 
7f36069a4000-7f36069a5000 ---p 00000000 00:00 0 
7f36069a5000-7f3606a46000 rw-p 00000000 00:00 0 
7f3606a46000-7f3606a47000 ---p 00000000 00:00 0 
7f3606a47000-7f3606ae8000 rw-p 00000000 00:00 0 
7f3606ae8000-7f3606ae9000 ---p 00000000 00:00 0 
7f3606ae9000-7f3606b8a000 rw-p 00000000 00:00 0 
7f3606b8a000-7f3606b8b000 ---p 00000000 00:00 0 
7f3606b8b000-7f3608c35000 rw-p 00000000 00:00 0 
7f3608c4b000-7f3608c4e000 r--p 00000000 08:75 2883958                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7f3608c4e000-7f3608c60000 r-xp 00003000 08:75 2883958                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7f3608c60000-7f3608c64000 r--p 00015000 08:75 2883958                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7f3608c64000-7f3608c65000 r--p 00018000 08:75 2883958                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7f3608c65000-7f3608c66000 rw-p 00019000 08:75 2883958                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7f3608c92000-7f3608c96000 r--s 00000000 08:75 7081184                    /usr/bin/ruby2.7
7f3608c96000-7f3608d97000 rw-p 00000000 00:00 0 
7f3608d97000-7f3609308000 r--p 00000000 08:75 7078529                    /usr/lib/locale/locale-archive
7f3609308000-7f360930b000 rw-p 00000000 00:00 0 
7f360930b000-7f360931a000 r--p 00000000 08:75 2884004                    /lib/x86_64-linux-gnu/libm-2.31.so
7f360931a000-7f36093c1000 r-xp 0000f000 08:75 2884004                    /lib/x86_64-linux-gnu/libm-2.31.so
7f36093c1000-7f3609458000 r--p 000b6000 08:75 2884004                    /lib/x86_64-linux-gnu/libm-2.31.so
7f3609458000-7f3609459000 r--p 0014c000 08:75 2884004                    /lib/x86_64-linux-gnu/libm-2.31.so
7f3609459000-7f360945a000 rw-p 0014d000 08:75 2884004                    /lib/x86_64-linux-gnu/libm-2.31.so
7f360945a000-7f360945c000 r--p 00000000 08:75 2883875                    /lib/x86_64-linux-gnu/libcrypt.so.1.1.0
7f360945c000-7f3609471000 r-xp 00002000 08:75 2883875                    /lib/x86_64-linux-gnu/libcrypt.so.1.1.0
7f3609471000-7f360948b000 r--p 00017000 08:75 2883875                    /lib/x86_64-linux-gnu/libcrypt.so.1.1.0
7f360948b000-7f360948c000 r--p 00030000 08:75 2883875                    /lib/x86_64-linux-gnu/libcrypt.so.1.1.0
7f360948c000-7f360948d000 rw-p 00031000 08:75 2883875                    /lib/x86_64-linux-gnu/libcrypt.so.1.1.0
7f360948d000-7f3609497000 rw-p 00000000 00:00 0 
7f3609497000-7f3609498000 r--p 00000000 08:75 2883996                    /lib/x86_64-linux-gnu/libdl-2.31.so
7f3609498000-7f360949a000 r-xp 00001000 08:75 2883996                    /lib/x86_64-linux-gnu/libdl-2.31.so
7f360949a000-7f360949b000 r--p 00003000 08:75 2883996                    /lib/x86_64-linux-gnu/libdl-2.31.so
7f360949b000-7f360949c000 r--p 00003000 08:75 2883996                    /lib/x86_64-linux-gnu/libdl-2.31.so
7f360949c000-7f360949d000 rw-p 00004000 08:75 2883996                    /lib/x86_64-linux-gnu/libdl-2.31.so
7f360949d000-7f36094a7000 r--p 00000000 08:75 7080866                    /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.0
7f36094a7000-7f3609507000 r-xp 0000a000 08:75 7080866                    /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.0
7f3609507000-7f360951e000 r--p 0006a000 08:75 7080866                    /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.0
7f360951e000-7f360951f000 ---p 00081000 08:75 7080866                    /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.0
7f360951f000-7f3609520000 r--p 00081000 08:75 7080866                    /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.0
7f3609520000-7f3609521000 rw-p 00082000 08:75 7080866                    /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.0
7f3609521000-7f3609524000 r--p 00000000 08:75 2924914                    /lib/x86_64-linux-gnu/librt-2.31.so
7f3609524000-7f3609528000 r-xp 00003000 08:75 2924914                    /lib/x86_64-linux-gnu/librt-2.31.so
7f3609528000-7f3609529000 r--p 00007000 08:75 2924914                    /lib/x86_64-linux-gnu/librt-2.31.so
7f3609529000-7f360952a000 ---p 00008000 08:75 2924914                    /lib/x86_64-linux-gnu/librt-2.31.so
7f360952a000-7f360952b000 r--p 00008000 08:75 2924914                    /lib/x86_64-linux-gnu/librt-2.31.so
7f360952b000-7f360952c000 rw-p 00009000 08:75 2924914                    /lib/x86_64-linux-gnu/librt-2.31.so
7f360952c000-7f3609533000 r--p 00000000 08:75 2924912                    /lib/x86_64-linux-gnu/libpthread-2.31.so
7f3609533000-7f3609544000 r-xp 00007000 08:75 2924912                    /lib/x86_64-linux-gnu/libpthread-2.31.so
7f3609544000-7f3609549000 r--p 00018000 08:75 2924912                    /lib/x86_64-linux-gnu/libpthread-2.31.so
7f3609549000-7f360954a000 r--p 0001c000 08:75 2924912                    /lib/x86_64-linux-gnu/libpthread-2.31.so
7f360954a000-7f360954b000 rw-p 0001d000 08:75 2924912                    /lib/x86_64-linux-gnu/libpthread-2.31.so
7f360954b000-7f360954f000 rw-p 00000000 00:00 0 
7f360954f000-7f3609574000 r--p 00000000 08:75 2883963                    /lib/x86_64-linux-gnu/libc-2.31.so
7f3609574000-7f36096ec000 r-xp 00025000 08:75 2883963                    /lib/x86_64-linux-gnu/libc-2.31.so
7f36096ec000-7f3609736000 r--p 0019d000 08:75 2883963                    /lib/x86_64-linux-gnu/libc-2.31.so
7f3609736000-7f3609737000 ---p 001e7000 08:75 2883963                    /lib/x86_64-linux-gnu/libc-2.31.so
7f3609737000-7f360973a000 r--p 001e7000 08:75 2883963                    /lib/x86_64-linux-gnu/libc-2.31.so
7f360973a000-7f360973d000 rw-p 001ea000 08:75 2883963                    /lib/x86_64-linux-gnu/libc-2.31.so
7f360973d000-7f3609741000 rw-p 00000000 00:00 0 
7f3609741000-7f3609767000 r--p 00000000 08:75 7081213                    /usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7.2
7f3609767000-7f3609977000 r-xp 00026000 08:75 7081213                    /usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7.2
7f3609977000-7f3609a6e000 r--p 00236000 08:75 7081213                    /usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7.2
7f3609a6e000-7f3609a6f000 ---p 0032d000 08:75 7081213                    /usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7.2
7f3609a6f000-7f3609a75000 r--p 0032d000 08:75 7081213                    /usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7.2
7f3609a75000-7f3609a76000 rw-p 00333000 08:75 7081213                    /usr/lib/x86_64-linux-gnu/libruby-2.7.so.2.7.2
7f3609a76000-7f3609a92000 rw-p 00000000 00:00 0 
7f3609a93000-7f3609a95000 r--p 00000000 08:75 7494549                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/pathname.so
7f3609a95000-7f3609a9a000 r-xp 00002000 08:75 7494549                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/pathname.so
7f3609a9a000-7f3609a9c000 r--p 00007000 08:75 7494549                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/pathname.so
7f3609a9c000-7f3609a9d000 r--p 00008000 08:75 7494549                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/pathname.so
7f3609a9d000-7f3609a9e000 rw-p 00009000 08:75 7494549                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/pathname.so
7f3609a9e000-7f3609aa0000 r--p 00000000 08:75 7494558                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/strscan.so
7f3609aa0000-7f3609aa3000 r-xp 00002000 08:75 7494558                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/strscan.so
7f3609aa3000-7f3609aa4000 r--p 00005000 08:75 7494558                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/strscan.so
7f3609aa4000-7f3609aa5000 ---p 00006000 08:75 7494558                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/strscan.so
7f3609aa5000-7f3609aa6000 r--p 00006000 08:75 7494558                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/strscan.so
7f3609aa6000-7f3609aa7000 rw-p 00007000 08:75 7494558                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/strscan.so
7f3609aa7000-7f3609aa9000 r--p 00000000 08:75 7494557                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/stringio.so
7f3609aa9000-7f3609aae000 r-xp 00002000 08:75 7494557                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/stringio.so
7f3609aae000-7f3609ab0000 r--p 00007000 08:75 7494557                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/stringio.so
7f3609ab0000-7f3609ab1000 r--p 00008000 08:75 7494557                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/stringio.so
7f3609ab1000-7f3609ab2000 rw-p 00009000 08:75 7494557                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/stringio.so
7f3609ab2000-7f3609ab3000 r--p 00000000 08:75 7494545                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/monitor.so
7f3609ab3000-7f3609ab4000 r-xp 00001000 08:75 7494545                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/monitor.so
7f3609ab4000-7f3609ab5000 r--p 00002000 08:75 7494545                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/monitor.so
7f3609ab5000-7f3609ab6000 r--p 00002000 08:75 7494545                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/monitor.so
7f3609ab6000-7f3609ab7000 rw-p 00003000 08:75 7494545                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/monitor.so
7f3609ab7000-7f3609ab8000 r--p 00000000 08:75 8527559                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/enc/trans/transdb.so
7f3609ab8000-7f3609aba000 r-xp 00001000 08:75 8527559                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/enc/trans/transdb.so
7f3609aba000-7f3609abb000 r--p 00003000 08:75 8527559                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/enc/trans/transdb.so
7f3609abb000-7f3609abc000 r--p 00003000 08:75 8527559                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/enc/trans/transdb.so
7f3609abc000-7f3609abd000 rw-p 00004000 08:75 8527559                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/enc/trans/transdb.so
7f3609abd000-7f3609abe000 r--p 00000000 08:75 8526955                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/enc/encdb.so
7f3609abe000-7f3609abf000 r-xp 00001000 08:75 8526955                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/enc/encdb.so
7f3609abf000-7f3609ac0000 r--p 00002000 08:75 8526955                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/enc/encdb.so
7f3609ac0000-7f3609ac1000 r--p 00002000 08:75 8526955                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/enc/encdb.so
7f3609ac1000-7f3609ac2000 rw-p 00003000 08:75 8526955                    /usr/lib/x86_64-linux-gnu/ruby/2.7.0/enc/encdb.so
7f3609ac2000-7f3609ac3000 r--p 00000000 08:75 2883889                    /lib/x86_64-linux-gnu/ld-2.31.so
7f3609ac3000-7f3609ae6000 r-xp 00001000 08:75 2883889                    /lib/x86_64-linux-gnu/ld-2.31.so
7f3609ae6000-7f3609aee000 r--p 00024000 08:75 2883889                    /lib/x86_64-linux-gnu/ld-2.31.so
7f3609aef000-7f3609af0000 r--p 0002c000 08:75 2883889                    /lib/x86_64-linux-gnu/ld-2.31.so
7f3609af0000-7f3609af1000 rw-p 0002d000 08:75 2883889                    /lib/x86_64-linux-gnu/ld-2.31.so
7f3609af1000-7f3609af2000 rw-p 00000000 00:00 0 
7ffe81d98000-7ffe82597000 rw-p 00000000 00:00 0                          [stack]
7ffe825f5000-7ffe825f8000 r--p 00000000 00:00 0                          [vvar]
7ffe825f8000-7ffe825f9000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0                  [vsyscall]

[1]    1164339 abort (core dumped)  ruby classify.rb
kokizzu commented 3 years ago

image

arjunmenon commented 3 years ago

Hi, use this classifier if you have a small training data. (< 1-2k). This library helps when data is sparse. If you have lot data at hand, I recommend using Facebooks FastText.