search

clab / dynet

DyNet: The Dynamic Neural Network Toolkit
Apache License 2.0
3.42k stars 703 forks source link

double free or corruption (fasttop) #1254

Open miguelballesteros opened 6 years ago

miguelballesteros commented 6 years ago

I translated the RNNG code (https://github.com/clab/rnng) into dynet 2.0.3 (rnng code was written in dynet's old version: cnn), just changing Parameter to Parameters... and similar stuff.

It trains perfectly fine and it decodes also perfectly fine, providing a F1 score that replicates what we had in the paper. Everything seems to work fine but when I decode I get the following error, after finishing and obtaining the output:

Test output in /tmp/parser_test_eval.25868.txt F1score: 89.1 Error in `./nt-parser': double free or corruption (fasttop): 0x0000000001a46220 ======= Backtrace: ========= /lib64/libc.so.6(+0x7c619)[0x2b853d97e619] /lib64/libstdc++.so.6(_ZNSsD1Ev+0x43)[0x2b853d19fe43] /lib64/libc.so.6(__cxa_finalize+0x9a)[0x2b853d93adda] /opt/share/boost-1.57.0/lib/libboost_program_options.so.1.57.0(+0x2a716)[0x2b8535793716]

This is the discriminative parser.

I tried with 2 different boost versions 1.57.0 and 1.59.0, same issue with all of them. In RNNG we used boost for stream processing and I still do in this local RNNG version. This means that I changed the code to use the newest Dynet saving/loading parameters routines that get rid of boost, but not for the rest of boost operations used in the RNNG code.

Is this a Dynet issue? or should I look somewhere else? any ideas?

Thank you, Miguel

neubig commented 6 years ago

Two things:

  1. if you can share code and an example command line argument with us it'd be easier for us to debug.
  2. could you run in a debugger, either Valgrind on Linux or lldb on Mac and post the output?
miguelballesteros commented 6 years ago

thanks! there you go: https://github.com/miguelballesteros/RNNG-Dynet2

In order to reproduce the error, just train a model for a couple of minutes, until it saves the first parameter set:

./nt-parser --dynet-mem 1700 -x -T [train.oracle] -d [dev.oracle] -C [PTB/dev.24] -P -t --pretrained_dim 100 -w [sskip.100.vectors] --lstm_input_dim 128 --hidden_dim 128 -D 0.2

and then, decode with that model:

./nt-parser --dynet-mem 1700 -x -T [train.oracle] -p [dev.oracle] -C [PTB/dev.24] -P --pretrained_dim 100 -w [sskip.100.vectors] --lstm_input_dim 128 --hidden_dim 128 -m [params-file] > output.txt

This should output the F1 score and then crash.

Thanks! Miguel

neubig commented 6 years ago

Thanks, we'll take a look.