search

getalp / disambiguate

Disambiguate is a tool for training and using state of the art neural WSD models
https://arxiv.org/abs/1905.05677
MIT License
58 stars 17 forks source link

Some issues regarding the execution of scripts #9

Open JamilProg opened 4 years ago

JamilProg commented 4 years ago

Hello everyone! :)

I am currently facing issues which are :

1) I can't launch the evaluate.sh script because of a threading error. I tried my best to find the source of this error, but was unsuccessful. A screenshot showing this error is made.

2) About decode.sh and train.sh, the process doesn't stop correctly, which makes the thing not debuggable. A screenshot showing this error is made.

3) About prepare.sh, I see a progress bar. Once it is complete, the process doesn't stop, which forces me to press Ctrl+C. Is this normal?

I am currently using my personal computer whose configuration is : GPU = NVIDIA GeForce GTX960M CPU = IntelCore i7 6700HQ 2.60GHz OS = Linux (Manjaro) from Dual Boot

Do you have any suggestion or advice?

Thank you very much for your help.

Best regards, Jamil

evaluate_error

decode_error

schwabdidier commented 4 years ago

Did you follow steps mentionned in readme.md ?

loic-vial commented 4 years ago

Hi, thank you for using our code !

We do not have enough information to help you with your problems.

Please provide us the following:

Thanks !

JamilProg commented 4 years ago

Before starting anything, I have installed : Python 3.6.2, Java (openjdk 10), Maven, PyTorch (working with CUDA, checked), and the model packages (Allen Bert & Transformers). I've also downloaded the repository disambiguate, the repository UFSAC, and the link of the data (DATADIR).

Then, I compiled using mvn in the UFSAC repo, then I launched compile.sh from disambiguate repo :

[jamilz@jamil-pc disambiguate-master]$ ./java/compile.sh [INFO] Scanning for projects... [INFO] [INFO] ------------------------< getalp:disambiguate >------------------------- [INFO] Building disambiguate 1.0-SNAPSHOT [INFO] --------------------------------[ jar ]--------------------------------- Downloading from central: http://central.maven.org/maven2/getalp/ufsac/1.0-SNAPSHOT/maven-metadata.xml [WARNING] Could not transfer metadata getalp:ufsac:1.0-SNAPSHOT/maven-metadata.xml from/to central (http://central.maven.org/maven2/): Transfer failed for http://central.maven.org/maven2/getalp/ufsac/1.0-SNAPSHOT/maven-metadata.xml [WARNING] Failure to transfer getalp:ufsac:1.0-SNAPSHOT/maven-metadata.xml from http://central.maven.org/maven2/ was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced. Original error: Could not transfer metadata getalp:ufsac:1.0-SNAPSHOT/maven-metadata.xml from/to central (http://central.maven.org/maven2/): Transfer failed for http://central.maven.org/maven2/getalp/ufsac/1.0-SNAPSHOT/maven-metadata.xml Downloading from central: http://central.maven.org/maven2/nz/ac/waikato/cms/weka/weka-dev/maven-metadata.xml Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/nz/ac/waikato/cms/weka/weka-dev/maven-metadata.xml [WARNING] Could not transfer metadata nz.ac.waikato.cms.weka:weka-dev/maven-metadata.xml from/to central (http://central.maven.org/maven2/): Transfer failed for http://central.maven.org/maven2/nz/ac/waikato/cms/weka/weka-dev/maven-metadata.xml [WARNING] Could not transfer metadata nz.ac.waikato.cms.weka:weka-dev/maven-metadata.xml from/to sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots): Transfer failed for https://oss.sonatype.org/content/repositories/snapshots/nz/ac/waikato/cms/weka/weka-dev/maven-metadata.xml Downloading from sonatype-nexus-snapshots: https://oss.sonatype.org/content/repositories/snapshots/de/bwaldvogel/liblinear/maven-metadata.xml Downloading from central: http://central.maven.org/maven2/de/bwaldvogel/liblinear/maven-metadata.xml [WARNING] Could not transfer metadata de.bwaldvogel:liblinear/maven-metadata.xml from/to central (http://central.maven.org/maven2/): Transfer failed for http://central.maven.org/maven2/de/bwaldvogel/liblinear/maven-metadata.xml [WARNING] Could not transfer metadata de.bwaldvogel:liblinear/maven-metadata.xml from/to sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots): Transfer failed for https://oss.sonatype.org/content/repositories/snapshots/de/bwaldvogel/liblinear/maven-metadata.xml [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ disambiguate --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /home/jamilz/Téléchargements/disambiguate-master/java/src/main/resources [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ disambiguate --- [INFO] Nothing to compile - all classes are up to date [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 3.265 s [INFO] Finished at: 2020-01-18T16:22:07+01:00 [INFO] ------------------------------------------------------------------------

Then, I tried to run decode.sh this way : [jamilz@jamil-pc disambiguate-master]$ ./decode.sh --data_path ../data/ --weights ../data/model_weights_wsd0 Either, the output is empty, either the output is the following (in both case, the process never stop) :
ligne 2 : 4472 Processus arrêté PYTHONPATH="$(dirname "$0")":$PYTHONPATH python3 -u -m "$@"

In the case of evaluate.sh, I ran the following command and I got this output (I have tried to change the corpus parameter but it didn't change anything) : [jamilz@jamil-pc disambiguate-master]$ ./evaluate.sh --data_path ../data/ --weights ../data/model_weights_wsd0 --corpus ../ufsac-public-2.1/semcor.xml Arguments: --help (default value) = false --python_path = ./python --data_path = ../data/ --weights = [../data/model_weights_wsd0] --corpus = [../ufsac-public-2.1/semcor.xml] --lowercase (default value) = false --sense_compression_hypernyms (default value) = true --sense_compression_instance_hypernyms (default value) = false --sense_compression_antonyms (default value) = false --sense_compression_file (default value) = --filter_lemma (default value) = true --clear_text (default value) = false --batch_size (default value) = 1

------ Evaluate the score of an ensemble of models

Evaluate on corpus ../ufsac-public-2.1/semcor.xml Evaluate without backoff (br-a01) [97,73] (br-a02) [98,46] (br-a11) [99,23] (br-a12) [99,63] (br-a13) [99,42] (br-a14) [98,67] (br-a15) [98,72] (br-b13) [98,87] (br-b20) [97,70] (br-c01) [99,03] (br-c02) [97,67] (br-c04) [99,40] (br-d01) [98,66] (br-d02) [96,78] (br-d03) [97,86] (br-d04) [97,79] (br-e01) [98,12] (br-e02) [99,34] (br-e04) [99,36] (br-e21) [97,64] (br-e24) [98,33] (br-e29) [98,31] (br-f03) [98,83] (br-f10) [99,02] (br-f19) [98,10] (br-f43) [98,46] (br-g01) [98,94] (br-g11) [97,49] (br-g15) [97,14] (br-h01) [98,88] (br-j01) [98,98] (br-j02) [98,16] (br-j03) [98,44] (br-j04) [97,24] (br-j05) [99,23] (br-j06) [98,44] (br-j07) [99,07] (br-j08) [98,17] (br-j09) [99,47] (br-j10) [99,24] (br-j11) [98,88] (br-j12) [99,36] (br-j13) [98,57] (br-j14) [98,47] (br-j15) [99,24] (br-j16) [98,40] (br-j17) [98,74] (br-j18) [98,27] (br-j19) [98,02] (br-j20) [98,95] (br-j22) [97,17] (br-j23) [98,68] (br-j37) [98,28] (br-j52) [98,07] (br-j53) [98,70] (br-j54) [98,48] (br-j55) [98,64] (br-j56) [98,30] (br-j57) [97,42] (br-j58) [98,76] (br-j59) [97,58] (br-j60) [98,88] (br-j70) [99,23] (br-k01) [98,11] (br-k02) [98,64] (br-k03) [98,18] (br-k04) [98,60] (br-k05) [97,72] (br-k06) [98,60] (br-k07) [97,96] (br-k08) [98,16] (br-k09) [98,44] (br-k10) [97,48] (br-k11) [98,76] (br-k12) [99,53] (br-k13) [98,02] (br-k14) [98,55] (br-k15) [98,81] (br-k16) [99,15] (br-k17) [99,10] (br-k18) [98,11] (br-k19) [98,29] (br-k20) [97,93] (br-k21) [98,97] (br-k22) [97,98] (br-k23) [99,42] (br-k24) [99,10] (br-k25) [98,83] (br-k26) [97,91] (br-k27) [99,31] (br-k28) [98,03] (br-k29) [97,81] (br-l11) [98,64] (br-l12) [98,71] (br-m01) [98,22] (br-m02) [98,56] (br-n05) [98,92] (br-p01) [98,44] (br-r05) [98,27] (br-r06) [98,31] (br-r07) [99,09] (br-r08) [97,83] (br-r09) [98,03] (br-e22) [97,24] (br-e23) [97,67] (br-e25) [98,79] (br-e26) [97,59] (br-e27) [99,43] (br-e28) [98,04] (br-e30) [98,80] (br-e31) [98,54] (br-f08) [99,00] (br-f13) [98,86] (br-f14) [97,31] (br-f15) [98,64] (br-f16) [98,86] (br-f17) [98,51] (br-f18) [99,30] (br-f20) [98,90] (br-f21) [98,93] (br-f22) [97,83] (br-f23) [98,45] (br-f24) [97,79] (br-f25) [98,55] (br-f33) [99,21] (br-f44) [99,08] (br-g12) [97,48] (br-g14) [97,85] (br-g16) [97,78] (br-g17) [98,08] (br-g18) [96,20] (br-g19) [98,75] (br-g20) [96,57] (br-g21) [96,65] (br-g22) [97,76] (br-g23) [98,50] (br-g28) [99,08] (br-g31) [97,76] (br-g39) [98,78] (br-g43) [96,23] (br-g44) [98,61] (br-h09) [96,42] (br-h11) [98,11] (br-h12) [99,01] (br-h13) [96,59] (br-h14) [98,94] (br-h15) [98,57] (br-h16) [97,17] (br-h17) [98,56] (br-h18) [97,35] (br-h21) [97,63] (br-h24) [99,47] (br-j29) [98,98] (br-j30) [97,18] (br-j31) [96,66] (br-j32) [99,20] (br-j33) [97,83] (br-j34) [98,35] (br-j35) [98,18] (br-j38) [98,87] (br-j41) [98,53] (br-j42) [97,47] (br-l08) [97,36] (br-l09) [98,51] (br-l10) [97,98] (br-l13) [98,28] (br-l14) [98,30] (br-l15) [98,36] (br-l16) [97,99] (br-l17) [97,75] (br-l18) [97,78] (br-n09) [98,27] (br-n10) [98,97] (br-n11) [98,56] (br-n12) [98,25] (br-n14) [98,69] (br-n15) [98,52] (br-n16) [97,88] (br-n17) [98,81] (br-n20) [98,58] (br-p07) [98,31] (br-p09) [95,49] (br-p10) [97,06] (br-p12) [98,67] (br-p24) [98,26] (br-r04) [98,66] (br-a03) [98,65] (br-a04) [96,28] (br-a05) [97,42] (br-a06) [97,00] (br-a07) [95,56] (br-a08) [95,13] (br-a09) [97,71] (br-a10) [96,65] (br-a16) [98,95] (br-a17) [95,11] (br-a18) [98,27] (br-a19) [95,93] (br-a20) [97,49] (br-a21) [99,16] (br-a22) [97,59] (br-a23) [96,39] (br-a24) [98,63] (br-a25) [94,69] (br-a26) [95,67] (br-a27) [94,04] (br-a28) [96,48] (br-a29) [96,60] (br-a30) [94,74] (br-a31) [94,50] (br-a32) [99,07] (br-a33) [97,41] (br-a34) [94,95] (br-a35) [95,05] (br-a36) [97,07] (br-a37) [98,74] (br-a38) [96,65] (br-a39) [96,59] (br-a40) [95,47] (br-a41) [95,78] (br-a42) [98,33] (br-a43) [96,97] (br-a44) [95,09] (br-b01) [99,57] (br-b02) [97,89] (br-b03) [97,13] (br-b04) [97,93] (br-b05) [94,27] (br-b06) [96,88] (br-b07) [96,70] (br-b08) [96,76] (br-b09) [98,88] (br-b10) [95,86] (br-b11) [95,81] (br-b12) [96,05] (br-b14) [98,08] (br-b15) [96,50] (br-b16) [94,25] (br-b17) [95,52] (br-b18) [95,77] (br-b19) [97,70] (br-b21) [96,63] (br-b22) [95,13] (br-b23) [98,42] (br-b24) [98,54] (br-b25) [97,13] (br-b26) [96,17] (br-b27) [98,43] (br-c03) [96,46] (br-c05) [96,19] (br-c06) [99,08] (br-c07) [95,19] (br-c08) [97,39] (br-c09) [95,89] (br-c10) [96,22] (br-c11) [96,97] (br-c12) [95,59] (br-c13) [98,37] (br-c14) [95,61] (br-c15) [97,70] (br-c16) [96,24] (br-c17) [97,20] (br-d05) [95,97] (br-d06) [94,68] (br-d07) [95,86] (br-d08) [94,57] (br-d09) [97,40] (br-d10) [98,83] (br-d11) [94,85] (br-d12) [97,04] (br-d13) [97,55] (br-d14) [96,17] (br-d15) [96,22] (br-d16) [97,32] (br-d17) [98,31] (br-e03) [98,13] (br-e05) [95,15] (br-e06) [96,36] (br-e07) [98,18] (br-e08) [97,54] (br-e09) [95,31] (br-e10) [96,24] (br-e11) [98,77] (br-e12) [93,01] (br-e13) [98,44] (br-e14) [98,92] (br-e15) [97,43] (br-e16) [98,07] (br-e17) [97,12] (br-e18) [95,73] (br-e19) [97,45] (br-e20) [95,74] (br-f01) [95,98] (br-f02) [96,12] (br-f04) [95,85] (br-f05) [97,79] (br-f06) [95,86] (br-f07) [99,59] (br-f09) [98,65] (br-f11) [96,06] (br-f12) [97,45] (br-g02) [95,39] (br-g03) [97,37] (br-g04) [96,19] (br-g05) [96,28] (br-g06) [98,08] (br-g07) [98,65] (br-g08) [98,60] (br-g09) [97,53] (br-g10) [99,23] (br-g13) [97,51] (br-h02) [96,30] (br-h03) [97,92] (br-h04) [98,03] (br-h05) [97,63] (br-h06) [94,77] (br-h07) [97,30] (br-h08) [94,65] (br-h10) [98,54] (br-j21) [96,43] (br-j24) [97,33] (br-j25) [96,72] (br-j26) [98,21] (br-j27) [97,56] (br-j28) [98,14] (br-l01) [96,84] (br-l02) [97,19] (br-l03) [96,11] (br-l04) [96,76] (br-l05) [97,16] (br-l06) [96,40] (br-l07) [96,99] (br-m03) [97,89] (br-m04) [96,72] (br-m05) [98,53] (br-m06) [97,19] (br-n01) [97,06] (br-n02) [97,98] (br-n03) [98,07] (br-n04) [95,87] (br-n06) [96,99] (br-n07) [95,98] (br-n08) [95,60] (br-p02) [93,11] (br-p03) [98,68] (br-p04) [97,49] (br-p05) [96,86] (br-p06) [97,32] (br-p08) [97,09] (br-r01) [98,15] (br-r02) [96,17] (br-r03) [95,85] ; good/bad/missed/total : 225117/4410/6/229533 ; C/P/R/F1 : 99,9974/98,0787/98,0761/98,0774 ; [n] good/bad/missed/total : 86387/1191/3/87581 ; [n] C/P/R/F1 : 99,9966/98,6401/98,6367/98,6384 ; [v] good/bad/missed/total : 86321/2730/0/89051 ; [v] C/P/R/F1 : 100,0000/96,9343/96,9343/96,9343 ; [a] good/bad/missed/total : 33436/313/3/33752 ; [a] C/P/R/F1 : 99,9911/99,0726/99,0638/99,0682 ; [r] good/bad/missed/total : 18973/176/0/19149 ; [r] C/P/R/F1 : 100,0000/99,0809/99,0809/99,0809 ; [x] good/bad/missed/total : 0/0/0/0 ; [x] C/P/R/F1 : NaN/NaN/NaN/NaN ; time : 10053.583 seconds Evaluate with monosemics (br-a01) [97,73] (br-a02) [98,46] (br-a11) [99,23] (br-a12) [99,63] (br-a13) [99,42] (br-a14) [98,67] (br-a15) [98,72] (br-b13) [98,87] (br-b20) [97,70] (br-c01) [99,03] (br-c02) [97,67] (br-c04) [99,40] (br-d01) [98,66] (br-d02) [96,78] (br-d03) [97,86] (br-d04) [97,79] (br-e01) [98,12] (br-e02) [99,34] (br-e04) [99,36] (br-e21) [97,64] (br-e24) [98,33] (br-e29) [98,31] (br-f03) [98,83] (br-f10) [99,02] (br-f19) [98,10] (br-f43) [98,46] (br-g01) [98,94] (br-g11) [97,49] (br-g15) [97,14] (br-h01) [98,88] (br-j01) [98,98] (br-j02) [98,16] (br-j03) [98,44] (br-j04) [97,29] (br-j05) [99,23] (br-j06) [98,49] (br-j07) [99,07] (br-j08) [98,17] (br-j09) [99,47] (br-j10) [99,24] (br-j11) [98,88] (br-j12) [99,36] (br-j13) [98,57] (br-j14) [98,47] (br-j15) [99,24] (br-j16) [98,40] (br-j17) [98,74] (br-j18) [98,27] (br-j19) [98,02] (br-j20) [98,95] (br-j22) [97,17] (br-j23) [98,68] (br-j37) [98,28] (br-j52) [98,07] (br-j53) [98,70] (br-j54) [98,48] (br-j55) [98,64] (br-j56) [98,30] (br-j57) [97,42] (br-j58) [98,76] (br-j59) [97,58] (br-j60) [98,88] (br-j70) [99,23] (br-k01) [98,11] (br-k02) [98,64] (br-k03) [98,18] (br-k04) [98,60] (br-k05) [97,72] (br-k06) [98,60] (br-k07) [97,96] (br-k08) [98,16] (br-k09) [98,44] (br-k10) [97,48] (br-k11) [98,76] (br-k12) [99,53] (br-k13) [98,02] (br-k14) [98,60] (br-k15) [98,81] (br-k16) [99,15] (br-k17) [99,10] (br-k18) [98,11] (br-k19) [98,29] (br-k20) [97,93] (br-k21) [98,97] (br-k22) [97,98] (br-k23) [99,42] (br-k24) [99,10] (br-k25) [98,83] (br-k26) [97,91] (br-k27) [99,31] (br-k28) [98,03] (br-k29) [97,81] (br-l11) [98,64] (br-l12) [98,71] (br-m01) [98,22] (br-m02) [98,56] (br-n05) [98,92] (br-p01) [98,44] (br-r05) [98,27] (br-r06) [98,31] (br-r07) [99,09] (br-r08) [97,83] (br-r09) [98,03] (br-e22) [97,24] (br-e23) [97,67] (br-e25) [98,79] (br-e26) [97,59] (br-e27) [99,43] (br-e28) [98,04] (br-e30) [98,80] (br-e31) [98,54] (br-f08) [99,00] (br-f13) [98,86] (br-f14) [97,31] (br-f15) [98,64] (br-f16) [98,86] (br-f17) [98,51] (br-f18) [99,30] (br-f20) [98,90] (br-f21) [98,93] (br-f22) [97,83] (br-f23) [98,45] (br-f24) [97,79] (br-f25) [98,55] (br-f33) [99,21] (br-f44) [99,08] (br-g12) [97,48] (br-g14) [97,85] (br-g16) [97,78] (br-g17) [98,08] (br-g18) [96,20] (br-g19) [98,80] (br-g20) [96,57] (br-g21) [96,65] (br-g22) [97,76] (br-g23) [98,50] (br-g28) [99,08] (br-g31) [97,76] (br-g39) [98,78] (br-g43) [96,23] (br-g44) [98,61] (br-h09) [96,42] (br-h11) [98,11] (br-h12) [99,01] (br-h13) [96,59] (br-h14) [98,94] (br-h15) [98,57] (br-h16) [97,17] (br-h17) [98,56] (br-h18) [97,35] (br-h21) [97,63] (br-h24) [99,47] (br-j29) [98,98] (br-j30) [97,18] (br-j31) [96,66] (br-j32) [99,20] (br-j33) [97,83] (br-j34) [98,35] (br-j35) [98,18] (br-j38) [98,87] (br-j41) [98,53] (br-j42) [97,47] (br-l08) [97,36] (br-l09) [98,51] (br-l10) [97,98] (br-l13) [98,28] (br-l14) [98,30] (br-l15) [98,36] (br-l16) [97,99] (br-l17) [97,75] (br-l18) [97,78] (br-n09) [98,27] (br-n10) [98,97] (br-n11) [98,56] (br-n12) [98,25] (br-n14) [98,69] (br-n15) [98,52] (br-n16) [97,88] (br-n17) [98,81] (br-n20) [98,58] (br-p07) [98,31] (br-p09) [95,49] (br-p10) [97,06] (br-p12) [98,67] (br-p24) [98,26] (br-r04) [98,66] (br-a03) [98,65] (br-a04) [96,28] (br-a05) [97,42] (br-a06) [97,00] (br-a07) [95,56] (br-a08) [95,13] (br-a09) [97,71] (br-a10) [96,65] (br-a16) [98,95] (br-a17) [95,11] (br-a18) [98,27] (br-a19) [95,93] (br-a20) [97,49] (br-a21) [99,16] (br-a22) [97,59] (br-a23) [96,39] (br-a24) [98,63] (br-a25) [94,69] (br-a26) [95,67] (br-a27) [94,04] (br-a28) [96,48] (br-a29) [96,60] (br-a30) [94,74] (br-a31) [94,50] (br-a32) [99,07] (br-a33) [97,41] (br-a34) [94,95] (br-a35) [95,05] (br-a36) [97,07] (br-a37) [98,74] (br-a38) [96,65] (br-a39) [96,59] (br-a40) [95,47] (br-a41) [95,78] (br-a42) [98,33] (br-a43) [96,97] (br-a44) [95,09] (br-b01) [99,57] (br-b02) [97,89] (br-b03) [97,13] (br-b04) [97,93] (br-b05) [94,27] (br-b06) [96,88] (br-b07) [96,70] (br-b08) [96,76] (br-b09) [98,88] (br-b10) [95,86] (br-b11) [95,81] (br-b12) [96,05] (br-b14) [98,08] (br-b15) [96,50] (br-b16) [94,25] (br-b17) [95,52] (br-b18) [95,77] (br-b19) [97,70] (br-b21) [96,63] (br-b22) [95,13] (br-b23) [98,42] (br-b24) [98,54] (br-b25) [97,13] (br-b26) [96,17] (br-b27) [98,43] (br-c03) [96,46] (br-c05) [96,19] (br-c06) [99,08] (br-c07) [95,19] (br-c08) [97,39] (br-c09) [95,89] (br-c10) [96,22] (br-c11) [96,97] (br-c12) [95,59] (br-c13) [98,37] (br-c14) [95,61] (br-c15) [97,70] (br-c16) [96,24] (br-c17) [97,20] (br-d05) [95,97] (br-d06) [94,68] (br-d07) [95,86] (br-d08) [94,57] (br-d09) [97,40] (br-d10) [98,83] (br-d11) [94,85] (br-d12) [97,04] (br-d13) [97,55] (br-d14) [96,17] (br-d15) [96,22] (br-d16) [97,32] (br-d17) [98,31] (br-e03) [98,13] (br-e05) [95,15] (br-e06) [96,36] (br-e07) [98,18] (br-e08) [97,54] (br-e09) [95,31] (br-e10) [96,24] (br-e11) [98,77] (br-e12) [93,01] (br-e13) [98,44] (br-e14) [98,92] (br-e15) [97,43] (br-e16) [98,07] (br-e17) [97,12] (br-e18) [95,73] (br-e19) [97,45] (br-e20) [95,74] (br-f01) [95,98] (br-f02) [96,12] (br-f04) [95,85] (br-f05) [97,79] (br-f06) [95,86] (br-f07) [99,59] (br-f09) [98,65] (br-f11) [96,06] (br-f12) [97,45] (br-g02) [95,39] (br-g03) [97,37] (br-g04) [96,19] (br-g05) [96,28] (br-g06) [98,08] (br-g07) [98,65] (br-g08) [98,60] (br-g09) [97,53] (br-g10) [99,23] (br-g13) [97,51] (br-h02) [96,30] (br-h03) [97,92] (br-h04) [98,03] (br-h05) [97,63] (br-h06) [94,77] (br-h07) [97,30] (br-h08) [94,65] (br-h10) [98,54] (br-j21) [96,43] (br-j24) [97,33] (br-j25) [96,72] (br-j26) [98,21] (br-j27) [97,56] (br-j28) [98,14] (br-l01) [96,84] (br-l02) [97,19] (br-l03) [96,11] (br-l04) [96,76] (br-l05) [97,16] (br-l06) [96,40] (br-l07) [96,99] (br-m03) [97,89] (br-m04) [96,72] (br-m05) [98,53] (br-m06) [97,19] (br-n01) [97,06] (br-n02) [97,98] (br-n03) [98,07] (br-n04) [95,87] (br-n06) [96,99] (br-n07) [95,98] (br-n08) [95,60] (br-p02) [93,11] (br-p03) [98,68] (br-p04) [97,49] (br-p05) [96,86] (br-p06) [97,32] (br-p08) [97,09] (br-r01) [98,15] (br-r02) [96,17] (br-r03) [95,85] ; good/bad/missed/total : 225121/4410/2/229533 ; C/P/R/F1 : 99,9991/98,0787/98,0778/98,0783 ; [n] good/bad/missed/total : 86388/1191/2/87581 ; [n] C/P/R/F1 : 99,9977/98,6401/98,6378/98,6390 ; [v] good/bad/missed/total : 86321/2730/0/89051 ; [v] C/P/R/F1 : 100,0000/96,9343/96,9343/96,9343 ; [a] good/bad/missed/total : 33439/313/0/33752 ; [a] C/P/R/F1 : 100,0000/99,0726/99,0726/99,0726 ; [r] good/bad/missed/total : 18973/176/0/19149 ; [r] C/P/R/F1 : 100,0000/99,0809/99,0809/99,0809 ; [x] good/bad/missed/total : 0/0/0/0 ; [x] C/P/R/F1 : NaN/NaN/NaN/NaN ; time : 0.651 seconds Evaluate with backoff first sense (br-a01) [97,73] (br-a02) [98,51] (br-a11) [99,23] (br-a12) [99,63] (br-a13) [99,42] (br-a14) [98,67] (br-a15) [98,72] (br-b13) [98,87] (br-b20) [97,70] (br-c01) [99,03] (br-c02) [97,67] (br-c04) [99,40] (br-d01) [98,66] (br-d02) [96,78] (br-d03) [97,86] (br-d04) [97,79] (br-e01) [98,12] (br-e02) [99,34] (br-e04) [99,36] (br-e21) [97,64] (br-e24) [98,33] (br-e29) [98,31] (br-f03) [98,83] (br-f10) [99,02] (br-f19) [98,10] (br-f43) [98,46] (br-g01) [98,94] (br-g11) [97,49] (br-g15) [97,14] (br-h01) [98,88] (br-j01) [98,98] (br-j02) [98,16] (br-j03) [98,44] (br-j04) [97,29] (br-j05) [99,23] (br-j06) [98,49] (br-j07) [99,07] (br-j08) [98,17] (br-j09) [99,47] (br-j10) [99,24] (br-j11) [98,88] (br-j12) [99,36] (br-j13) [98,57] (br-j14) [98,47] (br-j15) [99,24] (br-j16) [98,40] (br-j17) [98,74] (br-j18) [98,27] (br-j19) [98,02] (br-j20) [98,95] (br-j22) [97,17] (br-j23) [98,68] (br-j37) [98,28] (br-j52) [98,07] (br-j53) [98,70] (br-j54) [98,48] (br-j55) [98,64] (br-j56) [98,30] (br-j57) [97,42] (br-j58) [98,76] (br-j59) [97,58] (br-j60) [98,88] (br-j70) [99,23] (br-k01) [98,11] (br-k02) [98,64] (br-k03) [98,18] (br-k04) [98,60] (br-k05) [97,72] (br-k06) [98,60] (br-k07) [97,96] (br-k08) [98,16] (br-k09) [98,44] (br-k10) [97,48] (br-k11) [98,76] (br-k12) [99,53] (br-k13) [98,02] (br-k14) [98,60] (br-k15) [98,81] (br-k16) [99,15] (br-k17) [99,10] (br-k18) [98,11] (br-k19) [98,29] (br-k20) [97,93] (br-k21) [98,97] (br-k22) [97,98] (br-k23) [99,42] (br-k24) [99,10] (br-k25) [98,83] (br-k26) [97,91] (br-k27) [99,31] (br-k28) [98,03] (br-k29) [97,81] (br-l11) [98,64] (br-l12) [98,71] (br-m01) [98,22] (br-m02) [98,56] (br-n05) [98,92] (br-p01) [98,44] (br-r05) [98,27] (br-r06) [98,31] (br-r07) [99,09] (br-r08) [97,83] (br-r09) [98,03] (br-e22) [97,24] (br-e23) [97,67] (br-e25) [98,79] (br-e26) [97,59] (br-e27) [99,43] (br-e28) [98,04] (br-e30) [98,80] (br-e31) [98,54] (br-f08) [99,00] (br-f13) [98,86] (br-f14) [97,31] (br-f15) [98,64] (br-f16) [98,86] (br-f17) [98,51] (br-f18) [99,30] (br-f20) [98,90] (br-f21) [98,93] (br-f22) [97,83] (br-f23) [98,45] (br-f24) [97,79] (br-f25) [98,55] (br-f33) [99,21] (br-f44) [99,08] (br-g12) [97,54] (br-g14) [97,85] (br-g16) [97,78] (br-g17) [98,08] (br-g18) [96,20] (br-g19) [98,80] (br-g20) [96,57] (br-g21) [96,65] (br-g22) [97,76] (br-g23) [98,50] (br-g28) [99,08] (br-g31) [97,76] (br-g39) [98,78] (br-g43) [96,23] (br-g44) [98,61] (br-h09) [96,42] (br-h11) [98,11] (br-h12) [99,01] (br-h13) [96,59] (br-h14) [98,94] (br-h15) [98,57] (br-h16) [97,17] (br-h17) [98,56] (br-h18) [97,35] (br-h21) [97,63] (br-h24) [99,47] (br-j29) [98,98] (br-j30) [97,18] (br-j31) [96,66] (br-j32) [99,20] (br-j33) [97,83] (br-j34) [98,35] (br-j35) [98,18] (br-j38) [98,87] (br-j41) [98,53] (br-j42) [97,47] (br-l08) [97,36] (br-l09) [98,51] (br-l10) [97,98] (br-l13) [98,28] (br-l14) [98,30] (br-l15) [98,36] (br-l16) [97,99] (br-l17) [97,75] (br-l18) [97,78] (br-n09) [98,27] (br-n10) [98,97] (br-n11) [98,56] (br-n12) [98,25] (br-n14) [98,69] (br-n15) [98,52] (br-n16) [97,88] (br-n17) [98,81] (br-n20) [98,58] (br-p07) [98,31] (br-p09) [95,49] (br-p10) [97,06] (br-p12) [98,67] (br-p24) [98,26] (br-r04) [98,66] (br-a03) [98,65] (br-a04) [96,28] (br-a05) [97,42] (br-a06) [97,00] (br-a07) [95,56] (br-a08) [95,13] (br-a09) [97,71] (br-a10) [96,65] (br-a16) [98,95] (br-a17) [95,11] (br-a18) [98,27] (br-a19) [95,93] (br-a20) [97,49] (br-a21) [99,16] (br-a22) [97,59] (br-a23) [96,39] (br-a24) [98,63] (br-a25) [94,69] (br-a26) [95,67] (br-a27) [94,04] (br-a28) [96,48] (br-a29) [96,60] (br-a30) [94,74] (br-a31) [94,50] (br-a32) [99,07] (br-a33) [97,41] (br-a34) [94,95] (br-a35) [95,05] (br-a36) [97,07] (br-a37) [98,74] (br-a38) [96,65] (br-a39) [96,59] (br-a40) [95,47] (br-a41) [95,78] (br-a42) [98,33] (br-a43) [96,97] (br-a44) [95,09] (br-b01) [99,57] (br-b02) [97,89] (br-b03) [97,13] (br-b04) [97,93] (br-b05) [94,27] (br-b06) [96,88] (br-b07) [96,70] (br-b08) [96,76] (br-b09) [98,88] (br-b10) [95,86] (br-b11) [95,81] (br-b12) [96,05] (br-b14) [98,08] (br-b15) [96,50] (br-b16) [94,25] (br-b17) [95,52] (br-b18) [95,77] (br-b19) [97,70] (br-b21) [96,63] (br-b22) [95,13] (br-b23) [98,42] (br-b24) [98,54] (br-b25) [97,13] (br-b26) [96,17] (br-b27) [98,43] (br-c03) [96,46] (br-c05) [96,19] (br-c06) [99,08] (br-c07) [95,19] (br-c08) [97,39] (br-c09) [95,89] (br-c10) [96,22] (br-c11) [96,97] (br-c12) [95,59] (br-c13) [98,37] (br-c14) [95,61] (br-c15) [97,70] (br-c16) [96,24] (br-c17) [97,20] (br-d05) [95,97] (br-d06) [94,68] (br-d07) [95,86] (br-d08) [94,57] (br-d09) [97,40] (br-d10) [98,83] (br-d11) [94,85] (br-d12) [97,04] (br-d13) [97,55] (br-d14) [96,17] (br-d15) [96,22] (br-d16) [97,32] (br-d17) [98,31] (br-e03) [98,13] (br-e05) [95,15] (br-e06) [96,36] (br-e07) [98,18] (br-e08) [97,54] (br-e09) [95,31] (br-e10) [96,24] (br-e11) [98,77] (br-e12) [93,01] (br-e13) [98,44] (br-e14) [98,92] (br-e15) [97,43] (br-e16) [98,07] (br-e17) [97,12] (br-e18) [95,73] (br-e19) [97,45] (br-e20) [95,74] (br-f01) [95,98] (br-f02) [96,12] (br-f04) [95,85] (br-f05) [97,79] (br-f06) [95,86] (br-f07) [99,59] (br-f09) [98,65] (br-f11) [96,06] (br-f12) [97,45] (br-g02) [95,39] (br-g03) [97,37] (br-g04) [96,19] (br-g05) [96,28] (br-g06) [98,08] (br-g07) [98,65] (br-g08) [98,60] (br-g09) [97,53] (br-g10) [99,23] (br-g13) [97,51] (br-h02) [96,30] (br-h03) [97,92] (br-h04) [98,03] (br-h05) [97,63] (br-h06) [94,77] (br-h07) [97,30] (br-h08) [94,65] (br-h10) [98,54] (br-j21) [96,43] (br-j24) [97,33] (br-j25) [96,72] (br-j26) [98,21] (br-j27) [97,56] (br-j28) [98,14] (br-l01) [96,84] (br-l02) [97,19] (br-l03) [96,11] (br-l04) [96,76] (br-l05) [97,16] (br-l06) [96,40] (br-l07) [96,99] (br-m03) [97,89] (br-m04) [96,72] (br-m05) [98,53] (br-m06) [97,19] (br-n01) [97,06] (br-n02) [97,98] (br-n03) [98,07] (br-n04) [95,87] (br-n06) [96,99] (br-n07) [95,98] (br-n08) [95,60] (br-p02) [93,11] (br-p03) [98,68] (br-p04) [97,49] (br-p05) [96,86] (br-p06) [97,32] (br-p08) [97,09] (br-r01) [98,15] (br-r02) [96,17] (br-r03) [95,85] ; good/bad/missed/total : 225123/4410/0/229533 ; C/P/R/F1 : 100,0000/98,0787/98,0787/98,0787 ; [n] good/bad/missed/total : 86390/1191/0/87581 ; [n] C/P/R/F1 : 100,0000/98,6401/98,6401/98,6401 ; [v] good/bad/missed/total : 86321/2730/0/89051 ; [v] C/P/R/F1 : 100,0000/96,9343/96,9343/96,9343 ; [a] good/bad/missed/total : 33439/313/0/33752 ; [a] C/P/R/F1 : 100,0000/99,0726/99,0726/99,0726 ; [r] good/bad/missed/total : 18973/176/0/19149 ; [r] C/P/R/F1 : 100,0000/99,0809/99,0809/99,0809 ; [x] good/bad/missed/total : 0/0/0/0 ; [x] C/P/R/F1 : NaN/NaN/NaN/NaN ; time : 0.567 seconds

------ Evaluate the scores of individual models

Evaluate on corpus ../ufsac-public-2.1/semcor.xml 0 Evaluate without backoff (br-a01) ./python/launch.sh : ligne 2 : 5209 Processus arrêté PYTHONPATH="$(dirname "$0")":$PYTHONPATH python3 -u -m "$@" [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project disambiguate: An exception occured while executing the Java class. java.lang.NullPointerException -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project disambiguate: An exception occured while executing the Java class. java.lang.NullPointerException at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289) at org.apache.maven.cli.MavenCli.main (MavenCli.java:193) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:564) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347) Caused by: org.apache.maven.plugin.MojoExecutionException: An exception occured while executing the Java class. java.lang.NullPointerException at org.codehaus.mojo.exec.ExecJavaMojo.execute (ExecJavaMojo.java:339) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289) at org.apache.maven.cli.MavenCli.main (MavenCli.java:193) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:564) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347) Caused by: java.lang.RuntimeException: java.lang.NullPointerException at getalp.wsd.method.neural.NeuralDisambiguator.disambiguateFixedSentenceBatch (NeuralDisambiguator.java:519) at getalp.wsd.method.DisambiguatorContextSentenceBatch.disambiguateDynamicSentenceBatch (DisambiguatorContextSentenceBatch.java:54) at getalp.wsd.method.DisambiguatorContextSentenceBatch.disambiguate (DisambiguatorContextSentenceBatch.java:27) at getalp.wsd.method.Disambiguator.disambiguate (Disambiguator.java:31) at getalp.wsd.evaluation.WSDEvaluator.evaluate (WSDEvaluator.java:74) at NeuralWSDTest.evaluate_mean_scores (NeuralWSDTest.java:152) at NeuralWSDTest.test (NeuralWSDTest.java:106) at NeuralWSDTest.main (NeuralWSDTest.java:178) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:564) at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282) at java.lang.Thread.run (Thread.java:844) Caused by: java.lang.NullPointerException at getalp.wsd.method.neural.NeuralDisambiguator.readPredictOutput (NeuralDisambiguator.java:354) at getalp.wsd.method.neural.NeuralDisambiguator.disambiguateNoCatch (NeuralDisambiguator.java:507) at getalp.wsd.method.neural.NeuralDisambiguator.disambiguateFixedSentenceBatch (NeuralDisambiguator.java:515) at getalp.wsd.method.DisambiguatorContextSentenceBatch.disambiguateDynamicSentenceBatch (DisambiguatorContextSentenceBatch.java:54) at getalp.wsd.method.DisambiguatorContextSentenceBatch.disambiguate (DisambiguatorContextSentenceBatch.java:27) at getalp.wsd.method.Disambiguator.disambiguate (Disambiguator.java:31) at getalp.wsd.evaluation.WSDEvaluator.evaluate (WSDEvaluator.java:74) at NeuralWSDTest.evaluate_mean_scores (NeuralWSDTest.java:152) at NeuralWSDTest.test (NeuralWSDTest.java:106) at NeuralWSDTest.main (NeuralWSDTest.java:178) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:564) at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282) at java.lang.Thread.run (Thread.java:844) [ERROR] [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

PS : Sorry for the late answer, and thank you very much for both replies! :) EDIT : Removed the "code highlight" for the last error to make it more readable.

loic-vial commented 4 years ago

Hi,

First, about your issue on the decode script that "does nothing" , it expects some text from the standard input, did you provide some ?

About your second error, this is really weird... It seems that the Python process launched by the Java code is killed during the evaluation/decode process....

Can you run the following command: ./launch.sh getalp.wsd.predict --data_path [DATA_PATH] --weights [WEIGHTS] --disambiguate in the python folder, and with your paths for the data and the weights ? Then enter some text and see if there is an error. This is the Python script that is called inside the Java code.

JamilProg commented 4 years ago

Hello,

1) Indeed, this one is my bad, I forgot to put something in the standard input. It seems like I have the same problem with decode.sh :

[jamilz@jamil-pc disambiguate-master]$ cat foo My mouse is completely broken.

[jamilz@jamil-pc disambiguate-master]$ ./decode.sh --data_path ../data/ --weights ../data/model_weights_wsd0 < foo ./python/launch.sh : ligne 2 : 2009 Processus arrêté PYTHONPATH="$(dirname "$0")":$PYTHONPATH python3 -u -m "$@" [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project disambiguate: An exception occured while executing the Java class. java.lang.NullPointerException -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project disambiguate: An exception occured while executing the Java class. java.lang.NullPointerException at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289) at org.apache.maven.cli.MavenCli.main (MavenCli.java:193) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:564) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347) Caused by: org.apache.maven.plugin.MojoExecutionException: An exception occured while executing the Java class. java.lang.NullPointerException at org.codehaus.mojo.exec.ExecJavaMojo.execute (ExecJavaMojo.java:339) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289) at org.apache.maven.cli.MavenCli.main (MavenCli.java:193) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:564) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347) Caused by: java.lang.RuntimeException: java.lang.NullPointerException at getalp.wsd.method.neural.NeuralDisambiguator.disambiguateFixedSentenceBatch (NeuralDisambiguator.java:519) at getalp.wsd.method.DisambiguatorContextSentenceBatch.disambiguateDynamicSentenceBatch (DisambiguatorContextSentenceBatch.java:64) at NeuralWSDDecode.decodeSentenceBatch (NeuralWSDDecode.java:114) at NeuralWSDDecode.decode (NeuralWSDDecode.java:102) at NeuralWSDDecode.main (NeuralWSDDecode.java:21) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:564) at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282) at java.lang.Thread.run (Thread.java:844) Caused by: java.lang.NullPointerException at getalp.wsd.method.neural.NeuralDisambiguator.readPredictOutput (NeuralDisambiguator.java:354) at getalp.wsd.method.neural.NeuralDisambiguator.disambiguateNoCatch (NeuralDisambiguator.java:507) at getalp.wsd.method.neural.NeuralDisambiguator.disambiguateFixedSentenceBatch (NeuralDisambiguator.java:515) at getalp.wsd.method.DisambiguatorContextSentenceBatch.disambiguateDynamicSentenceBatch (DisambiguatorContextSentenceBatch.java:64) at NeuralWSDDecode.decodeSentenceBatch (NeuralWSDDecode.java:114) at NeuralWSDDecode.decode (NeuralWSDDecode.java:102) at NeuralWSDDecode.main (NeuralWSDDecode.java:21) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:564) at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282) at java.lang.Thread.run (Thread.java:844) [ERROR] [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

2) About the command to run from the python folder, here is the output :

[jamilz@jamil-pc python]$ ./launch.sh getalp.wsd.predict --data_path ../../data/ --weights ../../data/model_weights_wsd0 --disambiguate ./launch.sh : ligne 2 : 2773 Processus arrêté PYTHONPATH="$(dirname "$0")":$PYTHONPATH python3 -u -m "$@"

It seems like, as you said, the Python process keep getting killed in every case, I am currently trying to figure out why it does that.

Andry-Bal commented 4 years ago

Indeed, this problem takes place, since I have an exactly same output as in previous comment when running decode.sh script.

Andry-Bal commented 4 years ago

But when I run the script from python directory (as suggested above), I get the following output:

(env) andry-bal@andrybal-VirtualBox:~/Documents/NLP/disambiguate/python$ ./launch.sh getalp.wsd.predict --data_path ../../data --weights ../../data/model_weights_wsd0 --disambiguate Traceback (most recent call last): File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/usr/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/andry-bal/Documents/NLP/disambiguate/python/getalp/wsd/predict.py", line 29, in main() File "/home/andry-bal/Documents/NLP/disambiguate/python/getalp/wsd/predict.py", line 25, in main predicter.predict() File "/home/andry-bal/Documents/NLP/disambiguate/python/getalp/wsd/predicter.py", line 35, in predict ensemble = self.create_ensemble(config, self.ensemble_weights_path) File "/home/andry-bal/Documents/NLP/disambiguate/python/getalp/wsd/predicter.py", line 72, in create_ensemble ensemble[i].load_model_weights(ensemble_weights_paths[i]) File "/home/andry-bal/Documents/NLP/disambiguate/python/getalp/wsd/model.py", line 55, in load_model_weights self.create_model() File "/home/andry-bal/Documents/NLP/disambiguate/python/getalp/wsd/model.py", line 26, in create_model self.backend = TorchModel(self.config, self.config.data_config) File "/home/andry-bal/Documents/NLP/disambiguate/python/getalp/wsd/model.py", line 235, in init module = get_bert_embeddings(bert_path=config.input_bert_path[i], clear_text=data_config.input_clear_text[i]) File "/home/andry-bal/Documents/NLP/disambiguate/python/getalp/wsd/modules/embeddings/embeddings_bert.py", line 88, in get_bert_embeddings _bert_embeddings_wrapper[bert_path] = EmbeddingsBert(bert_path) File "/home/andry-bal/Documents/NLP/disambiguate/python/getalp/wsd/modules/embeddings/embeddings_bert.py", line 13, in init self.bert_embeddings = BertModel.from_pretrained(bert_path) File "/home/andry-bal/Documents/NLP/env/lib/python3.6/site-packages/pytorch_pretrained_bert/modeling.py", line 590, in from_pretrained archive.extractall(tempdir) File "/usr/lib/python3.6/tarfile.py", line 2010, in extractall numeric_owner=numeric_owner) File "/usr/lib/python3.6/tarfile.py", line 2052, in extract numeric_owner=numeric_owner) File "/usr/lib/python3.6/tarfile.py", line 2122, in _extract_member self.makefile(tarinfo, targetpath) File "/usr/lib/python3.6/tarfile.py", line 2171, in makefile copyfileobj(source, target, tarinfo.size, ReadError, bufsize) File "/usr/lib/python3.6/tarfile.py", line 249, in copyfileobj buf = src.read(bufsize) File "/usr/lib/python3.6/gzip.py", line 276, in read return self._buffer.read(size) File "/usr/lib/python3.6/_compression.py", line 68, in readinto data = self.read(len(byte_view)) File "/usr/lib/python3.6/gzip.py", line 482, in read raise EOFError("Compressed file ended before the " EOFError: Compressed file ended before the end-of-stream marker was reached

loic-vial commented 4 years ago

@Andry-Bal Thank you, it gives more informations for debugging :)

So it seems that the "weights" file is somewhat corrupted or something like that. Is your file located at ../../data/model_weights_wsd0 the pretrained weights that we provide ? Or something that you trained by yourself ? If this is the pretrained model, are you sure the download completed well ? If this is a model you trained yourself, are you sure this path point to the weights file, and the training process went well ?

JamilProg commented 4 years ago

The file located in ../../data/model_weights_wsd0 is the pretrained weights that you provide.

I re-downloaded it in order to check if it is because of file corruption. It didn't change anything.

In my opinion, it might be because of my computer because, as the process stops working in the middle of execution, it can be a problem with memory. I don't see what kind of other problem would kill a process like that.

loic-vial commented 4 years ago

Oh, yes you're right, it's probably a lack of memory, because the provided model is really large.

My advice is that if you use your personal computer, you should train a new model by yourself with a smaller BERT model (like Albert or Distilbert).

I'll keep the issue open to remind me to catch the error and try to display a more explicit message.

Andry-Bal commented 4 years ago

@coylz I have doubts, that this is a memory issue. I monitored RAM and disk during script execution and did not see anything suspicious (however I might be mistaken). However, I finally made it work. So, before I was running on Ubuntu 18.04.2 with Python 3.6.9, Java 11 (openjdk 11.0.5), Apache Maven 3.6.0 and had this errors reported above. Then I tried on Ubuntu 18.04.3 with Python 3.6.9, Java 8 (openjdk 1.8.0_232), Apache Maven 3.6.3 and surprisingly it worked like a charm without any problems. Maybe some dependencies compatibility problem.

schwabdidier commented 4 years ago

@Andry-Bal Thank you for comment, it will help both us and other users.

JamilProg commented 4 years ago

Thank you very much @coylz @Andry-Bal for the info & tips !

Basically, I have changed the version of Java & Maven (I took the same versions as @Andry-Bal). The output of decode.sh & evaluate.sh are the same, BUT the launch.sh from the python folder gives this output :

[jamilz@jamil-pc python]$ ./launch.sh getalp.wsd.predict --data_path ../../data/ --weights ../../data/model_weights_wsd0 --disambiguate Traceback (most recent call last): File "/home/jamilz/.pyenv/versions/3.6.2/lib/python3.6/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/home/jamilz/.pyenv/versions/3.6.2/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/jamilz/Téléchargements/disambiguate-master/python/getalp/wsd/predict.py", line 29, in main() File "/home/jamilz/Téléchargements/disambiguate-master/python/getalp/wsd/predict.py", line 25, in main predicter.predict() File "/home/jamilz/Téléchargements/disambiguate-master/python/getalp/wsd/predicter.py", line 35, in predict ensemble = self.create_ensemble(config, self.ensemble_weights_path) File "/home/jamilz/Téléchargements/disambiguate-master/python/getalp/wsd/predicter.py", line 72, in create_ensemble ensemble[i].load_model_weights(ensemble_weights_paths[i]) File "/home/jamilz/Téléchargements/disambiguate-master/python/getalp/wsd/model.py", line 55, in load_model_weights self.create_model() File "/home/jamilz/Téléchargements/disambiguate-master/python/getalp/wsd/model.py", line 26, in create_model self.backend = TorchModel(self.config, self.config.data_config) File "/home/jamilz/Téléchargements/disambiguate-master/python/getalp/wsd/model.py", line 263, in init self.cuda() File "/home/jamilz/.pyenv/versions/3.6.2/lib/python3.6/site-packages/torch/nn/modules/module.py", line 305, in cuda return self._apply(lambda t: t.cuda(device)) File "/home/jamilz/.pyenv/versions/3.6.2/lib/python3.6/site-packages/torch/nn/modules/module.py", line 202, in _apply module._apply(fn) File "/home/jamilz/.pyenv/versions/3.6.2/lib/python3.6/site-packages/torch/nn/modules/module.py", line 202, in _apply module._apply(fn) File "/home/jamilz/.pyenv/versions/3.6.2/lib/python3.6/site-packages/torch/nn/modules/module.py", line 202, in _apply module._apply(fn) [Previous line repeated 4 more times] File "/home/jamilz/.pyenv/versions/3.6.2/lib/python3.6/site-packages/torch/nn/modules/module.py", line 224, in _apply param_applied = fn(param) File "/home/jamilz/.pyenv/versions/3.6.2/lib/python3.6/site-packages/torch/nn/modules/module.py", line 305, in return self._apply(lambda t: t.cuda(device)) RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 1.96 GiB total capacity; 1.53 GiB already allocated; 17.25 MiB free; 7.58 MiB cached) [jamilz@jamil-pc python]$

So, it seems like I had both memory and dependencies compatibility problem. I think this is a fixable problem, but I don't have enough of time to go further (I have exams soon, and the deadline of this practical is tomorrow). I am sorry about the practical @schwabdidier , I really wanted to finish it.

Later after the exams, I might try using @coylz 's advices (training a little model) in order to help future users who will face this problem.