search

coli-saar / am-parser

Modular implementation of an AM dependency parser in AllenNLP.
Apache License 2.0
30 stars 10 forks source link

Trouble preprocessing AMR 2.0 #81

Closed sctratz closed 4 years ago

sctratz commented 4 years ago

I am running the following commands but I'm hitting exceptions when trying to preprocess AMR 2.0. Any thoughts?

git clone https://github.com/coli-saar/am-parser.git mkdir -p amr_preprocessed/amr2/corpus cp -r /mnt/corpora/LDC2017T10/abstract_meaning_representation_amr_2.0/data/amrs/split/* amr_preprocessed/amr2/corpus cd am-parser time bash scripts/preprocess-no-baseline.sh -m ../amr_preprocessed/amr2/corpus/

===================================================================

preprocessing log attached preprocessLog.txt

namednil commented 4 years ago

Can you try

time bash scripts/preprocess-no-baseline.sh -m ../amr_preprocessed/amr2/

instead? If that doesn't work, I'll properly look into it tomorrow.

sctratz commented 4 years ago

Ok, that seems to be working (at least so far; it is still running)

namednil commented 4 years ago

Just checking in: has it finished decomposing? If not, can you increase the number of threads and memory (lines 84 and 85)? If I recall correctly, we used 600 or 700 GB of RAM and 50 threads; then it took 24 hours.

sctratz commented 4 years ago

Yes, it finished decomposing. Thanks for checking in.